From c527fd1f14c27855a37f2e8ac5346ce8d940ced2 Mon Sep 17 00:00:00 2001 From: Tudor Florea Date: Thu, 16 Oct 2014 03:05:19 +0200 Subject: initial commit for Enea Linux 4.0-140929 Migrated from the internal git server on the daisy-enea-point-release branch Signed-off-by: Tudor Florea --- documentation/kernel-dev/kernel-dev-examples.xml | 918 +++++++++++++++++++++++ 1 file changed, 918 insertions(+) create mode 100644 documentation/kernel-dev/kernel-dev-examples.xml (limited to 'documentation/kernel-dev/kernel-dev-examples.xml') diff --git a/documentation/kernel-dev/kernel-dev-examples.xml b/documentation/kernel-dev/kernel-dev-examples.xml new file mode 100644 index 0000000000..9d9aef6d06 --- /dev/null +++ b/documentation/kernel-dev/kernel-dev-examples.xml @@ -0,0 +1,918 @@ + %poky; ] > + + + +Working with the Yocto Project Kernel + + +
+ Introduction + + This chapter describes how to accomplish tasks involving a kernel's tree structure. + The information is designed to help the developer that wants to modify the Yocto + Project kernel and contribute changes upstream to the Yocto Project. + The information covers the following: + + Tree construction + Build strategies + Workflow examples + + +
+ +
+ Tree Construction + + This section describes construction of the Yocto Project kernel source repositories + as accomplished by the Yocto Project team to create kernel repositories. + These kernel repositories are found under the heading "Yocto Linux Kernel" at + &YOCTO_GIT_URL;/cgit.cgi + and can be shipped as part of a Yocto Project release. + The team creates these repositories by + compiling and executing the set of feature descriptions for every BSP/feature + in the product. + Those feature descriptions list all necessary patches, + configuration, branching, tagging and feature divisions found in a kernel. + Thus, the Yocto Project kernel repository (or tree) is built. + + + The existence of this tree allows you to access and clone a particular + Yocto Project kernel repository and use it to build images based on their configurations + and features. + + + You can find the files used to describe all the valid features and BSPs + in the Yocto Project kernel in any clone of the Yocto Project kernel source repository + Git tree. + For example, the following command clones the Yocto Project baseline kernel that + branched off of linux.org version 3.4: + + $ git clone git://git.yoctoproject.org/linux-yocto-3.4 + + For another example of how to set up a local Git repository of the Yocto Project + kernel files, see the + "Yocto Project Kernel" bulleted + item in the Yocto Project Development Manual. + + + Once you have cloned the kernel Git repository on your local machine, you can + switch to the meta branch within the repository. + Here is an example that assumes the local Git repository for the kernel is in + a top-level directory named linux-yocto-3.4: + + $ cd ~/linux-yocto-3.4 + $ git checkout -b meta origin/meta + + Once you have checked out and switched to the meta branch, + you can see a snapshot of all the kernel configuration and feature descriptions that are + used to build that particular kernel repository. + These descriptions are in the form of .scc files. + + + You should realize, however, that browsing your local kernel repository + for feature descriptions and patches is not an effective way to determine what is in a + particular kernel branch. + Instead, you should use Git directly to discover the changes in a branch. + Using Git is an efficient and flexible way to inspect changes to the kernel. + For examples showing how to use Git to inspect kernel commits, see the following sections + in this chapter. + + Ground up reconstruction of the complete kernel tree is an action only taken by the + Yocto Project team during an active development cycle. + When you create a clone of the kernel Git repository, you are simply making it + efficiently available for building and development. + + + + The following steps describe what happens when the Yocto Project Team constructs + the Yocto Project kernel source Git repository (or tree) found at + given the + introduction of a new top-level kernel feature or BSP. + These are the actions that effectively create the tree + that includes the new feature, patch or BSP: + + A top-level kernel feature is passed to the kernel build subsystem. + Normally, this feature is a BSP for a particular kernel type. + The file that describes the top-level feature is located by searching + these system directories: + + The in-tree kernel-cache directories, which are located + in meta/cfg/kernel-cache + Areas pointed to by SRC_URI statements + found in recipes + + For a typical build, the target of the search is a + feature description in an .scc file + whose name follows this format: + + <bsp_name>-<kernel_type>.scc + + + Once located, the feature description is either compiled into a simple script + of actions, or into an existing equivalent script that is already part of the + shipped kernel. + Extra features are appended to the top-level feature description. + These features can come from the + KERNEL_FEATURES + variable in recipes. + Each extra feature is located, compiled and appended to the script + as described in step three. + The script is executed to produce a series of meta-* + directories. + These directories are descriptions of all the branches, tags, patches and configurations that + need to be applied to the base Git repository to completely create the + source (build) branch for the new BSP or feature. + The base repository is cloned, and the actions + listed in the meta-* directories are applied to the + tree. + The Git repository is left with the desired branch checked out and any + required branching, patching and tagging has been performed. + + + + The kernel tree is now ready for developer consumption to be locally cloned, + configured, and built into a Yocto Project kernel specific to some target hardware. + The generated meta-* directories add to the kernel + as shipped with the Yocto Project release. + Any add-ons and configuration data are applied to the end of an existing branch. + The full repository generation that is found in the + official Yocto Project kernel repositories at + http://git.yoctoproject.org/cgit.cgi + is the combination of all supported boards and configurations. + The technique the Yocto Project team uses is flexible and allows for seamless + blending of an immutable history with additional patches specific to a + deployment. + Any additions to the kernel become an integrated part of the branches. + + +
+ +
+ Build Strategy + + Once a local Git repository of the Yocto Project kernel exists on a development system, + you can consider the compilation phase of kernel development - building a kernel image. + Some prerequisites exist that are validated by the build process before compilation + starts: + + + + The + SRC_URI points + to the kernel Git repository. + A BSP build branch exists. + This branch has the following form: + + <kernel_type>/<bsp_name> + + + + + The OpenEmbedded build system makes sure these conditions exist before attempting compilation. + Other means, however, do exist, such as as bootstrapping a BSP, see + the "Workflow Examples". + + + + Before building a kernel, the build process verifies the tree + and configures the kernel by processing all of the + configuration "fragments" specified by feature descriptions in the .scc + files. + As the features are compiled, associated kernel configuration fragments are noted + and recorded in the meta-* series of directories in their compilation order. + The fragments are migrated, pre-processed and passed to the Linux Kernel + Configuration subsystem (lkc) as raw input in the form + of a .config file. + The lkc uses its own internal dependency constraints to do the final + processing of that information and generates the final .config file + that is used during compilation. + + + + Using the board's architecture and other relevant values from the board's template, + kernel compilation is started and a kernel image is produced. + + + + The other thing that you notice once you configure a kernel is that + the build process generates a build tree that is separate from your kernel's local Git + source repository tree. + This build tree has a name that uses the following form, where + ${MACHINE} is the metadata name of the machine (BSP) and "kernel_type" is one + of the Yocto Project supported kernel types (e.g. "standard"): + + linux-${MACHINE}-<kernel_type>-build + + + + + The existing support in the kernel.org tree achieves this + default functionality. + + + + This behavior means that all the generated files for a particular machine or BSP are now in + the build tree directory. + The files include the final .config file, all the .o + files, the .a files, and so forth. + Since each machine or BSP has its own separate build directory in its own separate branch + of the Git repository, you can easily switch between different builds. + +
+ +
+ Workflow Examples + + + As previously noted, the Yocto Project kernel has built-in Git integration. + However, these utilities are not the only way to work with the kernel repository. + The Yocto Project has not made changes to Git or to other tools that + would invalidate alternate workflows. + Additionally, the way the kernel repository is constructed results in using + only core Git functionality, thus allowing any number of tools or front ends to use the + resulting tree. + + + + This section contains several workflow examples. + Many of the examples use Git commands. + You can find Git documentation at + . + You can find a simple overview of using Git with the Yocto Project in the + "Git" + section of the Yocto Project Development Manual. + + +
+ Change Inspection: Changes/Commits + + + A common question when working with a kernel is: + "What changes have been applied to this tree?" + + + + In projects that have a collection of directories that + contain patches to the kernel, it is possible to inspect or "grep" the contents + of the directories to get a general feel for the changes. + This sort of patch inspection is not an efficient way to determine what has been + done to the kernel. + The reason it is inefficient is because there are many optional patches that are + selected based on the kernel type and the feature description. + Additionally, patches could exist in directories that are not included in the search. + + + + A more efficient way to determine what has changed in the branch is to use + Git and inspect or search the kernel tree. + This method gives you a full view of not only the source code modifications, + but also provides the reasons for the changes. + + +
+ What Changed in a Kernel? + + + Following are a few examples that show how to use Git commands to examine changes. + Because Git repositories in the Yocto Project do not break existing Git + functionality, and because there exists many permutations of these types of + Git commands, many methods exist by which you can discover changes. + + In the following examples, unless you provide a commit range, + kernel.org history is blended with Yocto Project + kernel changes. + You can form ranges by using branch names from the kernel tree as the + upper and lower commit markers with the Git commands. + You can see the branch names through the web interface to the + Yocto Project source repositories at + . + For example, the branch names for the linux-yocto-3.4 + kernel repository can be seen at + . + + To see a full range of the changes, use the + git whatchanged command and specify a commit range + for the branch (<commit>..<commit>). + + + + Here is an example that looks at what has changed in the + emenlow branch of the + linux-yocto-3.4 kernel. + The lower commit range is the commit associated with the + standard/base branch, while + the upper commit range is the commit associated with the + standard/emenlow branch. + + $ git whatchanged origin/standard/base..origin/standard/emenlow + + + + + To see a summary of changes use the git log command. + Here is an example using the same branches: + + $ git log --oneline origin/standard/base..origin/standard/emenlow + + The git log output might be more useful than + the git whatchanged as you get + a short, one-line summary of each change and not the entire commit. + + + + If you want to see code differences associated with all the changes, use + the git diff command. + Here is an example: + + $ git diff origin/standard/base..origin/standard/emenlow + + + + + You can see the commit log messages and the text differences using the + git show command: + Here is an example: + + $ git show origin/standard/base..origin/standard/emenlow + + + + + You can create individual patches for each change by using the + git format-patch command. + Here is an example that that creates patch files for each commit and + places them in your Documents directory: + + $ git format-patch -o $HOME/Documents origin/standard/base..origin/standard/emenlow + + +
+ +
+ Show a Particular Feature or Branch Change + + + Developers use tags in the Yocto Project kernel tree to divide changes for significant + features or branches. + Once you know a particular tag, you can use Git commands + to show changes associated with the tag and find the branches that contain + the feature. + + Because BSP branch, kernel.org, and feature tags are all + present, there could be many tags. + + The git show <tag> command shows changes that are tagged by + a feature. + Here is an example that shows changes tagged by the systemtap + feature: + + $ git show systemtap + + You can use the git branch --contains <tag> command + to show the branches that contain a particular feature. + This command shows the branches that contain the systemtap + feature: + + $ git branch --contains systemtap + + + + + You can use many other comparisons to isolate BSP and kernel changes. + For example, you can compare against kernel.org tags + such as the v3.4 tag. + +
+
+ +
+ Development: Saving Kernel Modifications + + + Another common operation is to build a BSP supplied by the Yocto Project, make some + changes, rebuild, and then test. + Those local changes often need to be exported, shared or otherwise maintained. + + + + Since the Yocto Project kernel source tree is backed by Git, this activity is + much easier as compared to with previous releases. + Because Git tracks file modifications, additions and deletions, it is easy + to modify the code and later realize that you need to save the changes. + It is also easy to determine what has changed. + This method also provides many tools to commit, undo and export those modifications. + + + + This section and its sub-sections, describe general application of Git's + push and pull commands, which are used to + get your changes upstream or source your code from an upstream repository. + The Yocto Project provides scripts that help you work in a collaborative development + environment. + For information on these scripts, see the + "Using Scripts to Push a Change + Upstream and Request a Pull" and + "Using Email to Submit a Patch" + sections in the Yocto Project Development Manual. + + + + There are many ways to save kernel modifications. + The technique employed + depends on the destination for the patches: + + + Bulk storage + Internal sharing either through patches or by using Git + External submissions + Exporting for integration into another Source Code + Manager (SCM) + + + + + Because of the following list of issues, the destination of the patches also influences + the method for gathering them: + + + Bisectability + Commit headers + Division of subsystems for separate submission or review + + + +
+ Bulk Export + + + This section describes how you can "bulk" export changes that have not + been separated or divided. + This situation works well when you are simply storing patches outside of the kernel + source repository, either permanently or temporarily, and you are not committing + incremental changes during development. + + This technique is not appropriate for full integration of upstream submission + because changes are not properly divided and do not provide an avenue for per-change + commit messages. + Therefore, this example assumes that changes have not been committed incrementally + during development and that you simply must gather and export them. + + + # bulk export of ALL modifications without separation or division + # of the changes + + $ git add . + $ git commit -s -a -m <msg> + or + $ git commit -s -a # and interact with $EDITOR + + + + + The previous operations capture all the local changes in the project source + tree in a single Git commit. + And, that commit is also stored in the project's source tree. + + + + Once the changes are exported, you can restore them manually using a template + or through integration with the default_kernel. + + +
+ +
+ Incremental/Planned Sharing + + + This section describes how to save modifications when you are making incremental + commits or practicing planned sharing. + The examples in this section assume that you have incrementally committed + changes to the tree during development and now need to export them. + The sections that follow + describe how you can export your changes internally through either patches or by + using Git commands. + + + + During development, the following commands are of interest. + For full Git documentation, refer to the Git documentation at + . + + + # edit a file + $ vi <path>/file + # stage the change + $ git add <path>/file + # commit the change + $ git commit -s + # remove a file + $ git rm <path>/file + # commit the change + $ git commit -s + + ... etc. + + + + + Distributed development with Git is possible when you use a universally + agreed-upon unique commit identifier (set by the creator of the commit) that maps to a + specific change set with a specific parent. + This identifier is created for you when + you create a commit, and is re-created when you amend, alter or re-apply + a commit. + As an individual in isolation, this is of no interest. + However, if you + intend to share your tree with normal Git push and + pull operations for + distributed development, you should consider the ramifications of changing a + commit that you have already shared with others. + + + + Assuming that the changes have not been pushed upstream, or pulled into + another repository, you can update both the commit content and commit messages + associated with development by using the following commands: + + + $ Git add <path>/file + $ Git commit --amend + $ Git rebase or Git rebase -i + + + + + Again, assuming that the changes have not been pushed upstream, and that + no pending works-in-progress exist (use git status to check), then + you can revert (undo) commits by using the following commands: + + + # remove the commit, update working tree and remove all + # traces of the change + $ git reset --hard HEAD^ + # remove the commit, but leave the files changed and staged for re-commit + $ git reset --soft HEAD^ + # remove the commit, leave file change, but not staged for commit + $ git reset --mixed HEAD^ + + + + + You can create branches, "cherry-pick" changes, or perform any number of Git + operations until the commits are in good order for pushing upstream + or for pull requests. + After a push or pull command, + commits are normally considered + "permanent" and you should not modify them. + If the commits need to be changed, you can incrementally do so with new commits. + These practices follow standard Git workflow and the kernel.org best + practices, which is recommended. + + It is recommended to tag or branch before adding changes to a Yocto Project + BSP or before creating a new one. + The reason for this recommendation is because the branch or tag provides a + reference point to facilitate locating and exporting local changes. + + + +
+ Exporting Changes Internally by Using Patches + + + This section describes how you can extract committed changes from a working directory + by exporting them as patches. + Once the changes have been extracted, you can use the patches for upstream submission, + place them in a Yocto Project template for automatic kernel patching, + or apply them in many other common uses. + + + + This example shows how to create a directory with sequentially numbered patches. + Once the directory is created, you can apply it to a repository using the + git am command to reproduce the original commit and all + the related information such as author, date, commit log, and so forth. + + The new commit identifiers (ID) will be generated upon re-application. + This action reflects that the commit is now applied to an underlying commit + with a different ID. + + + # <first-commit> can be a tag if one was created before development + # began. It can also be the parent branch if a branch was created + # before development began. + + $ git format-patch -o <dir> <first commit>..<last commit> + + + + + In other words: + + # Identify commits of interest. + + # If the tree was tagged before development + $ git format-patch -o <save dir> <tag> + + # If no tags are available + $ git format-patch -o <save dir> HEAD^ # last commit + $ git format-patch -o <save dir> HEAD^^ # last 2 commits + $ git whatchanged # identify last commit + $ git format-patch -o <save dir> <commit id> + $ git format-patch -o <save dir> <rev-list> + + +
+ +
+ Exporting Changes Internally by Using Git + + + This section describes how you can export changes from a working directory + by pushing the changes into a master repository or by making a pull request. + Once you have pushed the changes to the master repository, you can then + pull those same changes into a new kernel build at a later time. + + + + Use this command form to push the changes: + + $ git push ssh://<master_server>/<path_to_repo> + <local_branch>:<remote_branch> + + + + + For example, the following command pushes the changes from your local branch + yocto/standard/common-pc/base to the remote branch with the same name + in the master repository //git.mycompany.com/pub/git/kernel-3.4. + + $ git push ssh://git.mycompany.com/pub/git/kernel-3.4 \ + yocto/standard/common-pc/base:yocto/standard/common-pc/base + + + + + A pull request entails using the git request-pull command to compose + an email to the + maintainer requesting that a branch be pulled into the master repository, see + for an example. + + Other commands such as git stash or branching can also be used to save + changes, but are not covered in this document. + + +
+
+ +
+ Exporting Changes for External (Upstream) Submission + + + This section describes how to export changes for external upstream submission. + If the patch series is large or the maintainer prefers to pull + changes, you can submit these changes by using a pull request. + However, it is common to send patches as an email series. + This method allows easy review and integration of the changes. + + Before sending patches for review be sure you understand the + community standards for submitting and documenting changes and follow their best practices. + For example, kernel patches should follow standards such as: + + + + Documentation/SubmittingPatches (in any linux + kernel source tree) + + + + + + The messages used to commit changes are a large part of these standards. + Consequently, be sure that the headers for each commit have the required information. + For information on how to follow the Yocto Project commit message standards, see the + "How to Submit a + Change" section in the Yocto Project Development Manual. + + + + If the initial commits were not properly documented or do not meet those standards, + you can re-base by using the git rebase -i command to + manipulate the commits and + get them into the required format. + Other techniques such as branching and cherry-picking commits are also viable options. + + + + Once you complete the commits, you can generate the email that sends the patches + to the maintainer(s) or lists that review and integrate changes. + The command git send-email is commonly used to ensure + that patches are properly + formatted for easy application and avoid mailer-induced patch damage. + + + + The following is an example of dumping patches for external submission: + + # dump the last 4 commits + $ git format-patch --thread -n -o ~/rr/ HEAD^^^^ + $ git send-email --compose --subject '[RFC 0/N] <patch series summary>' \ + --to foo@yoctoproject.org --to bar@yoctoproject.org \ + --cc list@yoctoproject.org ~/rr + # the editor is invoked for the 0/N patch, and when complete the entire + # series is sent via email for review + + +
+ +
+ Exporting Changes for Import into Another SCM + + + When you want to export changes for import into another + Source Code Manager (SCM), you can use any of the previously discussed + techniques. + However, if the patches are manually applied to a secondary tree and then + that tree is checked into the SCM, you can lose change information such as + commit logs. + This process is not recommended. + + + + Many SCMs can directly import Git commits, or can translate Git patches so that + information is not lost. + Those facilities are SCM-dependent and you should use them whenever possible. + +
+
+ +
+ Working with the Yocto Project Kernel in Another SCM + + + This section describes kernel development in an SCM other than Git, + which is not the same as exporting changes to another SCM described earlier. + For this scenario, you use the OpenEmbedded build system to + develop the kernel in a different SCM. + The following must be true for you to accomplish this: + + The delivered Yocto Project kernel must be exported into the second + SCM. + Development must be exported from that secondary SCM into a + format that can be used by the OpenEmbedded build system. + + + +
+ Exporting the Delivered Kernel to the SCM + + + Depending on the SCM, it might be possible to export the entire Yocto Project + kernel Git repository, branches and all, into a new environment. + This method is preferred because it has the most flexibility and potential to maintain + the meta data associated with each commit. + + + + When a direct import mechanism is not available, it is still possible to + export a branch (or series of branches) and check them into a new repository. + + + + The following commands illustrate some of the steps you could use to + import the yocto/standard/common-pc/base + kernel into a secondary SCM: + + $ git checkout yocto/standard/common-pc/base + $ cd .. ; echo linux/.git > .cvsignore + $ cvs import -m "initial import" linux MY_COMPANY start + + + + + You could now relocate the CVS repository and use it in a centralized manner. + + + + The following commands illustrate how you can condense and merge two BSPs into a + second SCM: + + $ git checkout yocto/standard/common-pc/base + $ git merge yocto/standard/common-pc-64/base + # resolve any conflicts and commit them + $ cd .. ; echo linux/.git > .cvsignore + $ cvs import -m "initial import" linux MY_COMPANY start + + +
+ +
+ Importing Changes for the Build + + + Once development has reached a suitable point in the second development + environment, you need to export the changes as patches. + To export them, place the changes in a recipe and + automatically apply them to the kernel during patching. + +
+
+ +
+ Creating a BSP Based on an Existing Similar BSP + + + This section overviews the process of creating a BSP based on an + existing similar BSP. + The information is introductory in nature and does not provide step-by-step examples. + For detailed information on how to create a new BSP, see + the "Creating a New BSP Layer Using the yocto-bsp Script" section in the + Yocto Project Board Support Package (BSP) Developer's Guide, or see the + Transcript:_creating_one_generic_Atom_BSP_from_another + wiki page. + + + + The basic steps you need to follow are: + + Make sure you have set up a local Source Directory: + You must create a local + Source Directory + by either creating a Git repository (recommended) or + extracting a Yocto Project release tarball. + Choose an existing BSP available with the Yocto Project: + Try to map your board features as closely to the features of a BSP that is + already supported and exists in the Yocto Project. + Starting with something as close as possible to your board makes developing + your BSP easier. + You can find all the BSPs that are supported and ship with the Yocto Project + on the Yocto Project's Download page at + . + Be sure you have the Base BSP: + You need to either have a local Git repository of the base BSP set up or + have downloaded and extracted the files from a release BSP tarball. + Either method gives you access to the BSP source files. + Make a copy of the existing BSP, thus isolating your new + BSP work: + Copying the existing BSP file structure gives you a new area in which to work. + Make configuration and recipe changes to your new BSP: + Configuration changes involve the files in the BSP's conf + directory. + Changes include creating a machine-specific configuration file and editing the + layer.conf file. + The configuration changes identify the kernel you will be using. + Recipe changes include removing, modifying, or adding new recipe files that + instruct the build process on what features to include in the image. + Prepare for the build: + Before you actually initiate the build, you need to set up the build environment + by sourcing the environment initialization script. + After setting up the environment, you need to make some build configuration + changes to the local.conf and bblayers.conf + files. + Build the image: + The OpenEmbedded build system uses BitBake to create the image. + You need to decide on the type of image you are going to build (e.g. minimal, base, + core, sato, and so forth) and then start the build using the bitbake + command. + + +
+ +
+ "-dirty" String + + + If kernel images are being built with "-dirty" on the end of the version + string, this simply means that modifications in the source + directory have not been committed. + + $ git status + + + + + You can use the above Git command to report modified, removed, or added files. + You should commit those changes to the tree regardless of whether they will be saved, + exported, or used. + Once you commit the changes you need to rebuild the kernel. + + + + To brute force pickup and commit all such pending changes, enter the following: + + $ git add . + $ git commit -s -a -m "getting rid of -dirty" + + + + + Next, rebuild the kernel. + +
+
+ + -- cgit v1.2.3-54-g00ecf