summaryrefslogtreecommitdiffstats
path: root/documentation/kernel-manual
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2010-11-29 07:30:27 -0800
committerSaul Wold <Saul.Wold@intel.com>2010-12-10 22:01:17 -0800
commit2a993a0dac62a5b2ed36f710771fa73ad7a84094 (patch)
treed78de7787180496daf63c39e807e296e0dd6fbf7 /documentation/kernel-manual
parentd2e540116f1cc0bf93b7ed6f4828c56ab909489e (diff)
downloadpoky-2a993a0dac62a5b2ed36f710771fa73ad7a84094.tar.gz
documentation/kernel-manual: Added these files so the book could have chapters.
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Diffstat (limited to 'documentation/kernel-manual')
-rw-r--r--documentation/kernel-manual/kernel-concepts.xml315
-rw-r--r--documentation/kernel-manual/kernel-doc-intro.xml57
-rw-r--r--documentation/kernel-manual/kernel-how-to.xml1954
3 files changed, 2326 insertions, 0 deletions
diff --git a/documentation/kernel-manual/kernel-concepts.xml b/documentation/kernel-manual/kernel-concepts.xml
new file mode 100644
index 0000000000..b884f138e0
--- /dev/null
+++ b/documentation/kernel-manual/kernel-concepts.xml
@@ -0,0 +1,315 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<chapter id='kernel-concepts'>
5
6<title>Yocto Project Kernel Concepts</title>
7
8<section id='concepts-org'>
9 <title>Introduction</title>
10 <para>
11 This chapter provides conceptual information about the Yocto Project kernel:
12 <itemizedlist>
13 <listitem><para>Kernel Goals</para></listitem>
14 <listitem><para>Yocto Project Kernel Development and Maintenance Overview</para></listitem>
15 <listitem><para>Kernel Architecture</para></listitem>
16 <listitem><para>Kernel Tools</para></listitem>
17 </itemizedlist>
18 </para>
19</section>
20
21 <section id='kernel-goals'>
22 <title>Kernel Goals</title>
23 <para>
24 The complexity of embedded kernel design has increased dramatically.
25 Whether it is managing multiple implementations of a particular feature or tuning and
26 optimizing board specific features, flexibility and maintainability are key concerns.
27 The Yocto Project Linux kernel is presented with the embedded
28 developer's needs in mind and has evolved to assist in these key concerns.
29 For example, prior methods such as applying hundreds of patches to an extracted
30 tarball have been replaced with proven techniques that allow easy inspection,
31 bisection and analysis of changes.
32 Application of these techniques also creates a platform for performing integration and
33 collaboration with the thousands of upstream development projects.
34 </para>
35 <para>
36 With all these considerations in mind, the Yocto Project kernel and development team
37 strives to attain these goals:
38 <itemizedlist>
39 <listitem><para>Allow the end user to leverage community best practices to seamlessly
40 manage the development, build and debug cycles.</para></listitem>
41 <listitem><para>Create a platform for performing integration and collaboration with the
42 thousands of upstream development projects that exist.</para></listitem>
43 <listitem><para>Provide mechanisms that support many different work flows, front-ends and
44 management techniques.</para></listitem>
45 <listitem><para>Deliver the most up-to-date kernel possible while still ensuring that
46 the baseline kernel is the the most stable official release.</para></listitem>
47 <listitem><para>Include major technological features as part of Yocto Project's up-rev
48 strategy.</para></listitem>
49 <listitem><para>Present a git tree, that just like the upstream kernel.org tree, has a
50 clear and continuous history.</para></listitem>
51 <listitem><para>Deliver a key set of supported kernel types, where each type is tailored
52 to a specific use case (i.g. networking, consumer, devices, and so forth).</para></listitem>
53 <listitem><para>Employ a git branching strategy that from a customer's point of view
54 results in a linear path from the baseline kernel.org, through a select group of features and
55 ends with their BSP-specific commits.</para></listitem>
56 </itemizedlist>
57 </para>
58 </section>
59
60 <section id='kernel-big-picture'>
61 <title>Yocto Project Kernel Development and Maintenance Overview</title>
62 <para>
63 Yocto Project kernel, like other kernels, is based off the Linux kernel release
64 from <ulink url='http://www.kernel.org'></ulink>.
65 At the beginning of our major development cycle, we choose our Yocto Project kernel
66 based on factors like release timing, the anticipated release timing of "final" (i.e. non "rc")
67 upstream kernel.org versions, and Yocto Project feature requirements.
68 Typically this will be a kernel that is in the
69 final stages of development by the community (i.e. still in the release
70 candidate or "rc" phase) and not yet a final release.
71 But by being in the final stages of external development, we know that the
72 kernel.org final release will clearly land within the early stages of
73 the Yocto Project development window.
74 </para>
75 <para>
76 This balance allows us to deliver the most up-to-date kernel
77 as possible, while still ensuring that we have a stable official release as
78 our baseline kernel version.
79 </para>
80 <para>
81 The following figure represents the overall place the Yocto Project kernel fills.
82 </para>
83 <para>
84 <imagedata fileref="figures/kernel-big-picture.png" width="6in" depth="6in" align="center" scale="100" />
85 </para>
86 <para>
87 In the figure the ultimate source for the Yocto Project kernel is a released kernel
88 from kernel.org.
89 In addition to a foundational kernel from kernel.org the commercially released
90 Yocto Project kernel contains a mix of important new mainline
91 developments, non-mainline developments, Board Support Package (BSP) developments,
92 and custom features.
93 These additions result in a commercially released Yocto Project kernel that caters
94 to specific embedded designer needs for targeted hardware.
95 </para>
96 <para>
97 Once a Yocto Project kernel is officially released the Yocto Project team goes into
98 their next development cycle, or "uprev" cycle.
99 It is important to note that the most sustainable and stable way
100 to include feature development upstream is through a kernel uprev process.
101 Back-porting of hundreds of individual fixes and minor features from various
102 kernel versions is not sustainable and can easily compromise quality.
103 During the uprev cycle, the Yocto Project team uses an ongoing analysis of
104 kernel development, BSP support, and release timing to select the best
105 possible kernel.org version.
106 The team continually monitors community kernel
107 development to look for significant features of interest.
108 The illustration depicts this by showing the team looking back to kernel.org for new features,
109 BSP features, and significant bug fixes.
110 The team does consider back-porting large features if they have a significant advantage.
111 User or community demand can also trigger a back-port or creation of new
112 functionality in the Yocto Project baseline kernel during the uprev cycle.
113 </para>
114 <para>
115 Generally speaking, every new kernel both adds features and introduces new bugs.
116 These consequences are the basic properties of upstream kernel development and are
117 managed by the Yocto Project team's kernel strategy.
118 It is the Yocto Project team's policy to not back-port minor features to the released kernel.
119 They only consider back-porting significant technological jumps - and, that is done
120 after a complete gap analysis.
121 The reason for this policy is that simply back-porting any small to medium sized change
122 from an evolving kernel can easily create mismatches, incompatibilities and very
123 subtle errors.
124 </para>
125 <para>
126 These policies result in both a stable and a cutting
127 edge kernel that mixes forward ports of existing features and significant and critical
128 new functionality.
129 Forward porting functionality in the Yocto Project kernel can be thought of as a
130 "micro uprev."
131 The many “micro uprevs” produce a kernel version with a mix of
132 important new mainline, non-mainline, BSP developments and feature integrations.
133 This kernel gives insight into new features and allows focused
134 amounts of testing to be done on the kernel, which prevents
135 surprises when selecting the next major uprev.
136 The quality of these cutting edge kernels is evolving and the kernels are used in very special
137 cases for BSP and feature development.
138 </para>
139 </section>
140
141 <section id='kernel-architecture'>
142 <title>Kernel Architecture</title>
143 <para>
144 This section describes the architecture of the Yocto Project kernel and provides information
145 on the mechanisms used to achieve that architecture.
146 </para>
147
148 <section id='architecture-overview'>
149 <title>Overview</title>
150 <para>
151 As mentioned earlier, a key goal of Yocto Project is to present the developer with
152 a kernel that has a clear and continuous history that is visible to the user.
153 The architecture and mechanisms used achieve that goal in a manner similar to the
154 upstream kernel.org.
155
156 </para>
157 <para>
158 You can think of the Yocto Project kernel as consisting of a baseline kernel with
159 added features logically structured on top of the baseline.
160 The features are tagged and organized by way of a branching strategy implemented by the
161 source code manager (SCM) git.
162 The result is that the user has the ability to see the added features and
163 the commits that make up those features.
164 In addition to being able to see added features, the user can also view the history of what
165 made up the baseline kernel as well.
166 </para>
167 <para>
168 The following illustration shows the conceptual Yocto Project kernel.
169 </para>
170 <para>
171 <imagedata fileref="figures/kernel-architecture-overview.png" width="6in" depth="7in" align="center" scale="100" />
172 </para>
173 <para>
174 In the illustration, the "kernel.org Branch Point" marks the specific spot (or release) from
175 which the Yocto Project kernel is created. From this point "up" in the tree features and
176 differences are organized and tagged.
177 </para>
178 <para>
179 The "Yocto Project Baseline Kernel" contains functionality that is common to every kernel
180 type and BSP that is organized further up the tree. Placing these common features in the
181 tree this way means features don't have to be duplicated along individual branches of the
182 structure.
183 </para>
184 <para>
185 From the Yocto Project Baseline Kernel branch points represent specific functionality
186 for individual BSPs as well as real-time kernels.
187 The illustration represents this through three BSP-specific branches and a real-time
188 kernel branch.
189 Each branch represents some unique functionality for the BSP or a real-time kernel.
190 </para>
191 <para>
192 The real-time kernel branch has common features for all real-time kernels and contains
193 more branches for individual BSP-specific real-time kernels.
194 The illustration shows three branches as an example.
195 Each branch points the way to specific, unique features for a respective real-time
196 kernel as they apply to a given BSP.
197 </para>
198 <para>
199 The resulting tree structure presents a clear path of markers (or branches) to the user
200 that for all practical purposes is the kernel needed for any given set of requirements.
201 </para>
202 </section>
203
204 <section id='branching-and-workflow'>
205 <title>Branching Strategy and Workflow</title>
206 <para>
207 The Yocto Project team creates kernel branches at points where functionality is
208 no longer shared and thus, needs to be isolated.
209 For example, board-specific incompatibilities would require different functionality
210 and would require a branch to separate the features.
211 Likewise, for specific kernel features the same branching strategy is used.
212 This branching strategy results in a tree that has features organized to be specific
213 for particular functionality, single kernel types, or a subset of kernel types.
214 This strategy results in not having to store the same feature twice internally in the
215 tree.
216 Rather we store the unique differences required to apply the feature onto the kernel type
217 in question.
218 </para>
219 <para>
220 BSP-specific code additions are handled in a similar manner to kernel-specific additions.
221 Some BSPs only make sense given certain kernel types.
222 So, for these types, we create branches off the end of that kernel type for all
223 of the BSPs that are supported on that kernel type.
224 From the perspective of the tools that create the BSP branch, the BSP is really no
225 different than a feature.
226 Consequently, the same branching strategy applies to BSPs as it does to features.
227 So again, rather than store the BSP twice, only the unique differences for the BSP across
228 the supported multiple kernels are uniquely stored.
229 </para>
230 <para>
231 While this strategy results in a tree with a significant number of branches, it is
232 important to realize that from the customer's point of view, there is a linear
233 path that travels from the baseline kernel.org, through a select group of features and
234 ends with their BSP-specific commits.
235 In other words, the divisions of the kernel are transparent and are not relevant
236 to the developer on a day-to-day basis.
237 From the customer's perspective, this is the "master" branch.
238 They do not need not be aware of the existence of any other branches at all.
239 Of course there is value in the existence of these branches
240 in the tree, should a person decide to explore them.
241 For example, a comparison between two BSPs at either the commit level or at the line-by-line
242 code diff level is now a trivial operation.
243 </para>
244 <para>
245 Working with the kernel as a structured tree follows recognized community best practices.
246 In particular, the kernel as shipped with the product should be
247 considered an 'upstream source' and viewed as a series of
248 historical and documented modifications (commits).
249 These modifications represent the development and stabilization done
250 by the Yocto Project kernel development team.
251 </para>
252 <para>
253 Because commits only change at significant release points in the product life cycle,
254 developers can work on a branch created
255 from the last relevant commit in the shipped Yocto Project kernel.
256 As mentioned previously, the structure is transparent to the user
257 because the kernel tree is left in this state after cloning and building the kernel.
258 </para>
259 </section>
260
261 <section id='source-code-manager-git'>
262 <title>Source Code Manager - git</title>
263 <para>
264 The Source Code Manager (SCM) is git and it is the obvious mechanism for meeting the
265 previously mentioned goals.
266 Not only is it the SCM for kernel.org but git continues to grow in popularity and
267 supports many different work flows, front-ends and management techniques.
268 </para>
269 <note><para>
270 It should be noted that you can use as much, or as little, of what git has to offer
271 as is appropriate to your project.
272 </para></note>
273 </section>
274 </section>
275
276 <section id='kernel-tools'>
277 <title>Kernel Tools</title>
278 <para>
279Since most standard workflows involve moving forward with an existing tree by
280continuing to add and alter the underlying baseline, the tools that manage
281Yocto Project's kernel construction are largely hidden from the developer to
282present a simplified view of the kernel for ease of use.
283</para>
284<para>
285The fundamental properties of the tools that manage and construct the
286kernel are:
287<itemizedlist>
288 <listitem><para>the ability to group patches into named, reusable features</para></listitem>
289 <listitem><para>to allow top down control of included features</para></listitem>
290 <listitem><para>the binding of kernel configuration to kernel patches/features</para></listitem>
291 <listitem><para>the presentation of a seamless git repository that blends Yocto Project value with the kernel.org history and development</para></listitem>
292</itemizedlist>
293</para>
294<!--<para>
295The tools that construct a kernel tree will be discussed later in this
296document. The following tools form the foundation of the Yocto Project
297kernel toolkit:
298<itemizedlist>
299 <listitem><para>git : distributed revision control system created by Linus Torvalds</para></listitem>
300 <listitem><para>guilt: quilt on top of git</para></listitem>
301 <listitem><para>*cfg : kernel configuration management and classification</para></listitem>
302 <listitem><para>kgit*: Yocto Project kernel tree creation and management tools</para></listitem>
303 <listitem><para>scc : series &amp; configuration compiler</para></listitem>
304</itemizedlist>
305</para> -->
306 </section>
307
308
309
310
311
312</chapter>
313<!--
314vim: expandtab tw=80 ts=4
315-->
diff --git a/documentation/kernel-manual/kernel-doc-intro.xml b/documentation/kernel-manual/kernel-doc-intro.xml
new file mode 100644
index 0000000000..c9715d4bff
--- /dev/null
+++ b/documentation/kernel-manual/kernel-doc-intro.xml
@@ -0,0 +1,57 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<chapter id='kernel-doc-intro'>
5
6<title>Yocto Project Kernel Architecture and Use Manual</title>
7
8<section id='book-intro'>
9 <title>Introduction</title>
10 <para>
11 Yocto Project presents the kernel as a fully patched, history-clean git
12 repository.
13 The git tree represents the selected features, board support,
14 and configurations extensively tested by Yocto Project.
15 The Yocto Project kernel allows the end user to leverage community
16 best practices to seamlessly manage the development, build and debug cycles.
17 </para>
18 <para>
19 This manual describes the Yocto Project kernel by providing information
20 on its history, organization, benefits, and use.
21 The manual consists of two sections:
22 <itemizedlist>
23 <listitem><para>Concepts - Describes concepts behind the kernel.
24 You will understand how the kernel is organized and why it is organized in
25 the way it is. You will understand the benefits of the kernel's organization
26 and the mechanisms used to work with the kernel and how to apply it in your
27 design process.</para></listitem>
28 <listitem><para>Using the Kernel - Describes best practices and "how-to" information
29 that lets you put the kernel to practical use. Some examples are "How to Build a
30 Project Specific Tree", "How to Examine Changes in a Branch", and "Saving Kernel
31 Modifications."</para></listitem>
32 </itemizedlist>
33 </para>
34 <para>
35 For more information on the kernel, see the following links:
36 <itemizedlist>
37 <listitem><para><ulink url='http://ldn.linuxfoundation.org/book/1-a-guide-kernel-development-process'></ulink></para></listitem>
38 <listitem><para><ulink url='http://userweb.kernel.org/~akpm/stuff/tpp.txt'></ulink></para></listitem>
39 <listitem><para><ulink url='http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=Documentation/HOWTO;hb=HEAD'></ulink></para></listitem>
40 </itemizedlist>
41 <para>
42 You can find more information on Yocto Project by visiting the website at
43 <ulink url='http://www.yoctoproject.org'></ulink>.
44 </para>
45 </para>
46</section>
47
48
49
50
51
52
53
54</chapter>
55<!--
56vim: expandtab tw=80 ts=4
57-->
diff --git a/documentation/kernel-manual/kernel-how-to.xml b/documentation/kernel-manual/kernel-how-to.xml
new file mode 100644
index 0000000000..7f3eac3d3b
--- /dev/null
+++ b/documentation/kernel-manual/kernel-how-to.xml
@@ -0,0 +1,1954 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<chapter id='kernel-how-to'>
5
6<title>Working with the Yocto Project Kernel</title>
7
8
9<section id='actions-org'>
10 <title>Introduction</title>
11 <para>
12 This chapter describes how to accomplish tasks involving the kernel's tree structure.
13 The information covers the following:
14 <itemizedlist>
15 <listitem><para>Tree construction</para></listitem>
16 <listitem><para>Build strategies</para></listitem>
17<!-- <listitem><para>Series &amp; Configuration Compiler</para></listitem>
18 <listitem><para>kgit</para></listitem> -->
19 <listitem><para>Workflow examples</para></listitem>
20 <listitem><para>Source Code Manager (SCM)</para></listitem>
21<!-- <listitem><para>Board Support Package (BSP) template migration</para></listitem> -->
22 <listitem><para>BSP creation</para></listitem>
23 <listitem><para>Patching</para></listitem>
24 <listitem><para>Updating BSP patches and configuration</para></listitem>
25<!-- <listitem><para>guilt</para></listitem>
26 <listitem><para>scc file example</para></listitem> -->
27 <listitem><para>"dirty" string</para></listitem>
28<!-- <listitem><para>Transition kernel layer</para></listitem> -->
29 </itemizedlist>
30 </para>
31</section>
32
33 <section id='tree-construction'>
34 <title>Tree Construction</title>
35 <para>
36The Yocto Project kernel repository, as shipped with the product, is created by
37compiling and executing the set of feature descriptions for every BSP/feature
38in the product. Those feature descriptions list all necessary patches,
39configuration, branching, tagging and feature divisions found in the kernel.
40</para>
41<para>
42The files used to describe all the valid features and BSPs in the Yocto Project
43kernel can be found in any clone of the kernel git tree. The directory
44wrs/cfg/kernel-cache/ is a snapshot of all the kernel configuration and
45feature descriptions (.scc) that were used to build the kernel repository.
46It should however be noted, that browsing the snapshot of feature
47descriptions and patches is not an effective way to determine what is in a
48particular kernel branch. Using git directly to get insight into the changes
49in a branch is more efficient and a more flexible way to inspect changes to
50the kernel. Examples of using git to inspect kernel commits are in the
51following sections.
52</para>
53<para>
54As a reminder, it is envisioned that a ground up reconstruction of the
55complete kernel tree is an action only taken by Yocto Project team during an
56active development cycle. When an end user creates a project, it takes
57advantage of this complete tree in order to efficiently place a git tree
58within their project.
59</para>
60<para>
61The general flow of the project specific kernel tree construction is as follows:
62<orderedlist>
63 <listitem><para>a top level kernel feature is passed to the kernel build subsystem,
64 normally this is a BSP for a particular kernel type.</para></listitem>
65
66 <listitem><para>the file that describes the top level feature is located by searching
67 system directories:</para>
68
69 <itemizedlist>
70 <listitem><para>the kernel-cache under linux/wrs/cfg/kernel-cache</para></listitem>
71<!-- <listitem><para>kernel-*-cache directories in layers</para></listitem> -->
72 <listitem><para>recipe SRC_URIs</para></listitem>
73<!-- <listitem><para>configured and default templates</para></listitem> -->
74 </itemizedlist>
75
76 <para>In a typical build a feature description of the format:
77 &lt;bsp name&gt;-&lt;kernel type&gt;.scc is the target of the search.
78 </para></listitem>
79
80 <listitem><para>once located, the feature description is compiled into a simple script
81 of actions, or an existing equivalent script which was part of the
82 shipped kernel is located.</para></listitem>
83
84 <listitem><para>extra features are appended to the top level feature description. Extra
85 features can come from the KERNEL_FEATURES variable in recipes.</para></listitem>
86
87 <listitem><para>each extra feature is located, compiled and appended to the script from
88 step #3</para></listitem>
89
90 <listitem><para>the script is executed, and a meta-series is produced. The meta-series
91 is a description of all the branches, tags, patches and configuration that
92 need to be applied to the base git repository to completely create the
93 "bsp_name-kernel_type".</para></listitem>
94
95 <listitem><para>the base repository is cloned, and the actions
96 listed in the meta-series are applied to the tree.</para></listitem>
97
98 <listitem><para>the git repository is left with the desired branch checked out and any
99 required branching, patching and tagging has been performed.</para></listitem>
100</orderedlist>
101</para>
102
103<para>
104The tree is now ready for configuration and compilation. Those two topics will
105be covered below.
106</para>
107
108<note><para>The end user generated meta-series adds to the kernel as shipped with
109 the Yocto Project release. Any add-ons and configuration data are applied
110 to the end of an existing branch. The full repository generation that
111 is found in the linux-2.6-windriver.git is the combination of all
112 supported boards and configurations.
113</para></note>
114
115<para>
116This technique is flexible and allows the seamless blending of an immutable
117history with additional deployment specific patches. Any additions to the
118kernel become an integrated part of the branches.
119</para>
120
121<!-- <note><para>It is key that feature descriptions indicate if any branches are
122 required, since the build system cannot automatically decide where a
123 BSP should branch or if that branch point needs a name with
124 significance. There is a single restriction enforced by the compilation
125 phase:
126 </para>
127 <para>A BSP must create a branch of the format &lt;bsp name&gt;-&lt;kernel type&gt;.</para>
128
129 <para>This means that all merged/support BSPs must indicate where to start
130 its branch from, with the right name, in its .scc files. The scc
131 section describes the available branching commands in more detail.
132 </para>
133</note> -->
134
135<para>
136A summary of end user tree construction activities follow:
137<itemizedlist>
138 <listitem><para>compile and link a full top-down kernel description from feature descriptions</para></listitem>
139 <listitem><para>execute the complete description to generate a meta-series</para></listitem>
140 <listitem><para>interpret the meta-series to create a customized git repository for the
141 board</para></listitem>
142 <listitem><para>migrate configuration fragments and configure the kernel</para></listitem>
143 <listitem><para>checkout the BSP branch and build</para></listitem>
144</itemizedlist>
145</para>
146 </section>
147
148 <section id='build-strategy'>
149 <title>Build Strategy</title>
150<para>
151There are some prerequisites that must be met before starting the compilation
152phase of the kernel build system:
153</para>
154<itemizedlist>
155 <listitem><para>There must be a kernel git repository indicated in the SRC_URI.</para></listitem>
156 <listitem><para>There must be a branch &lt;bsp name&gt;-&lt;kernel type&gt;.</para></listitem>
157</itemizedlist>
158
159<para>
160These are typically met by running tree construction/patching phase of the
161build system, but can be achieved by other means. Examples of alternate work
162flows such as bootstrapping a BSP are provided below.
163</para>
164<para>
165Before building a kernel it is configured by processing all of the
166configuration "fragments" specified by the scc feature descriptions. As the
167features are compiled, associated kernel configuration fragments are noted
168and recorded in the meta-series in their compilation order. The
169fragments are migrated, pre-processed and passed to the Linux Kernel
170Configuration subsystem (lkc) as raw input in the form of a .config file.
171The lkc uses its own internal dependency constraints to do the final
172processing of that information and generates the final .config that will
173be used during compilation.
174</para>
175<para>
176Kernel compilation is started, using the board's architecture and other
177relevant values from the board template, and a kernel image is produced.
178</para>
179<para>
180The other thing that you will first see once you configure a kernel is that
181it will generate a build tree that is separate from your git source tree.
182This build dir will be called "linux-&lt;BSPname&gt;-&lt;kerntype&gt;-build" where
183kerntype is one of standard kernel types. This functionality is done by making
184use of the existing support that is within the kernel.org tree by default.
185</para>
186<para>
187What this means, is that all the generated files (that includes the final
188".config" itself, all ".o" and ".a" etc) are now in this directory. Since
189the git source tree can contain any number of BSPs, all on their own branch,
190you now can easily switch between builds of BSPs as well, since each one also
191has their own separate build directory.
192</para>
193 </section>
194
195<!-- <section id='scc'>
196 <title>Series &amp; Configuration Compiler (SCC)</title>
197<para>
198In early versions of the product, kernel patches were simply listed in a flat
199file called "patches.list", and then quilt was added as a tool to help
200traverse this list, which in quilt terms was called a "series" file.
201</para>
202<para>
203Before the 2.0 release, it was already apparent that a static series file was
204too inflexible, and that the series file had to become more dynamic and rely
205on certain state (like kernel type) in order to determine whether a patch was
206to be used or not. The 2.0 release already made use of some stateful
207construction of series files, but since the delivery mechanism was unchanged
208(tar + patches + series files), most people were not aware of anything really
209different. The 3.0 release continues with this stateful construction of
210series files, but since the delivery mechanism is changed (git + branches) it
211now is more apparent to people.
212</para>
213<para>
214As was previously mentioned, scc is a "series and configuration
215compiler". Its role is to combine feature descriptions into a format that can
216be used to generate a meta-series. A meta series contains all the required
217information to construct a complete set of branches that are required to
218build a desired board and feature set. The meta series is interpreted by the
219kgit tools to create a git repository that could be built.
220</para>
221<para>
222To illustrate how scc works, a feature description must first be understood.
223A feature description is simply a small bash shell script that is executed by
224scc in a controlled environment. Each feature description describes a set of
225operations that add patches, modify existing patches or configure the
226kernel. It is key that feature descriptions can include other features, and
227hence allow the division of patches and configuration into named, reusable
228containers.
229</para>
230<para>
231Each feature description can use any of the following valid scc commands:
232<itemizedlist>
233 <listitem><para>shell constructs: bash conditionals and other utilities can be used in a feature
234 description. During compilation, the working directory is the feature
235 description itself, so any command that is "raw shell" and not from the
236 list of supported commands, can not directly modify a git repository.</para></listitem>
237
238 <listitem><para>patch &lt;relative path&gt;/&lt;patch name&gt;: outputs a patch to be included in a feature's patch set. Only the name of
239 the patch is supplied, the path is calculated from the currently set
240 patch directory, which is normally the feature directory itself.</para></listitem>
241
242 <listitem><para>patch_trigger &gt;condition&lt; &gt;action&lt; &lt;tgt&gt;: indicate that a trigger should be set to perform an action on a
243 patch.</para>
244
245<para>The conditions can be:
246
247 <itemizedlist>
248 <listitem><para>arch:&lt;comma separated arch list or "all"&gt;</para></listitem>
249 <listitem><para>plat:&lt;comma separated platform list or "all"&gt;</para></listitem>
250 </itemizedlist></para>
251<para>The action can be:
252 <itemizedlist>
253 <listitem><para>exclude: This is used in exceptional situations where a patch
254 cannot be applied for certain reasons (arch or platform).
255 When the trigger is satisfied the patch will be removed from
256 the patch list.</para></listitem>
257 <listitem><para>include: This is used to include a patch only for a specific trigger.
258 Like exclude, this should only be used when necessary.
259 It takes 1 argument, the patch to include.</para></listitem>
260 </itemizedlist></para></listitem>
261
262 <listitem><para>include &lt;feature name&gt; [after &lt;feature&gt;]: includes a feature for processing. The feature is "expanded" at the
263 position of the include directive. This means that any patches,
264 configuration or sub-includes of the feature will appear in the final
265 series before the commands that follow the include.</para>
266 <para>
267 include searches the include directories for a matching feature name,
268 include directories are passed to scc by the caller using -I &lt;path&gt; and
269 is transparent to the feature script. This means that &lt;feature name&gt; must
270 be relative to one of the search paths. For example, if
271 /opt/kernel-cache/feat/sched.scc is to be included and scc is invoked
272 with -I /opt/kernel-cache, then a feature would issue "include
273 feat/sched.scc" to include the feature.
274</para>
275<para>
276 The optional "after" directive allows a feature to modify the existing
277 order of includes and insert a feature after the named feature is
278 processed. Note: the "include foo after bar" must be issued before "bar"
279 is processed, so is normally only used by a new top level feature to
280 modify the order of features in something it is including.</para></listitem>
281
282 <listitem><para>exclude &lt;feature name&gt;: Indicates that a particular feature should *not* be included even if an
283 'include' directive is found. The exclude must be issued before the
284 include is processed, so is normally only used by a new top level feature
285 to modify the order of features in something it is including.</para></listitem>
286
287 <listitem><para>git &lt;command&gt;: Issues any git command during tree construction. Note: this command is
288 not validated/sanitized so care must be taken to not damage the
289 tree. This can be used to script branching, tagging, pulls or other git
290 operations.</para></listitem>
291
292 <listitem><para>dir &lt;directory&gt;: changes the working directory for "patch" directives. This can be used to
293 shorten a long sequence of patches by not requiring a common relative
294 directory to be issued each time.</para></listitem>
295
296 <listitem><para>kconf &lt;type&gt; &lt;fragment name&gt;: associates a kernel config frag with the feature.
297 &lt;type&gt; can be
298 "hardware" or "non-hardware" and is used by the kernel configuration
299 subsystem to audit configuration. &lt;fragment name&gt; is the name of a file
300 in the current feature directory that contains a series of kernel
301 configuration options. There is no restriction on the chosen fragment
302 name, although a suffix of ".cfg" is recommended. Multiple fragment
303 specifications are supported.</para></listitem>
304
305 <listitem><para>branch &lt;branch name&gt;: creates a branch in the tree. All subsequent patch commands will be
306 applied to the new branch and changes isolated from the rest of the
307 repository.</para></listitem>
308
309 <listitem><para>scc_leaf &lt;base feature&gt; &lt;branch name&gt;: Performs a combination feature include and branch. This is mainly a
310 convenience directive, but has significance to some build system bindings
311 as a sentinel to indicate that this intends to create a branch that is
312 valid for kernel compilation.</para></listitem>
313
314 <listitem><para>tag &lt;tag name&gt;: Tags the tree. The tag will be applied in processing order, so will
315 be after already applied patches and precede patches yet to be applied.</para></listitem>
316
317 <listitem><para>define &lt;var&gt; &lt;value&gt;: Creates a variable with a particular value that can be used in subsequent
318 feature descriptions.</para></listitem>
319</itemizedlist>
320
321</para>
322 </section> -->
323
324<!-- <section id='kgit-tools'>
325 <title>kgit Tools</title>
326<para>
327The kgit tools are responsible for constructing and maintaining the Wind
328River kernel repository. These activities include importing, exporting, and
329applying patches as well as sanity checking and branch management. From the
330developers perspective, the kgit tools are hidden and rarely require
331interactive use. But one tool in particular that warrants further description
332is "kgit-meta".
333</para>
334<para>
335kgit-meta is the actual application of feature description(s) to a kernel repo.
336In other words, it is responsible for interpreting the meta series generated
337from a scc compiled script. As a result, kgit-meta is coupled to the set of
338commands permitted in a .scc feature description (listed in the scc section).
339kgit-meta understands both the meta series format and how to use git and
340guilt to modify a base git repository. It processes a meta-series line by
341line, branching, tagging, patching and tracking changes that are made to the
342base git repository.
343</para>
344<para>
345Once kgit-meta has processed a meta-series, it leaves the repository with the
346last branch checked out, and creates the necessary guilt infrastructure to
347inspect the tree, or add to it via using guilt. As was previously mentioned,
348guilt is not required, but is provided as a convenience. Other utilities such
349as quilt, stgit, git or others can also be used to manipulate the git
350repository.
351</para>
352 </section> -->
353
354 <section id='workflow-examples'>
355 <title>Workflow Examples</title>
356
357 <para>
358As previously noted, the Yocto Project kernel has built in git/guilt
359integration, but these utilities are not the only way to work with the kernel
360repository. Yocto Project has not made changes to git, or other tools that
361invalidate alternate workflows. Additionally, the way the kernel repository
362is constructed uses only core git functionality allowing any number of tools
363or front ends to use the resulting tree.</para>
364<para>
365This section contains several workflow examples.
366</para>
367
368 <section id='change-inspection-kernel-changes-commits'>
369 <title>Change Inspection: Kernel Changes/Commits</title>
370<para>
371A common question when working with a BSP/kernel is: "What changes have been applied to this tree?"
372</para>
373<para>
374In some projects, where a collection of directories that
375contained patches to the kernel, those patches could be inspected, grep'd or
376otherwise used to get a general feeling for changes. This sort of patch
377inspection is not an efficient way to determine what has been done to the
378kernel, since there are many optional patches that are selected based on the
379kernel type and feature description, not to mention patches that are actually
380in directories that are not being searched.
381</para>
382<para>
383A more effective way to determine what has changed in the kernel is to use
384git and inspect / search the kernel tree. This is a full view of not only the
385source code modifications, but the reasoning behind the changes.
386</para>
387 <section id='what-changed-in-a-bsp'>
388 <title>What Changed in a BSP?</title>
389<para>
390These examples could continue for some time, since the Yocto Project git
391repository doesn't break existing git functionality and there are nearly
392endless permutations of those commands. Also note that unless a commit range
393is given (&lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;), kernel.org history is blended
394with Yocto Project changes
395</para>
396<literallayout class='monospaced'>
397 # full description of the changes
398 &gt; git whatchanged &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
399 &gt; eg: git whatchanged standard..common_pc-standard
400
401 # summary of the changes
402 &gt; git log &dash;&dash;pretty=oneline &dash;&dash;abbrev-commit &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
403
404 # source code changes (one combined diff)
405 &gt; git diff &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
406 &gt; git show &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
407
408 # dump individual patches per commit
409 &gt; git format-patch -o &lt;dir&gt; &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
410
411 # determine the change history of a particular file
412 &gt; git whatchanged &lt;path to file&gt;
413
414 # determine the commits which touch each line in a file
415 &gt; git blame &lt;path to file&gt;
416</literallayout>
417 </section>
418
419 <section id='show-a-particular-feature-or-branch-change'>
420 <title>Show a Particular Feature or Branch Change</title>
421<para>
422Significant features or branches are tagged in the Yocto Project tree to divide
423changes. Remember to first determine (or add) the tag of interest. Note:
424there will be many tags, since each BSP branch is tagged, kernel.org tags and
425feature tags are all present.
426</para>
427<literallayout class='monospaced'>
428 # show the changes tagged by a feature
429 &gt; git show &lt;tag&gt;
430 &gt; eg: git show yaffs2
431
432 # determine which branches contain a feature
433 &gt; git branch &dash;&dash;contains &lt;tag&gt;
434
435 # show the changes in a kernel type
436 &gt; git whatchanged wrs_base..&lt;kernel type&gt;
437 &gt; eg: git whatchanged wrs_base..standard
438</literallayout>
439<para>
440Many other comparisons can be done to isolate BSP changes, such as comparing
441to kernel.org tags (v2.6.27.18, etc), per subsystem comparisons (git
442whatchanged mm) or many other types of checks.
443</para>
444 </section>
445 </section>
446
447 <section id='development-saving-kernel-modifications'>
448 <title>Development: Saving Kernel Modifications</title>
449<para>
450Another common operation is to build a Yocto Project supplied BSP, make some
451changes, rebuild and test. Those local changes often need to be exported,
452shared or otherwise maintained.
453</para>
454<para>
455Since the Yocto Project kernel source tree is backed by git, this activity is
456greatly simplified and is much easier than in previous releases. git tracks
457file modifications, additions and deletions, which allows the developer to
458modify the code and later realize that the changes should be saved, and
459easily determine what was changed. It also provides many tools to commit,
460undo and export those modifications.
461</para>
462<para>
463There are many ways to perform this action, and the technique employed
464depends on the destination for the patches, which could be any of:
465<itemizedlist>
466 <listitem><para>bulk storage</para></listitem>
467 <listitem><para>internal sharing either through patches or using git</para></listitem>
468 <listitem><para>external submission</para></listitem>
469 <listitem><para>export for integration into another SCM</para></listitem>
470</itemizedlist>
471</para>
472<para>
473The destination of the patches also incluences the method of gathering them
474due to issues such as:
475<itemizedlist>
476 <listitem><para>bisectability</para></listitem>
477 <listitem><para>commit headers</para></listitem>
478 <listitem><para>division of subsystems for separate submission / review</para></listitem>
479</itemizedlist>
480</para>
481
482 <section id='bulk-export'>
483 <title>Bulk Export</title>
484<para>
485If patches are simply being stored outside of the kernel source repository,
486either permanently or temporarily, then there are several methods that can be
487used.
488</para>
489<para>
490Note the "bulk" in this discussion, these techniques are not appropriate for
491full integration of upstream submission, since they do not properly divide
492changes or provide an avenue for per-change commit messages. This example
493assumes that changes have not been committed incrementally during development
494and simply must be gathered and exported.
495<literallayout class='monospaced'>
496 # bulk export of ALL modifications without separation or division
497 # of the changes
498
499 &gt; git add .
500 &gt; git commit -s -a -m &gt;commit message&lt;
501 or
502 &gt; git commit -s -a # and interact with $EDITOR
503</literallayout>
504</para>
505<para>
506These operations have captured all the local changes in the project source
507tree in a single git commit, and that commit is also stored in the project's
508source tree.
509</para>
510<para>
511Once exported, those changes can then be restored manually, via a template or
512through integration with the default_kernel. Those topics are covered in
513future sections.
514</para>
515 </section>
516
517 <section id='incremental-planned-sharing'>
518 <title>Incremental/Planned Sharing</title>
519<para>
520Note: unlike the previous "bulk" section, the following examples assume that
521changes have been incrementally committed to the tree during development and
522now are being exported.
523</para>
524<para>
525During development the following commands will be of interest, but for full
526git documentation refer to the git man pages or an online resource such as
527http://github.com
528<literallayout class='monospaced'>
529 # edit a file
530 &gt; vi &gt;path&lt;/file
531 # stage the change
532 &gt; git add &gt;path&lt;/file
533 # commit the change
534 &gt; git commit -s
535 # remove a file
536 &gt; git rm &gt;path&lt;/file
537 # commit the change
538 &gt; git commit -s
539
540 ... etc.
541</literallayout>
542</para>
543<para>
544Distributed development with git is possible by having a universally agreed
545upon unique commit identifier (set by the creator of the commit) mapping to a
546specific changeset with a specific parent. This ID is created for you when
547you create a commit, and will be re-created when you amend/alter or re-apply
548a commit. As an individual in isolation, this is of no interest, but if you
549intend to share your tree with normal git push/pull operations for
550distributed development, you should consider the ramifications of changing a
551commit that you've already shared with others.
552</para>
553<para>
554Assuming that the changes have *not* been pushed upstream, or pulled into
555another repository, both the commit content and commit messages associated
556with development can be update via:
557<literallayout class='monospaced'>
558 &gt; git add &gt;path&lt;/file
559 &gt; git commit &dash;&dash;amend
560 &gt; git rebase or git rebase -i
561</literallayout>
562</para>
563<para>
564Again, assuming that the changes have *not* been pushed upstream, and that
565there are no pending works in progress (use "git status" to check) then
566commits can be reverted (undone) via:
567<literallayout class='monospaced'>
568 # remove the commit, update working tree and remove all
569 # traces of the change
570 &gt; git reset &dash;&dash;hard HEAD^
571 # remove the commit, but leave the files changed and staged for re-commit
572 &gt; git reset &dash;&dash;soft HEAD^
573 # remove the commit, leave file change, but not staged for commit
574 &gt; git reset &dash;&dash;mixed HEAD^
575</literallayout>
576</para>
577<para>
578Branches can be created, changes cherry-picked or any number of git
579operations performed until the commits are in good order for pushing upstream
580or pull requests. After a push or pull, commits are normally considered
581'permanent' and should not be modified, only incrementally changed in new
582commits. This is standard "git" workflow and Yocto Project recommends the
583kernel.org best practices.
584</para>
585<note><para>It is recommend to tag or branch before adding changes to a Yocto Project
586 BSP (or creating a new one), since the branch or tag provides a
587 reference point to facilitate locating and exporting local changes.
588</para></note>
589
590 <section id='export-internally-via-patches'>
591 <title>Export Internally Via Patches</title>
592<para>
593Committed changes can be extracted from a working directory by exporting them
594as patches. Those patches can be used for upstream submission, placed in a
595Yocto Project template for automatic kernel patching or many other common uses.
596
597<literallayout class='monospaced'>
598 # &gt;first commit&gt; can be a tag if one was created before development
599 # began. It can also be the parent branch if a branch was created
600 # before development began.
601
602 &gt; git format-patch -o &lt;dir&gt; &lt;first commit&gt;..&lt;last commit&gt;
603</literallayout>
604</para>
605
606<para>
607 In other words:
608<literallayout class='monospaced'>
609 # identify commits of interest.
610
611 # if the tree was tagged before development
612 &gt; git format-patch -o &lt;save dir&gt; &lt;tag&gt;
613
614 # if no tags are available
615 &gt; git format-patch -o &lt;save dir&gt; HEAD^ # last commit
616 &gt; git format-patch -o &lt;save dir&gt; HEAD^^ # last 2 commits
617 &gt; git whatchanged # identify last commit
618 &gt; git format-patch -o &lt;save dir&gt; &lt;commit id&gt;
619 &gt; git format-patch -o &lt;save dir&gt; &lt;rev-list&gt;
620</literallayout>
621</para>
622
623<para>
624The result is a directory with sequentially numbered patches, that when
625applied to a repository using "git am", will reproduce the original commit
626and all related information (author, date, commit log, etc) will be
627preserved. Note that new commit IDs will be generated upon reapplication,
628reflecting that the commit is now applied to an underlying commit with a
629different ID.
630</para>
631<!--<para>
632See the "template patching" example for how to use the patches to
633automatically apply to a new kernel build.
634</para> -->
635 </section>
636
637 <section id='export-internally-via-git'>
638 <title>Export Internally Via git</title>
639<para>
640Committed changes can also be exported from a working directory by pushing
641(or by making a pull request) the changes into a master repository. Those
642same change can then be pulled into a new kernel build at a later time using this command form:
643<literallayout class='monospaced'>
644 git push ssh://&lt;master server&gt;/&lt;path to repo&gt; &lt;local branch&gt;:&lt;remote branch&gt;
645</literallayout>
646For example:
647<literallayout class='monospaced'>
648 &gt; push ssh://git.mycompany.com/pub/git/kernel-2.6.27 common_pc-standard:common_pc-standard
649</literallayout>
650A pull request entails using "git request-pull" to compose an email to the
651maintainer requesting that a branch be pulled into the master repository, see
652http://github.com/guides/pull-requests for an example.
653</para>
654<para>
655Other commands such as 'git stash' or branching can also be used to save
656changes, but are not covered in this document.
657</para>
658<para>
659See the section "importing from another SCM" for how a git push to the
660default_kernel, can be used to automatically update the builds of all users
661of a central git repository.
662</para>
663 </section>
664 </section>
665
666 <section id='export-for-external-upstream-submission'>
667 <title>Export for External (Upstream) Submission</title>
668<para>
669If patches are to be sent for external submission, they can be done via a
670pull request if the patch series is large or the maintainer prefers to pull
671changes. But commonly, patches are sent as email series for easy review and
672integration.
673</para>
674<note><para>
675Before sending patches for review ensure that you understand the
676standard of the community in question and follow their best practices. For
677example, kernel patches should follow standards such as:
678<itemizedlist>
679 <listitem><para><ulink url='http://userweb.kernel.org/~akpm/stuff/tpp.txt'></ulink></para></listitem>
680 <listitem><para><ulink url='http://linux.yyz.us/patch-format.html'></ulink></para></listitem>
681 <listitem><para>Documentation/SubmittingPatches (in any linux kernel source tree)</para></listitem>
682</itemizedlist>
683</para></note>
684<para>
685The messages used to commit changes are a large part of these standards, so
686ensure that the headers for each commit have the required information. If the
687initial commits were not properly documented or don't meet those standards
688rebasing via git rebase -i offer an opportunity to manipulate the commits and
689get them into the required format. Other techniques such as branching and
690cherry picking commits are also viable options.
691</para>
692<para>
693Once complete, patches are sent via email to the maintainer(s) or lists that
694review and integrate changes. "git send-email" is commonly used to ensure
695that patches are properly formatted for easy application and avoid mailer
696induced patch damage.
697</para>
698<para>
699An example of dumping patches for external submission follows:
700<literallayout class='monospaced'>
701 # dump the last 4 commits
702 &gt; git format-patch &dash;&dash;thread -n -o ~/rr/ HEAD^^^^
703 &gt; git send-email &dash;&dash;compose &dash;&dash;subject '[RFC 0/N] &lt;patch series summary&gt;' \
704 &dash;&dash;to foo@yoctoproject.org &dash;&dash;to bar@yoctoproject.org \
705 &dash;&dash;cc list@yoctoproject.org ~/rr
706 # the editor is invoked for the 0/N patch, and when complete the entire
707 # series is sent via email for review
708</literallayout>
709</para>
710 </section>
711
712 <section id='export-for-import-into-other-scm'>
713 <title>Export for Import into Other SCM</title>
714<para>
715Using any one of the previously discussed techniques, commits can be exported
716as patches for import into another SCM. Note however, that if those patches
717are manually applied to a secondary tree and then that secondary tree is
718checked into the SCM, then it often results in lost information (like commit
719logs) and so it is not recommended.
720</para>
721<para>
722Many SCMs can directly import git commits, or can translate git patches to
723not lose information. Those facilities are SCM dependent and should be used
724whenever possible.
725</para>
726 </section>
727 </section>
728
729 <section id='scm-working-with-the-yocto-project-kernel-in-another-scm'>
730 <title>SCM: Working with the Yocto Project Kernel in Another SCM</title>
731<para>
732This is not the same as the exporting of patches to another SCM, but instead
733is concerned with kernel development that is done completely in another
734environment, but built with the Yocto Project build system. In this scenario two
735things must happen:
736<itemizedlist>
737 <listitem><para>The delivered Yocto Project kernel must be exported into the second
738 SCM.</para></listitem>
739 <listitem><para>Development must be exported from that secondary SCM into a
740 format that can be used by the Yocto Project build system.</para></listitem>
741</itemizedlist>
742</para>
743 <section id='exporting-delivered-kernel-to-scm'>
744 <title>Exporting Delivered Kernel to SCM</title>
745<para>
746Depending on the SCM it may be possible to export the entire Yocto Project
747kernel git repository, branches and all, into a new environment. This is the
748preferred method, since it has the most flexibility and potential to maintain
749the meta data associated with each commit.
750</para>
751<para>
752When a direct import mechanism is not available, it is still possible to
753export a branch (or series of branches) and check them into a new
754repository.
755</para>
756<para>
757The following commands illustrate some of the steps that could be used to
758import the common_pc-standard kernel into a secondary SCM
759<literallayout class='monospaced'>
760 &gt; git checkout common_pc-standard
761 &gt; cd .. ; echo linux/.git &gt; .cvsignore
762 &gt; cvs import -m "initial import" linux MY_COMPANY start
763</literallayout>
764The CVS repo could now be relocated and used in a centralized manner.
765</para>
766<para>
767The following commands illustrate how two BSPs could be condensed and merged
768into a second SCM:
769<literallayout class='monospaced'>
770 &gt; git checkout common_pc-standard
771 &gt; git merge cav_ebt5800-standard
772 # resolve any conflicts and commit them
773 &gt; cd .. ; echo linux/.git &gt; .cvsignore
774 &gt; cvs import -m "initial import" linux MY_COMPANY start
775</literallayout>
776</para>
777 </section>
778
779 <section id='importing-changes-for-build'>
780 <title>Importing Changes for Build</title>
781<para>
782Once development has reached a suitable point in the second development
783environment, changes can either be exported as patches or imported into git
784directly (if a conversion/import mechanism is available for the SCM).
785</para>
786<para>
787If changes are exported as patches, they can be placed in a recipe and
788automatically applied to the kernel during patching.
789</para>
790<!--<para>
791If changes are imported directly into git, they must be propagated to the
792wrll-linux-2.6.27/git/default_kernel bare clone of each individual build
793to be present when the kernel is checked out.
794</para>
795<para>
796The following example illustrates one variant of this workflow:
797<literallayout class='monospaced'>
798 # on master git repository
799 &gt; cd linux-2.6.27
800 &gt; git tag -d common_pc-standard-mark
801 &gt; git pull ssh://&lt;foo&gt;@&lt;bar&gt;/pub/git/kernel-2.6.27 common_pc-standard:common_pc-standard
802 &gt; git tag common_pc-standard-mark
803
804 # on each build machine (or NFS share, etc)
805 &gt; cd wrll-linux-2.6.27/git/default_kernel
806 &gt; git fetch ssh://&lt;foo&gt;@&lt;master server&gt;/pub/git/kernel-2.6.27
807
808 # in the build, perform a from-scratch build of Linux and the new changes
809 # will be checked out and built.
810 &gt; make linux
811</literallayout>
812</para> -->
813 </section>
814 </section>
815
816<!-- <section id='bsp-template-migration-from-2'>
817 <title>BSP: Template Migration from 2.0</title>
818<para>
819The move to a git-backed kernel build system in 3.0 introduced a small new
820requirement for any BSP that is not integrated into the GA release of the
821product: branching information.
822</para>
823<para>
824As was previously mentioned in the background sections, branching information
825is always required, since the kernel build system cannot make intelligent
826branching decisions and must rely on the developer. This branching
827information is provided via a .scc file.
828</para>
829<para>
830A BSP template in 2.0 contained build system information (config.sh, etc) and
831kernel patching information in the 'linux' subdirectory. The same holds true
832in 3.0, with only minor changes in the kernel patching directory.
833The ".smudge" files are now ".scc" files and now contain a full description
834 of the kernel branching, patching and configuration for the BSP. Where in
835 2.0, they only contained kernel patching information.
836</para>
837<para>
838The following illustrates the migration of a simple 2.0 BSP template to the
839new 3.0 kernel build system.
840</para>
841<note><para>
842Note: all operations are from the root of a customer layer.
843</para></note>
844<literallayout class='monospaced'>
845 templates/
846 `&dash;&dash; board
847 `&dash;&dash; my_board
848 |&dash;&dash; config.sh
849 |&dash;&dash; include
850 `&dash;&dash; linux
851 `&dash;&dash; 2.6.x
852 |&dash;&dash; knl-base.cfg
853 |&dash;&dash; bsp.patch
854 `&dash;&dash; my_bsp.smudge
855
856 &gt; mv templates/board/my_board/linux/2.6.x/* templates/board/my_board/linux
857 &gt; rm -rf templates/board/my_board/linux/2.6.x/
858 &gt; mv templates/board/my_board/linux/my_bsp.smudge \
859 templates/board/my_board/linux/my_bsp-standard.scc
860 &gt; echo "kconf hardware knl-base.cfg" &gt;&gt; \
861 templates/board/my_board/linux/my_bsp-standard.scc
862 &gt; vi templates/board/my_board/linux/my_bsp-standard.scc
863 # add the following at the top of the file
864 scc_leaf ktypes/standard my_bsp-standard
865
866 templates/
867 `&dash;&dash; board
868 `&dash;&dash; my_board
869 |&dash;&dash; config.sh
870 |&dash;&dash; include
871 `&dash;&dash; linux
872 |&dash;&dash; knl-base.cfg
873 |&dash;&dash; bsp.patch
874 `&dash;&dash; my_bsp-standard.scc
875</literallayout>
876<para>
877That's it. Configure and build.
878</para>
879<note><para>There is a naming convention for the .scc file, which allows the build
880 system to locate suitable feature descriptions for a board:
881</para></note>
882<literallayout class='monospaced'>
883 &lt;bsp name&gt;-&lt;kernel type&gt;.scc
884</literallayout>
885<para>
886 if this naming convention isn't followed your feature description will
887 not be located and a build error thrown.
888</para>
889 </section> -->
890
891
892
893 <section id='bsp-creating'>
894 <title>BSP: Creating</title>
895 <para>
896 This section provides an example for creating a BSP based on an existing, and hopefully,
897 similar one.
898 Follow these steps and keep in mind your particular situation and differences:
899 <orderedlist>
900 <listitem><para>Get a machine configuration file that matches your machine.</para>
901 <para>You can start with something in <filename>meta/conf/machine</filename>.
902 Or, <filename>meta-emenlow/conf/machine</filename> has an example in its own layer.</para>
903 <para>The most up-to-date machines that are probably most similar to yours and that you might want
904 to look at are <filename>meta/conf/machine/atom-pc.conf</filename> and
905 <filename>meta-emenlow/conf/machine/emenlow.conf</filename>.
906 Both of these were either just added or upgraded to use the Yocto Project kernel
907 at <ulink url='http://git.pokylinux.org/cgit/cgit.cgi/linux-2.6-windriver/'></ulink>.
908 The main difference between them is that "emenlow" is in its own layer.
909 It is in its own layer because it needs extra machine-specific packages such as its
910 own video driver and other supporting packages.
911 The "atom-pc" is simpler and does not need any special packages - everything it needs can
912 be specified in the configuration file.
913 The "atom-pc" machine also supports all of Asus eee901, Acer Aspire One, Toshiba NB305,
914 and the Intel&reg; Embedded Development Board 1-N450 with no changes.</para>
915 <para>If you want to make minor changes to support a slightly different machine, you can
916 create a new configuration file for it and add it alongside the others.
917 You might consider keeping the common stuff separate and including it.</para>
918 <para>Similarly, you can also use multiple configuration files for different machines even
919 if you do it as a separate layer like meta-emenlow.</para>
920 <para>As an example consider this:
921 <itemizedlist>
922 <listitem><para>Copy meta-emenlow</para></listitem>
923 <listitem><para>Fix or remove anything you do not need.
924 For this example the only thing left was the kernel directory with a linux-yocto_git.bbappend
925 file (linux-yocto is the kernel listed in
926 <filename>meta-crownbay/conf/machine/crownbay.conf</filename>.
927 Finally, a new entry to the <filename>build/donf/bblayers.conf</filename> was added so the
928 new layer could be found by Bitbake.</para></listitem>
929 </itemizedlist>
930 </para></listitem>
931 <listitem><para>Get an image with a working kernel built.</para>
932 <para>For the kernel to compile successfully, you need to create a branch in the git repository
933 specifically named for your machine.
934 So first create a bare clone of the Yocto Project git repository, and then create a
935 local clone of that:
936 <literallayout class='monospaced'>
937 $ git clone &dash;&dash;bare git://git.pokylinux.org/linux-2.6-windriver.git
938 linux-2.6-windriver.git
939 $ git clone linux-2.6-windriver.git linux-2.6-windriver
940 </literallayout>
941 </para>
942 <para>Now create a branch in the local clone and push it to the bare clone:
943 <literallayout class='monospaced'>
944 $ git checkout -b crownbay-standard origin/standard $ git push origin crownbay-standard:crownbay-standard
945 </literallayout>
946 </para>
947 <para>At this point, your git tree should be set up well enough to compile.</para></listitem>
948 <listitem><para>Point the build at the new kernel git tree.</para>
949 <para>You can do this by commenting out the SRC_URI variable in
950 <filename>meta/recipes-kernel/linux/linux-yocto_git.bb</filename> and using a SRC_URI
951 that points to your new bare git tree.
952 You should also be able to do this in <filename>linux-yocto_git.bbappend</filename> in the layer:
953 <literallayout class='monospaced'>
954 # To use a staged, on-disk bare clone of a Wind River Kernel, use a variant of the
955 # below SRC_URI = "git://///path/to/kernel/default_kernel.git;fullclone=1"
956 #
957 SRC_URI = "git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;fullclone=1;branch=${KBRANCH};name=machine
958\
959 git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;noclone=1;branch=wrs_meta;name=meta"
960 </literallayout>
961 </para>
962 <para>After doing that, select the machine in <filename>build/conf/local.conf</filename>:
963 <literallayout class='monospaced'>
964 #
965 MACHINE ?= "crownbay"
966 #
967 </literallayout>
968 </para>
969 <para>You should now be able to build and boot an image with the new kernel:
970 <literallayout class='monospaced'>
971 $ bitbake poky-image-sato-live
972 </literallayout>
973 </para>
974 <para>Of course, that will give you a kernel with the default config, which is probably
975 not what you want.
976 If you just want to set some kernel config options, you can do that by putting them in a files.
977 For example inserting the following into some <filename>.cfg</filename> file:
978 <literallayout class='monospaced'>
979 CONFIG_NETDEV_1000=y
980 CONFIG_E1000E=y
981 </literallayout>
982 </para>
983 <para>And, another <filename>.cfg</filename> file would contain:
984 <literallayout class='monospaced'>
985 CONFIG_LOG_BUF_SHIFT=18
986
987 http://git.pokylinux.org/cgit/cgit.cgi/linux-2.6-windriver/
988
989 SRC_URI_append_crownbay = " file://some.cfg \
990 file://other.cfg \
991 "
992 </literallayout>
993 </para>
994 <para>You could also add these directly to the git repo's wrs_meta branch as well.
995 However, the former method is probably easier.</para></listitem>
996 <listitem><para>If you're also adding patches to the kernel, you can do the same thing.
997 Put your patches in the SRC_URI as well (plus .cfg for their kernel config options if needed).</para>
998 <para>Practically speaking, to generate the patches, you'd go to the source in the build tree:
999 <literallayout class='monospaced'>
1000 build/tmp/work/crownbay-poky-linux/linux-yocto-2.6.34+git0+d1cd5c80ee97e81e130be8c3de3965b770f320d6_0+
10010431115c9d720fee5bb105f6a7411efb4f851d26-r13/linux
1002 </literallayout>
1003 </para>
1004 <para>Then, modify the code there, using quilt to save the changes, and recompile
1005 (bitbake -c compile -f)
1006 until it works.</para></listitem>
1007 <listitem><para>Once you have the final patch from quilt, copy it to the
1008 SRC_URI location, and it should be
1009 applied the next time you do a clean build.
1010 Of course, since you have a branch for the BSP in git, it would be better to put it there instead.
1011 For example, in this case, commit the patch to the crownbay-standard branch, and during the
1012 next build it will be applied from there.</para></listitem>
1013 </orderedlist>
1014 </para>
1015 </section>
1016
1017
1018
1019
1020
1021
1022
1023
1024<!-- <section id='bsp-creating-a-new-bsp'>
1025 <title>BSP: Creating a New BSP</title>
1026<para>
1027Although it is obvious that the structure of a new BSP uses the migrated
1028directory structure from the previous example,the first question is whether
1029or not the BSP is started from scratch.
1030</para>
1031<para>
1032If Yocto Project has a similar BSP, it is often easier to clone and update,
1033rather than start from scratch. If the mainline kernel has support, it is
1034easier to branch from the -standard kernel and begin development (and not be
1035concerned with undoing existing changes). This section covers both options.
1036</para>
1037<para>
1038In almost every scenario, the LDAT build system bindings must be completed
1039before either cloning or starting a new BSP from scratch. This is simply
1040because the board template files are required to configure a project/build
1041and create the necessary environment to begin working directly with the
1042kernel. If it is desired to start immediately with kernel development and
1043then add LDAT bindings, see the "bootstrapping a BSP" section.
1044</para>
1045 <section id='creating-from-scratch'>
1046 <title>Creating the BSP from Scratch</title>
1047<para>
1048To create the BSP from scratch you need to do the following:
1049<orderedlist>
1050 <listitem><para>Create a board template for the new BSP in a layer.</para></listitem>
1051 <listitem><para>Configure a build with the board.</para></listitem>
1052 <listitem><para>Configure a kernel.</para></listitem>
1053</orderedlist>
1054</para>
1055<para>
1056Following is an example showing all three steps. You start by creating a board template for the new BSP in a layer.
1057<literallayout class='monospaced'>
1058 templates/
1059 `&dash;&dash; board
1060 `&dash;&dash; my_bsp
1061 |&dash;&dash; include
1062 |&dash;&dash; config.sh
1063 `&dash;&dash; linux
1064 |&dash;&dash; my_bsp.cfg
1065 `&dash;&dash; my_bsp-standard.scc
1066
1067 &gt; cat config.sh
1068 TARGET_BOARD="my_bsp"
1069 TARGET_LINUX_LINKS="bzImage"
1070 TARGET_SUPPORTED_KERNEL="standard"
1071 TARGET_SUPPORTED_ROOTFS="glibc_std"
1072 BANNER="This BSP is *NOT* supported"
1073 TARGET_PROCFAM="pentium4"
1074 TARGET_PLATFORMS="GPP"
1075
1076 &gt; cat include
1077 cpu/x86_32_i686
1078 karch/i386
1079
1080 &gt; cat linux/my_bsp-standard.scc
1081 scc_leaf ktypes/standard/standard.scc my_bsp-standard
1082
1083 &gt; cat linux/my_bsp.cfg
1084 CONFIG_X86=y
1085 CONFIG_SMP=y
1086 CONFIG_VT=y
1087 # etc, etc, etc
1088</literallayout>
1089</para>
1090<para>
1091Something like the following can now be added to a board build, and
1092a project can be started:
1093<literallayout class='monospaced'>
1094 &dash;&dash;enable-board=my_bsp \
1095 &dash;&dash;with-layer=custom_bsp
1096</literallayout>
1097</para>
1098<para>
1099Now you can configure a kernel:
1100<literallayout class='monospaced'>
1101 &gt; make -C build linux.config
1102</literallayout>
1103</para>
1104<para>
1105You now have a kernel tree, which is branched and has no patches, ready for
1106development.
1107</para>
1108 </section> -->
1109
1110<!-- <section id='cloning-an-existing-bsp'>
1111 <title>Cloning an Existing BSP</title>
1112<para>
1113Cloning an existing BSP from the shipped product is similar to the "from
1114scratch" option and there are two distinct ways to achieve this goal:
1115<itemizedlist>
1116 <listitem><para>Create a board template for the new BSP in a layer.</para></listitem>
1117 <listitem><para>Clone the .scc and board config.</para></listitem>
1118</itemizedlist>
1119</para>
1120<para>
1121The first method is similar to the from scratch BSP where you create a board template for the new
1122BSP. Although in this case, copying an existing board template from
1123wrll-wrlinux/templates/board would be appropriate, since we are cloning an
1124existing BSP. Edit the config.sh, include and other board options for the new
1125BSP.
1126</para>
1127<para>
1128The second method is to clone the .scc and board config.
1129To do this, in the newly created board template, create a linux subdirectory and export
1130the .scc and configuration from the source BSP in the published Yocto Project
1131kernel. During construction, all of the configuration and patches were
1132captured, so it is simply a matter of extracting them.
1133</para>
1134<para>
1135Extraction can be accomplished using four different techniques:
1136<itemizedlist>
1137 <listitem><para>Config and patches from the bare default_kernel.</para></listitem>
1138 <listitem><para>Clone default_kernel and checkout wrs_base.</para></listitem>
1139 <listitem><para>Clone default_kernel and checkout BSP branch.</para></listitem>
1140 <listitem><para>Branch from the Yocto Project BSP.</para></listitem>
1141</itemizedlist>
1142</para>
1143<para>
1144Technique 1: config and patches from the bare default_kernel
1145<literallayout class='monospaced'>
1146 &gt; cd layers/wrll-linux-2.6.27/git/default_kernel
1147 &gt; git show checkpoint_end | filterdiff -i '*common_pc*' | patch -s -p2 -d /tmp
1148
1149 # This will create two directories: cfg and patches.
1150
1151 &gt; cd /tmp/cfg/kernel-cache/bsp/common_pc/
1152
1153 # This directory contains all the patches and .scc files used to construct
1154 # the BSP in the shipped tree. Copy the patches to the new BSP template,
1155 # and add them to the .scc file created above. See "template patching" if
1156 # more details are required.
1157</literallayout>
1158</para>
1159<para>
1160Technique 2: clone default_kernel and checkout wrs_base
1161<literallayout class='monospaced'>
1162 &gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
1163 &gt; cd windriver-2.6.27
1164 &gt; git checkout wrs_base
1165 &gt; cd wrs/cfg/kernel-cache/bsp/common_pc
1166
1167# again, this directory has all the patches and .scc files used to construct
1168# the BSP
1169</literallayout>
1170</para>
1171<para>
1172Technique 3: clone default_kernel and checkout BSP branch
1173<literallayout class='monospaced'>
1174 &gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
1175 &gt; cd windriver-2.6.27
1176 &gt; git checkout common_pc-standard
1177 &gt; git whatchanged
1178 # browse patches and determine which ones are of interest, say there are
1179 # 3 patches of interest
1180 &gt; git format-patch -o &lt;path to BSP template&gt;/linux HEAD^^^
1181 # update the .scc file to add the patches, see "template patches" if
1182 # more details are required
1183</literallayout>
1184</para>
1185<para>
1186Technique #4: branch from the Yocto Project BSP
1187<note><para>This is potentially the most "different" technique, but is actually
1188 the easiest to support and leverages the infrastructure. rtcore BSPs
1189 are created in a similar manner to this.
1190</para></note>
1191</para>
1192<para>
1193In this technique the .scc file in the board template is slightly different
1194 and indicates that the BSP should branch after the base Yocto Project BSP
1195 of the correct kernel type, so to start a new BSP that inherits the
1196 kernel patches of the common_pc-standard, the following would be done:
1197<literallayout class='monospaced'>
1198 &gt; cat linux/my_bsp-standard.scc
1199 scc_leaf bsp/common_pc/common_pc-standard.scc my_bsp-standard
1200</literallayout>
1201</para>
1202<para>
1203 And only kernel configuration (not patches) need be contained in the
1204 board template.
1205</para>
1206<para>
1207 This has the advantage of automatically picking up updates to the BSP
1208 and not duplicating any patches for a similar board.
1209</para>
1210 </section> -->
1211
1212 <!-- <section id='bsp-bootstrapping'>
1213 <title>BSP: Bootstrapping</title>
1214<para>
1215The previous examples created the board templates and configured a build
1216before beginning work on a new BSP. It is also possible for advanced users to
1217simply treat the Yocto Project git repository as an upstream source and begin
1218BSP development directly on the repository. This is the closest match to how
1219the kernel community at large would operate.
1220</para>
1221<para>
1222Two techniques exist to accomplish this:
1223</para>
1224<para>
1225Technique 1: upstream workflow
1226<literallayout class='monospaced'>
1227 &gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
1228 &gt; cd windriver-2.6.27
1229 &gt; git checkout -b my_bsp-standard common_pc-standard
1230
1231 # edit files, import patches, generally do BSP development
1232
1233 # at this point we can create the BSP template, and export the kernel
1234 # changes using one of the techniques discussed in that section. For
1235 # example, It is possible to push these changes, directly into the
1236 # default_kernel and never directly manipulate or export patch files
1237</literallayout>
1238</para>
1239<para>
1240Technique 2: Yocto Project kernel build workflow
1241</para>
1242<para>
1243 Create the BSP branch from the appropriate kernel type
1244<literallayout class='monospaced'>
1245 &gt; cd linux
1246 # the naming convention for auto-build is &lt;bsp&gt;-&lt;kernel type&gt;
1247 &gt; git checkout -b my_bsp-standard standard
1248</literallayout>
1249</para>
1250<para>
1251Make changes, import patches, etc.
1252<literallayout class='monospaced'>
1253 &gt; ../../host-cross/bin/guilt init
1254 # 'wrs/patches/my_bsp-standard' has now been created to
1255 # manage the branches patches
1256
1257 # option 1: edit files, guilt import
1258 &gt; ../../host-cross/bin/guilt new extra-version.patch
1259 &gt; vi Makefile
1260 &gt; ../../host-cross/bin/guilt refresh
1261 # add a header
1262 &gt; ../../host-cross/bin/guilt header -e
1263 # describe the patch using best practices, like the example below:
1264
1265 &dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt; cut here
1266 From: Bruce Ashfield &lt;bruce.ashfield@windriver.com&gt;
1267
1268 Adds an extra version to the kernel
1269
1270 Modify the main EXTRAVERSION to show our bsp name
1271
1272 Signed-off-by: Bruce Ashfield &lt;bruce.ashfield@windriver.com&gt;
1273 &dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt; cut here
1274
1275 # option 2: import patches
1276 &gt; git am &lt;patch&gt;
1277 or
1278 &gt; git apply &lt;patch&gt;
1279 &gt; git add &lt;files&gt;
1280 &gt; git commit -s
1281
1282 # configure the board, save relevant options
1283 &gt; make ARCH=&lt;arch&gt; menuconfig
1284
1285 # save the cfg changes for reconfiguration
1286 &gt; mkdir wrs/cfg/&lt;cache&gt;/my_bsp
1287 &gt; vi wrs/cfg/&lt;cache&gt;/my_bsp/my_bsp.cfg
1288
1289 # classify the patches
1290 &gt; ../../host-cross/bin/kgit classify create &lt;kernel-foo-cache&gt;/my_bsp/my_bsp
1291 # test build
1292 &gt; cd ..
1293 &gt; make linux TARGET_BOARD=my_bsp kprofile=my_bsp use_current_branch=1
1294</literallayout>
1295</para>
1296<para>
1297 Assuming the patches have been exported to the correct location, Future
1298 builds will now find the board, apply the patches to the base tree and make
1299 the relevant branches and structures and the special build options are no
1300 longer required.
1301</para>
1302 </section>
1303 </section> -->
1304
1305<!-- <section id='patching'>
1306 <title>Patching</title>
1307<para>
1308The most common way to apply patches to the kernel is via a template.
1309However, for more advanced applications (such as the sharing of patches between
1310multiple sub-features) it is possible to patch the kernel-cache.
1311This section covers both scenarios.
1312</para>
1313 <section id='patching-template'>
1314 <title>Patching: Template</title>
1315<para>
1316kernel
1317templates follow the same rules as any LDAT template. A directory should be
1318created in a recognized template location, with a 'linux' subdirectory. The
1319'linux' directory triggers LDAT to pass the dir as a potential patch location
1320to the kernel build system. Any .scc files found in that directory, will be
1321automatically appended to the end of the BSP branch (for the configured
1322board).
1323</para>
1324<para>
1325This behavior is essentially the same since previous product
1326releases. The only exception is the use of ".scc", which allows kernel
1327configuration AND patches to be applied in a template.
1328</para>
1329<note><para>
1330If creating a full template is not required, a .scc file can be placed at
1331the top of the build, along with configuration and patches. The build
1332system will pickup the .scc and add it onto the patch list automatically
1333</para></note>
1334<para>
1335As an example, consider a simple template to update a BP:
1336<literallayout class='monospaced'>
1337 &gt; cat templates/feature/extra_version/linux/extra_version.scc
1338 patch 0001-extraversion-add-Wind-River-identifier.patch
1339</literallayout>
1340</para>
1341<para>
1342To illustrate how the previous template patch was created, the following
1343steps were performed:
1344<literallayout class='monospaced'>
1345 &gt; cd &lt;board build&gt;/build/linux
1346 &gt; vi Makefile
1347 # modify EXTRAVERSION to have a unique string
1348 &gt; git commit -s -m "extraversion: add Yocto Project identifier" Makefile
1349 &gt; git format-patch -o &lt;path to layer&gt;/templates/feature/extra_version/linux/
1350 &gt; echo "patch 0001-extraversion-add-Wind-River-identifier.patch" &gt; \
1351 &lt;path to layer&gt;/templates/feature/extra_version/linux/extra_version.scc
1352</literallayout>
1353</para>
1354<para>
1355This next example creates a template with a linux subdirectory, just as we
1356 always have for previous releases.
1357<literallayout class='monospaced'>
1358 &gt; mkdir templates/features/my_feature/linux
1359</literallayout>
1360</para>
1361<para>
1362 In that directory place your feature description, your
1363 patch and configuration (if required).
1364<literallayout class='monospaced'>
1365 &gt; ls templates/features/my_feature/linux
1366
1367 version.patch
1368 my_feature.scc
1369 my_feature.cfg
1370</literallayout>
1371</para>
1372<para>
1373 The .scc file describes the patches, configuration and
1374 where in the patch order the feature should be inserted.
1375<literallayout class='monospaced'>
1376 patch version.patch
1377 kconf non-hardware my_feature.cfg
1378</literallayout>
1379</para>
1380<para>
1381 Configure your build with the new template
1382<literallayout class='monospaced'>
1383 &dash;&dash;with-template=features/my_feature
1384</literallayout>
1385</para>
1386<para>
1387Build the kernel
1388<literallayout class='monospaced'>
1389 &gt; make linux
1390</literallayout>
1391</para>
1392 </section>
1393
1394 <section id='patching-kernel-cache'>
1395 <title>Patching: Kernel Cache</title>
1396<para>
1397As previously mentioned, this example is included for completeness, and is for more advanced
1398applications (such as the sharing of patches between multiple sub-features).
1399Most patching should be done via templates, since that interface is
1400guaranteed not to change and the kernel-cache interface carries no such
1401guarantee.
1402</para>
1403<para>
1404At the top of a layer, create a kernel cache. The build system will recognize
1405any directory of the name 'kernel-*-cache' as a kernel cache.
1406<literallayout class='monospaced'>
1407 &gt; cd &lt;my layer&gt;
1408 &gt;mkdir kernel-temp-cache
1409</literallayout>
1410</para>
1411<para>
1412Make a directory with the BSP
1413<literallayout class='monospaced'>
1414 &gt; mkdir kernel-temp-cache
1415 &gt; mkdir kernel-temp-cache/my_feat
1416</literallayout>
1417</para>
1418<para>
1419Create the feature files as they were in technique #1
1420<literallayout class='monospaced'>
1421 &gt; echo "patch my_patch.path" &gt; kernel-temp-cache/my_feat/my_feature.scc
1422</literallayout>
1423</para>
1424<para>
1425Configure the build with the feature added to the kernel type
1426<literallayout class='monospaced'>
1427 &dash;&dash;with-kernel=standard+my_feat/my_feature.scc
1428</literallayout>
1429</para>
1430<para>
1431Build the kernel
1432<literallayout class='monospaced'>
1433 &gt; make linux
1434</literallayout>
1435</para>
1436 </section>
1437 </section>
1438
1439 <section id='bsp-updating-patches-and-configuration'>
1440 <title>BSP: Updating Patches and Configuration</title>
1441<para>
1442As was described in the "template patching" example, it is simple
1443to add patches to a BSP via a template, but often, it is desirable
1444to experiment and test patches before committing them to a template.
1445You can do this by modifying the BSP source.
1446</para>
1447<para>
1448Start as follows:
1449<literallayout class='monospaced'>
1450 &gt; cd linux
1451 &gt; git checkout &lt;bspname&gt;-&lt;kernel name&gt;
1452
1453 &gt; git am &lt;patch&gt;
1454</literallayout>
1455</para>
1456<para>
1457Or you can do this:
1458<literallayout class='monospaced'>
1459 &gt; kgit-import -t patch &lt;patch&gt;
1460
1461 &gt; cd ..
1462 &gt; make linux
1463</literallayout>
1464</para>
1465<para>
1466For details on conflict resolution and patch application, see the
1467git manual, or other suitable online references.
1468<literallayout class='monospaced'>
1469 &gt; git am &lt;mbox&gt;
1470 # conflict
1471 &gt; git apply &dash;&dash;reject .git/rebase-apply/0001
1472 # resolve conflict
1473 &gt; git am &dash;&dash;resolved (or git am &dash;&dash;skip, git am &dash;&dash;abort)
1474 # continue until complete
1475</literallayout>
1476</para>
1477<para>
1478Here is another example:
1479<literallayout class='monospaced'>
1480 # merge the patches
1481 # 1) single patch
1482 &gt; git am &lt;mbox&gt;
1483 &gt; git apply &lt;patch&lt;
1484 &gt; kgit import -t patch &lt;patch&gt;
1485
1486 # 2) multiple patches
1487 &gt; git am &lt;mbox&gt;
1488 &gt; kgit import -t dir &lt;dir&gt;
1489
1490 # if kgit -t dir is used, a patch resolution cycle such
1491 # as this can be used:
1492
1493 &gt; kgit import -t dir &lt;dir&gt;
1494 # locate rejects and resolve
1495 # options:
1496 &gt; wiggle &dash;&dash;replace &lt;path to file&gt; &lt;path to reject&gt;
1497 &gt; guilt refresh
1498 or
1499 &gt; # manual resolution
1500 &gt; git add &lt;files&gt;
1501 &gt; git commit -s
1502 or
1503 &gt; git apply &dash;&dash;reject .git/rebase-apply/0001
1504 &gt; git add &lt;files&gt;
1505 &gt; git am &dash;&dash;resolved
1506 or
1507 &gt; # merge tool of choice
1508
1509 # continue series:
1510
1511 &gt; kgit import -t dir &lt;dir&gt;
1512 or
1513 &gt; git am &dash;&dash;continue
1514</literallayout>
1515</para>
1516<para>
1517Once all the patches have been tested and are satisfactory, they
1518should be exported via the techniques described in "saving kernel
1519modifications."
1520</para>
1521<para>
1522Once the kernel has been patched and configured for a BSP, it's
1523configuration commonly needs to be modified. This can be done by
1524running [menu|x]config on the kernel tree, or working with
1525configuration fragments.
1526</para>
1527<para>
1528Using menuconfig, the operation is as follows:
1529<literallayout class='monospaced'>
1530 &gt; make linux.menuconfig
1531 &gt; make linux.rebuild
1532</literallayout>
1533</para>
1534<para>
1535Once complete, the changes are in linux-&lt;bsp&gt;-&lt;kernel type&gt;-build/.config.
1536To permanently save these changes, compare the .config before and after the
1537menuconfig, and place those changes in a configuration fragment in the
1538template of your choice.
1539</para>
1540<para>
1541Using configuration fragments, the operation is as follows (using the
1542si_is8620 as an example BSP):
1543<literallayout class='monospaced'>
1544 &gt; vi linux/wrs/cfg/kernel-cache/bsp/si_is8620/si_is8620.cfg
1545 &gt; make linux.reconfig
1546 &gt; make linux.rebuild
1547</literallayout>
1548</para>
1549<para>
1550The modified configuration fragment can simply be copied out of the
1551linux/wrs/.. directory and placed in the appropriate template for future
1552application.
1553</para>
1554 </section>
1555
1556 <section id='tools-guilt'>
1557 <title>Tools: guilt</title>
1558<para>
1559Yocto Project has guilt integrated as a kernel tool; therefore users that are
1560familiar with quilt may wish to use this tool to pop, push and refresh
1561their patches. Note: guilt should only be used for local operations, once
1562a set of changes has been pushed or pulled, they should no longer be popped
1563or refresh by guilt, since popping, refreshing and re-pushing patches
1564changes their commit IDs and creating non-fast forward branches.
1565</para>
1566<para>
1567The following example illustrates how to add patches a Yocto Project
1568BSP branch via guilt:
1569<literallayout class='monospaced'>
1570 &gt; cd build/linux
1571 &gt; git checkout common_pc-standard
1572 &gt; guilt new extra.patch
1573 # edit files, make changes, etc
1574 &gt; guilt refresh
1575 &gt; guilt top
1576 extra.patch
1577
1578 # export that patch to an external location
1579 &gt; kgit export -p top /tmp
1580</literallayout>
1581</para>
1582<para>
1583Other guilt operations of interest are:
1584<literallayout class='monospaced'>
1585 > guilt push, guilt push -a
1586 > guilt pop
1587 > guilt applied, guilt unapplied
1588 > guilt top
1589 > guilt refresh
1590 > guilt header -e
1591 > guilt next
1592</literallayout>
1593</para>
1594<note><para>
1595Guilt only uses git commands and git plumbing to perform its operations,
1596anything that guilt does can also be done using git directly. It is provided
1597as a convenience utility, but is not required and the developer can use whatever
1598tools or workflow they wish.
1599</para></note>
1600<para>
1601The following builds from the above instructions to show how guilt can be
1602used to assist in getting your BSP kernel patches ready. You should follow
1603the above instructions up to and including 'make linux.config'. In this
1604example I will create a new commit (patch) from scratch and import another
1605fictitious patch from some external public git tree (ie, a commit with full
1606message, signoff etc.). Please ensure you have host-cross/bin in your path.
1607<literallayout class='monospaced'>
1608 %> cd linux
1609 %> guilt-init
1610 %> guilt-new -m fill_me_in_please first_one.patch
1611 %> touch somefile.txt
1612 %> guilt-add somefile.txt
1613 %> guilt-header -e
1614 %> guilt-refresh
1615 %> guilt-import path_to_some_patch/patch_filename
1616 %> guilt-push
1617</literallayout>
1618</para>
1619<para>
1620Here are a few notes about the above:
1621<itemizedlist>
1622 <listitem><para>guilt-header -e &dash;&dash; this will open editing of the patch header in
1623 EDITOR. As with a git commit the first line is the short log and
1624 should be just that short and concise message about the commit. Follow
1625 the short log with lines of text that will be the long description but
1626 note Do not put a blank line after the short log. As usual you will
1627 want to follow this with a blank line and then a signoff line.</para></listitem>
1628
1629 <listitem><para>The last line in the example above has 2 dots on the end. If you
1630 don't add the 2 periods on the end guilt will think you are sending
1631 just one patch. The wrong one!</para></listitem>
1632
1633 <listitem><para>The advantage to using guilt over not using guilt is that if you have a
1634 review comment in the first patch (first_one.patch in the case of this
1635 example) it is very easy to use guilt to pop the other patches off
1636 allowing you to make the necessary changes without having to use more
1637 inventive git type strategies.</para></listitem>
1638</itemizedlist>
1639</para>
1640 </section>
1641
1642 <section id='tools-scc-file-example'>
1643 <title>Tools: scc File Example</title>
1644<para>
1645This section provides some scc file examples: leaf node, 'normal' mode, and transforms.
1646</para>
1647 <section id='leaf-node'>
1648 <title>Leaf Node</title>
1649<para>
1650The following example is a BSP branch with no child branches - a leaf on the tree.
1651<literallayout class='monospaced'>
1652 # these are optional, but allow standalone tree construction
1653 define WRS_BOARD &lt;name&gt;
1654 define WRS_KERNEL &lt;kern type&gt;
1655 define WRS_ARCH &lt;arch&gt;
1656
1657 scc_leaf ktypes/standard common_pc-standard
1658 # ^ ^
1659 # +&dash;&dash; parent + branch name
1660
1661 include common_pc.scc
1662 # ^
1663 # +&dash;&dash;&dash; include another feature
1664</literallayout>
1665</para>
1666 </section>
1667
1668 <section id='normal-mode'>
1669 <title>'Normal' Mode</title>
1670<para>
1671Here is an example of 'normal' mode:
1672<literallayout class='monospaced'>
1673 # +&dash;&dash;&dash;&dash; name of file to read
1674 # v
1675 kconf hardware common_pc.cfg
1676 # ^ ^
1677 # | +&dash;&dash; 'type: hardware or non-hardware
1678 # |
1679 # +&dash;&dash;&dash; kernel config
1680
1681 # patches
1682 patch 0002-atl2-add-atl2-driver.patch
1683 patch 0003-net-remove-LLTX-in-atl2-driver.patch
1684 patch 0004-net-add-net-poll-support-for-atl2-driver.patch
1685</literallayout>
1686</para>
1687
1688 </section>
1689
1690 <section id='transforms'>
1691 <title>Transforms</title>
1692<para>
1693This section shows an example of transforms:
1694<literallayout class='monospaced'>
1695 # either of the next two options will trigger an 'auto'
1696 # branch from existing ones, since they change the commit
1697 # order and hence must construct their own branch
1698
1699 # this changes the order of future includes, if the
1700 # passed feature is detected, the first feature is
1701 # included AFTER it
1702 include features/rt/rt.scc after features/kgdb/kgdb
1703 # this also changes the order of existing branches
1704 # this prevents the named feature from ever being
1705 # included
1706 exclude features/dynamic_ftrace/dynamic_ftrace.scc
1707
1708 # inherit the standard kernel
1709 include ktypes/standard/standard
1710
1711
1712 # LTT supplies this, so we don't want the sub-chunk from RT.
1713 patch_trigger arch:all exclude ftrace-upstream-tracepoints.patch
1714 # ...but we still want the one unique tracepoint it added.
1715 patch tracepoint-add-for-sched_resched_task.patch
1716
1717 # these will change the named patches in the series into
1718 # &lt;patch name&gt;.patch.&lt;feature name&gt;
1719 # where the substituted patch is in this directory
1720 patch_trigger arch:all ctx_mod dynamic_printk.patch
1721 patch_trigger arch:all ctx_mod 0001-Implement-futex-macros-for-ARM.patch
1722 # unconditionally exclude a patch
1723 patch_trigger arch:all exclude ftrace-fix-ARM-crash.patch
1724</literallayout>
1725</para>
1726 </section>
1727 </section> -->
1728
1729 <section id='tip-dirty-string'>
1730 <title>"-dirty" String</title>
1731<para>
1732If kernel images are being built with -dirty on the end of the version
1733string, this simply means that there are modification in the source
1734directory that haven't been committed.
1735<literallayout class='monospaced'>
1736 &gt; git status
1737</literallayout>
1738</para>
1739<para>
1740The above git command will indicate modified, removed or added files. Those changes should
1741be committed to the tree (even if they will never be saved, or exported
1742for future use) and the kernel rebuilt.
1743</para>
1744<para>
1745To brute force pickup and commit all such pending changes enter the following:
1746<literallayout class='monospaced'>
1747 &gt; git add .
1748 &gt; git commit -s -a -m "getting rid of -dirty"
1749</literallayout>
1750</para>
1751<para>
1752And then rebuild the kernel
1753</para>
1754 </section>
1755
1756 <section id='kernel-transition-kernel-layer'>
1757 <title>Kernel: Transition Kernel Layer</title>
1758<para>
1759In order to temporarily use a different base kernel in Yocto Project
1760Linux 3.0 you need to do the following:
1761<orderedlist>
1762 <listitem><para>Create a custom kernel layer.</para></listitem>
1763 <listitem><para>Create a git repository of the transition kernel.</para></listitem>
1764</orderedlist>
1765</para>
1766<para>
1767Once those requirements are met multiple boards and kernels can
1768be built. The cost of setup is only paid once and then additional
1769BSPs and options can be added.
1770</para>
1771<para>
1772This creates a transition kernel layer to evaluate functionality
1773of some other kernel with the goal of easing transition to an
1774integrated and validated Yocto Project kernel.
1775</para>
1776<!--<para>
1777The next few sections describe the process:
1778</para> -->
1779 <!-- <section id='creating-a-custom-kernel-layer'>
1780 <title>Creating a Custom Kernel Layer</title>
1781<para>
1782The custom kernel layer must have the following minimum
1783elements:
1784<itemizedlist>
1785 <listitem><para>An include of the shipped Yocto Project kernel layer.</para></listitem>
1786 <listitem><para>A kernel-cache with an override of the standard kernel type.</para></listitem>
1787</itemizedlist>
1788</para>
1789<para>
1790This allows the inheritance of the kernel build infrastructure,
1791while overriding the list of patches that should be applied to
1792the base kernel.
1793</para>
1794<para>
1795The kernel layer can optionally include an override to the base
1796Yocto Project Linux BSP to inhibit the application of BSP specific
1797patches. If a custom BSP is being used, this is not required.
1798</para>
1799 </section> -->
1800
1801 <!-- <section id='git-repo-of-the-transition-kernel'>
1802 <title>git Repo of the Transition Kernel</title>
1803<para>
1804The kernel build system requires a base kernel repository to
1805seed the build process. This repository must be found in the
1806same layer as the build infrastructure (i.e wrll-linux-2.6.27)
1807in the 'git' subdir, with the name 'default_kernel'
1808</para>
1809<para>Since Yocto Project Linux ships with a default_kernel
1810(the validated Yocto Project kernel) in the wrll-linux-2.6.27
1811kernel layer, that must be removed and replaced with the
1812transition kernel.
1813</para>
1814<para>If the Yocto Project install cannot be directly modified
1815with the new default kernel, then the path to the transition
1816kernel layer's 'git' subdir must be passed to the build
1817process via:
1818<programlisting>
1819linux_GIT_BASE=&lt;absolute path to layer&gt;/git
1820</programlisting>
1821</para>
1822<para>
1823If the transition kernel has not been delivered via git,
1824then a git repo should be created, and bare cloned into
1825place. Creating this repository is as simple as:
1826<literallayout class='monospaced'>
1827 &gt; tar zxvf temp_kernel.tgz
1828 &gt; cd temp_kernel
1829 &gt; git init
1830 &gt; git add .
1831 &gt; git commit -a -m "Transition kernel baseline"
1832
1833 'temp_kernel' can now be cloned into place via:
1834
1835 &gt; cd &lt;path to git base&gt;/git
1836 &gt; git clone &dash;&dash;bare &lt;path to temp_kernel/temp_kernel default_kernel
1837</literallayout>
1838</para>
1839 </section> -->
1840
1841 <!-- <section id='building-the-kernel'>
1842 <title>Building the Kernel</title>
1843<para>
1844Once these prerequisites have been met, the kernel can be
1845built with:
1846<literallayout class='monospaced'>
1847 &gt; make linux
1848</literallayout>
1849</para>
1850<para>
1851The new base kernel will be cloned into place and have any patches
1852indicated in the transition kernel's cache (or templates) applied.
1853The kernel build will detect the non-Yocto Project base repo and
1854use the HEAD of the tree for the build.
1855</para>
1856 </section> -->
1857
1858 <!-- <section id='example'>
1859 <title>Example</title>
1860<para>
1861This example creates a kernel layer to build the latest
1862kernel.org tree as the 'common_pc' BSP.
1863<literallayout class='monospaced'>
1864 &gt; cd &lt;path to layers&gt;
1865 &gt; mkdir wrll-linux-my_version
1866 &gt; cd wrll-linux-my_version
1867 &gt; echo "wrll-linux-2.6.27" &gt; include
1868 &gt; mkdir -p kernel-cache/ktypes/standard
1869 &gt; mkdir -p kernel-cache/bsp/common_pc
1870 &gt; echo "v2.6.29" &gt; kernel-cache/kver
1871 &gt; echo "branch common_pc-standard" &gt; kernel-cache/bsp/common_pc/common_pc.scc
1872 &gt; echo "kconf hardware common_pc.cfg" &gt;&gt; kernel-cache/bsp/common_pc/common_pc.scc
1873 &gt; echo "CONFIG_FOO=y" &gt; kernel-cache/bsp/common_pc/common_pc.cfg
1874 &gt; mkdir git
1875 &gt; cd git
1876 &gt; git clone &dash;&dash;bare git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git default_kernel
1877</literallayout>
1878</para>
1879<para>
1880Configure a build to use the new layer. This means that:
1881<literallayout class='monospaced'>
1882 &dash;&dash;enable-kernel-version=my_version
1883</literallayout>
1884</para>
1885<para>
1886Should be used to override the shipped default.
1887</para>
1888<para>
1889To build the kernel:
1890<literallayout class='monospaced'>
1891 &gt; cd build
1892 &gt; make linux_GIT_BASE=&lt;layer path&gt;/wrll-linux-my_version/git linux
1893</literallayout>
1894</para>
1895<para>
1896If this is to build without some user intervention (passing of the
1897GIT_BASE), you must do the clone into the wrll-linux-2.6.27/git directory.
1898</para>
1899<note><para>Unless you define valid "hardware.kcf" and "non-hardware.kcf" some
1900non fatal warnings will be seen. They can be fixed by populating these
1901files in the kernel-cache with valid hardware and non hardware config
1902options.
1903</para></note>
1904 </section> -->
1905 </section>
1906 </section>
1907
1908
1909
1910
1911
1912<!-- <itemizedlist>
1913 <listitem><para>Introduction to this section.</para></listitem>
1914 <listitem><para>Constructing a project-specific kernel tree.</para></listitem>
1915 <listitem><para>Building the kernel.</para></listitem>
1916 <listitem><para>Seeing what has changed.</para></listitem>
1917 <listitem><para>Seeing what has changed in a particular branch.</para></listitem>
1918 <listitem><para>Modifying the kernel.</para></listitem>
1919 <listitem><para>Saving modifications.</para></listitem>
1920 <listitem><para>Storing patches outside of the kernel source repository (bulk export).</para></listitem>
1921 <listitem><para>Working with incremental changes.</para></listitem>
1922 <listitem><para>Extracting commited changes from a working directory (exporting internally through
1923 patches.</para></listitem>
1924 <listitem><para>Pushing commited changes.</para></listitem>
1925 <listitem><para>Exporting for external (upstream) submission.</para></listitem>
1926 <listitem><para>Exporting for import into another Source Control Manager (SCM).</para></listitem>
1927 <listitem><para>Working with the Yocto Project kernel in another SCM.</para>
1928 <itemizedlist>
1929 <listitem><para>Exporting the delivered kernel to an SCM.</para></listitem>
1930 <listitem><para>Importing changed for the build.</para></listitem>
1931 </itemizedlist></listitem>
1932 <listitem><para>Migrating templates from version 2.0.</para></listitem>
1933 <listitem><para>Creating a new Board Support Package (BSP).</para>
1934 <itemizedlist>
1935 <listitem><para>Creating from scratch.</para></listitem>
1936 <listitem><para>Cloning.</para></listitem>
1937 </itemizedlist></listitem>
1938 <listitem><para>BSP bootstrapping.</para></listitem>
1939 <listitem><para>Applying patches to the kernel through a template.</para></listitem>
1940 <listitem><para>Applying patches to the kernel without using a template.</para></listitem>
1941 <listitem><para>Updating patches and configurations for a BSP.</para></listitem>
1942 <listitem><para>Using guilt to add and export patches.</para></listitem>
1943 <listitem><para>Using scc.</para></listitem>
1944 <listitem><para>Building a 'dirty' image.</para></listitem>
1945 <listitem><para>Temporarily using a different base kernel.</para></listitem>
1946 <listitem><para>Creating a custom kernel layer.</para></listitem>
1947 <listitem><para>Creating the git repository of the transition kernel.</para></listitem>
1948 </itemizedlist> -->
1949
1950
1951</chapter>
1952<!--
1953vim: expandtab tw=80 ts=4
1954-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-17 22:41:24 +0000