summaryrefslogtreecommitdiffstats
path: root/documentation/kernel-manual
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2010-11-29 07:28:51 -0800
committerSaul Wold <Saul.Wold@intel.com>2010-12-10 22:01:16 -0800
commitd2e540116f1cc0bf93b7ed6f4828c56ab909489e (patch)
tree488a9fad0a793e4f0743e2296432744c82e0dd20 /documentation/kernel-manual
parentc272830d8cc76835bbd2722848563244cc02f0f5 (diff)
downloadpoky-d2e540116f1cc0bf93b7ed6f4828c56ab909489e.tar.gz
documentation/kernel-manual/yocto-project-kernel-manual.xml: Removed this file so manual could be structured into a book with chapters.
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Diffstat (limited to 'documentation/kernel-manual')
-rw-r--r--documentation/kernel-manual/yocto-project-kernel-manual.xml2305
1 files changed, 0 insertions, 2305 deletions
diff --git a/documentation/kernel-manual/yocto-project-kernel-manual.xml b/documentation/kernel-manual/yocto-project-kernel-manual.xml
deleted file mode 100644
index c0747b2540..0000000000
--- a/documentation/kernel-manual/yocto-project-kernel-manual.xml
+++ /dev/null
@@ -1,2305 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<chapter id='kernel'>
5
6<title>Yocto Project Kernel Architecture and Use Manual</title>
7
8<section id='introduction'>
9 <title>Introduction</title>
10 <para>
11 Yocto Project presents the kernel as a fully patched, history-clean git
12 repository.
13 The git tree represents the selected features, board support,
14 and configurations extensively tested by Yocto Project.
15 The Yocto Project kernel allows the end user to leverage community
16 best practices to seamlessly manage the development, build and debug cycles.
17 </para>
18 <para>
19 This manual describes the Yocto Project kernel by providing information
20 on its history, organization, benefits, and use.
21 The manual consists of two sections:
22 <itemizedlist>
23 <listitem><para>Concepts - Describes concepts behind the kernel.
24 You will understand how the kernel is organized and why it is organized in
25 the way it is. You will understand the benefits of the kernel's organization
26 and the mechanisms used to work with the kernel and how to apply it in your
27 design process.</para></listitem>
28 <listitem><para>Using the Kernel - Describes best practices and "how-to" information
29 that lets you put the kernel to practical use. Some examples are "How to Build a
30 Project Specific Tree", "How to Examine Changes in a Branch", and "Saving Kernel
31 Modifications."</para></listitem>
32 </itemizedlist>
33 </para>
34 <para>
35 For more information on the kernel, see the following links:
36 <itemizedlist>
37 <listitem><para><ulink url='http://ldn.linuxfoundation.org/book/1-a-guide-kernel-development-process'></ulink></para></listitem>
38 <listitem><para><ulink url='http://userweb.kernel.org/~akpm/stuff/tpp.txt'></ulink></para></listitem>
39 <listitem><para><ulink url='http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=Documentation/HOWTO;hb=HEAD'></ulink></para></listitem>
40 </itemizedlist>
41 <para>
42 You can find more information on Yocto Project by visiting the website at
43 <ulink url='http://www.yoctoproject.org'></ulink>.
44 </para>
45 </para>
46</section>
47
48<section id='concepts'>
49 <title>Concepts</title>
50 <para>
51 This section provides conceptual information about the Yocto Project kernel:
52 <itemizedlist>
53 <listitem><para>Kernel Goals</para></listitem>
54 <listitem><para>Yocto Project Kernel Development and Maintenance Overview</para></listitem>
55 <listitem><para>Kernel Architecture</para></listitem>
56 <listitem><para>Kernel Tools</para></listitem>
57 </itemizedlist>
58 </para>
59 <section id='kernel-goals'>
60 <title>Kernel Goals</title>
61 <para>
62 The complexity of embedded kernel design has increased dramatically.
63 Whether it is managing multiple implementations of a particular feature or tuning and
64 optimizing board specific features, flexibility and maintainability are key concerns.
65 The Yocto Project Linux kernel is presented with the embedded
66 developer's needs in mind and has evolved to assist in these key concerns.
67 For example, prior methods such as applying hundreds of patches to an extracted
68 tarball have been replaced with proven techniques that allow easy inspection,
69 bisection and analysis of changes.
70 Application of these techniques also creates a platform for performing integration and
71 collaboration with the thousands of upstream development projects.
72 </para>
73 <para>
74 With all these considerations in mind, the Yocto Project kernel and development team
75 strives to attain these goals:
76 <itemizedlist>
77 <listitem><para>Allow the end user to leverage community best practices to seamlessly
78 manage the development, build and debug cycles.</para></listitem>
79 <listitem><para>Create a platform for performing integration and collaboration with the
80 thousands of upstream development projects that exist.</para></listitem>
81 <listitem><para>Provide mechanisms that support many different work flows, front-ends and
82 management techniques.</para></listitem>
83 <listitem><para>Deliver the most up-to-date kernel possible while still ensuring that
84 the baseline kernel is the the most stable official release.</para></listitem>
85 <listitem><para>Include major technological features as part of Yocto Project's up-rev
86 strategy.</para></listitem>
87 <listitem><para>Present a git tree, that just like the upstream kernel.org tree, has a
88 clear and continuous history.</para></listitem>
89 <listitem><para>Deliver a key set of supported kernel types, where each type is tailored
90 to a specific use case (i.g. networking, consumer, devices, and so forth).</para></listitem>
91 <listitem><para>Employ a git branching strategy that from a customer's point of view
92 results in a linear path from the baseline kernel.org, through a select group of features and
93 ends with their BSP-specific commits.</para></listitem>
94 </itemizedlist>
95 </para>
96 </section>
97
98 <section id='kernel-big-picture'>
99 <title>Yocto Project Kernel Development and Maintenance Overview</title>
100 <para>
101 Yocto Project kernel, like other kernels, is based off the Linux kernel release
102 from <ulink url='http://www.kernel.org'></ulink>.
103 At the beginning of our major development cycle, we choose our Yocto Project kernel
104 based on factors like release timing, the anticipated release timing of "final" (i.e. non "rc")
105 upstream kernel.org versions, and Yocto Project feature requirements.
106 Typically this will be a kernel that is in the
107 final stages of development by the community (i.e. still in the release
108 candidate or "rc" phase) and not yet a final release.
109 But by being in the final stages of external development, we know that the
110 kernel.org final release will clearly land within the early stages of
111 the Yocto Project development window.
112 </para>
113 <para>
114 This balance allows us to deliver the most up-to-date kernel
115 as possible, while still ensuring that we have a stable official release as
116 our baseline kernel version.
117 </para>
118 <para>
119 The following figure represents the overall place the Yocto Project kernel fills.
120 </para>
121 <para>
122 <imagedata fileref="figures/kernel-big-picture.png" width="6in" depth="6in" align="center" scale="100" />
123 </para>
124 <para>
125 In the figure the ultimate source for the Yocto Project kernel is a released kernel
126 from kernel.org.
127 In addition to a foundational kernel from kernel.org the commercially released
128 Yocto Project kernel contains a mix of important new mainline
129 developments, non-mainline developments, Board Support Package (BSP) developments,
130 and custom features.
131 These additions result in a commercially released Yocto Project kernel that caters
132 to specific embedded designer needs for targeted hardware.
133 </para>
134 <para>
135 Once a Yocto Project kernel is officially released the Yocto Project team goes into
136 their next development cycle, or "uprev" cycle.
137 It is important to note that the most sustainable and stable way
138 to include feature development upstream is through a kernel uprev process.
139 Back-porting of hundreds of individual fixes and minor features from various
140 kernel versions is not sustainable and can easily compromise quality.
141 During the uprev cycle, the Yocto Project team uses an ongoing analysis of
142 kernel development, BSP support, and release timing to select the best
143 possible kernel.org version.
144 The team continually monitors community kernel
145 development to look for significant features of interest.
146 The illustration depicts this by showing the team looking back to kernel.org for new features,
147 BSP features, and significant bug fixes.
148 The team does consider back-porting large features if they have a significant advantage.
149 User or community demand can also trigger a back-port or creation of new
150 functionality in the Yocto Project baseline kernel during the uprev cycle.
151 </para>
152 <para>
153 Generally speaking, every new kernel both adds features and introduces new bugs.
154 These consequences are the basic properties of upstream kernel development and are
155 managed by the Yocto Project team's kernel strategy.
156 It is the Yocto Project team's policy to not back-port minor features to the released kernel.
157 They only consider back-porting significant technological jumps - and, that is done
158 after a complete gap analysis.
159 The reason for this policy is that simply back-porting any small to medium sized change
160 from an evolving kernel can easily create mismatches, incompatibilities and very
161 subtle errors.
162 </para>
163 <para>
164 These policies result in both a stable and a cutting
165 edge kernel that mixes forward ports of existing features and significant and critical
166 new functionality.
167 Forward porting functionality in the Yocto Project kernel can be thought of as a
168 "micro uprev."
169 The many “micro uprevs” produce a kernel version with a mix of
170 important new mainline, non-mainline, BSP developments and feature integrations.
171 This kernel gives insight into new features and allows focused
172 amounts of testing to be done on the kernel, which prevents
173 surprises when selecting the next major uprev.
174 The quality of these cutting edge kernels is evolving and the kernels are used in very special
175 cases for BSP and feature development.
176 </para>
177 </section>
178
179 <section id='kernel-architecture'>
180 <title>Kernel Architecture</title>
181 <para>
182 This section describes the architecture of the Yocto Project kernel and provides information
183 on the mechanisms used to achieve that architecture.
184 </para>
185
186 <section id='architecture-overview'>
187 <title>Overview</title>
188 <para>
189 As mentioned earlier, a key goal of Yocto Project is to present the developer with
190 a kernel that has a clear and continuous history that is visible to the user.
191 The architecture and mechanisms used achieve that goal in a manner similar to the
192 upstream kernel.org.
193
194 </para>
195 <para>
196 You can think of the Yocto Project kernel as consisting of a baseline kernel with
197 added features logically structured on top of the baseline.
198 The features are tagged and organized by way of a branching strategy implemented by the
199 source code manager (SCM) git.
200 The result is that the user has the ability to see the added features and
201 the commits that make up those features.
202 In addition to being able to see added features, the user can also view the history of what
203 made up the baseline kernel as well.
204 </para>
205 <para>
206 The following illustration shows the conceptual Yocto Project kernel.
207 </para>
208 <para>
209 <imagedata fileref="figures/kernel-architecture-overview.png" width="6in" depth="7in" align="center" scale="100" />
210 </para>
211 <para>
212 In the illustration, the "kernel.org Branch Point" marks the specific spot (or release) from
213 which the Yocto Project kernel is created. From this point "up" in the tree features and
214 differences are organized and tagged.
215 </para>
216 <para>
217 The "Yocto Project Baseline Kernel" contains functionality that is common to every kernel
218 type and BSP that is organized further up the tree. Placing these common features in the
219 tree this way means features don't have to be duplicated along individual branches of the
220 structure.
221 </para>
222 <para>
223 From the Yocto Project Baseline Kernel branch points represent specific functionality
224 for individual BSPs as well as real-time kernels.
225 The illustration represents this through three BSP-specific branches and a real-time
226 kernel branch.
227 Each branch represents some unique functionality for the BSP or a real-time kernel.
228 </para>
229 <para>
230 The real-time kernel branch has common features for all real-time kernels and contains
231 more branches for individual BSP-specific real-time kernels.
232 The illustration shows three branches as an example.
233 Each branch points the way to specific, unique features for a respective real-time
234 kernel as they apply to a given BSP.
235 </para>
236 <para>
237 The resulting tree structure presents a clear path of markers (or branches) to the user
238 that for all practical purposes is the kernel needed for any given set of requirements.
239 </para>
240 </section>
241
242 <section id='branching-and-workflow'>
243 <title>Branching Strategy and Workflow</title>
244 <para>
245 The Yocto Project team creates kernel branches at points where functionality is
246 no longer shared and thus, needs to be isolated.
247 For example, board-specific incompatibilities would require different functionality
248 and would require a branch to separate the features.
249 Likewise, for specific kernel features the same branching strategy is used.
250 This branching strategy results in a tree that has features organized to be specific
251 for particular functionality, single kernel types, or a subset of kernel types.
252 This strategy results in not having to store the same feature twice internally in the
253 tree.
254 Rather we store the unique differences required to apply the feature onto the kernel type
255 in question.
256 </para>
257 <para>
258 BSP-specific code additions are handled in a similar manner to kernel-specific additions.
259 Some BSPs only make sense given certain kernel types.
260 So, for these types, we create branches off the end of that kernel type for all
261 of the BSPs that are supported on that kernel type.
262 From the perspective of the tools that create the BSP branch, the BSP is really no
263 different than a feature.
264 Consequently, the same branching strategy applies to BSPs as it does to features.
265 So again, rather than store the BSP twice, only the unique differences for the BSP across
266 the supported multiple kernels are uniquely stored.
267 </para>
268 <para>
269 While this strategy results in a tree with a significant number of branches, it is
270 important to realize that from the customer's point of view, there is a linear
271 path that travels from the baseline kernel.org, through a select group of features and
272 ends with their BSP-specific commits.
273 In other words, the divisions of the kernel are transparent and are not relevant
274 to the developer on a day-to-day basis.
275 From the customer's perspective, this is the "master" branch.
276 They do not need not be aware of the existence of any other branches at all.
277 Of course there is value in the existence of these branches
278 in the tree, should a person decide to explore them.
279 For example, a comparison between two BSPs at either the commit level or at the line-by-line
280 code diff level is now a trivial operation.
281 </para>
282 <para>
283 Working with the kernel as a structured tree follows recognized community best practices.
284 In particular, the kernel as shipped with the product should be
285 considered an 'upstream source' and viewed as a series of
286 historical and documented modifications (commits).
287 These modifications represent the development and stabilization done
288 by the Yocto Project kernel development team.
289 </para>
290 <para>
291 Because commits only change at significant release points in the product life cycle,
292 developers can work on a branch created
293 from the last relevant commit in the shipped Yocto Project kernel.
294 As mentioned previously, the structure is transparent to the user
295 because the kernel tree is left in this state after cloning and building the kernel.
296 </para>
297 </section>
298
299 <section id='source-code-manager-git'>
300 <title>Source Code Manager - git</title>
301 <para>
302 The Source Code Manager (SCM) is git and it is the obvious mechanism for meeting the
303 previously mentioned goals.
304 Not only is it the SCM for kernel.org but git continues to grow in popularity and
305 supports many different work flows, front-ends and management techniques.
306 </para>
307 <note><para>
308 It should be noted that you can use as much, or as little, of what git has to offer
309 as is appropriate to your project.
310 </para></note>
311 </section>
312 </section>
313
314 <section id='kernel-tools'>
315 <title>Kernel Tools</title>
316 <para>
317Since most standard workflows involve moving forward with an existing tree by
318continuing to add and alter the underlying baseline, the tools that manage
319Yocto Project's kernel construction are largely hidden from the developer to
320present a simplified view of the kernel for ease of use.
321</para>
322<para>
323The fundamental properties of the tools that manage and construct the
324kernel are:
325<itemizedlist>
326 <listitem><para>the ability to group patches into named, reusable features</para></listitem>
327 <listitem><para>to allow top down control of included features</para></listitem>
328 <listitem><para>the binding of kernel configuration to kernel patches/features</para></listitem>
329 <listitem><para>the presentation of a seamless git repository that blends Yocto Project value with the kernel.org history and development</para></listitem>
330</itemizedlist>
331</para>
332<!--<para>
333The tools that construct a kernel tree will be discussed later in this
334document. The following tools form the foundation of the Yocto Project
335kernel toolkit:
336<itemizedlist>
337 <listitem><para>git : distributed revision control system created by Linus Torvalds</para></listitem>
338 <listitem><para>guilt: quilt on top of git</para></listitem>
339 <listitem><para>*cfg : kernel configuration management and classification</para></listitem>
340 <listitem><para>kgit*: Yocto Project kernel tree creation and management tools</para></listitem>
341 <listitem><para>scc : series &amp; configuration compiler</para></listitem>
342</itemizedlist>
343</para> -->
344 </section>
345</section>
346
347
348
349
350<!-- <section id='concepts2'>
351 <title>Kernel Concepts</title>
352 <itemizedlist>
353 <listitem><para>What tools and commands are used with the kernel.</para></listitem>
354 <listitem><para>Source Control Manager (SCM).</para></listitem>
355 <listitem><para>What are some workflows that you can apply using the kernel.</para></listitem>
356 </itemizedlist>
357</section> -->
358
359<section id='actions'>
360 <title>How to Get Things Accomplished with the Kernel</title>
361 <para>
362 This section describes how to accomplish tasks involving the kernel's tree structure.
363 The information covers the following:
364 <itemizedlist>
365 <listitem><para>Tree construction</para></listitem>
366 <listitem><para>Build strategies</para></listitem>
367<!-- <listitem><para>Series &amp; Configuration Compiler</para></listitem>
368 <listitem><para>kgit</para></listitem> -->
369 <listitem><para>Workflow examples</para></listitem>
370 <listitem><para>Source Code Manager (SCM)</para></listitem>
371<!-- <listitem><para>Board Support Package (BSP) template migration</para></listitem> -->
372 <listitem><para>BSP creation</para></listitem>
373 <listitem><para>Patching</para></listitem>
374 <listitem><para>Updating BSP patches and configuration</para></listitem>
375<!-- <listitem><para>guilt</para></listitem>
376 <listitem><para>scc file example</para></listitem> -->
377 <listitem><para>"dirty" string</para></listitem>
378<!-- <listitem><para>Transition kernel layer</para></listitem> -->
379 </itemizedlist>
380 </para>
381
382 <section id='tree-construction'>
383 <title>Tree Construction</title>
384 <para>
385The Yocto Project kernel repository, as shipped with the product, is created by
386compiling and executing the set of feature descriptions for every BSP/feature
387in the product. Those feature descriptions list all necessary patches,
388configuration, branching, tagging and feature divisions found in the kernel.
389</para>
390<para>
391The files used to describe all the valid features and BSPs in the Yocto Project
392kernel can be found in any clone of the kernel git tree. The directory
393wrs/cfg/kernel-cache/ is a snapshot of all the kernel configuration and
394feature descriptions (.scc) that were used to build the kernel repository.
395It should however be noted, that browsing the snapshot of feature
396descriptions and patches is not an effective way to determine what is in a
397particular kernel branch. Using git directly to get insight into the changes
398in a branch is more efficient and a more flexible way to inspect changes to
399the kernel. Examples of using git to inspect kernel commits are in the
400following sections.
401</para>
402<para>
403As a reminder, it is envisioned that a ground up reconstruction of the
404complete kernel tree is an action only taken by Yocto Project team during an
405active development cycle. When an end user creates a project, it takes
406advantage of this complete tree in order to efficiently place a git tree
407within their project.
408</para>
409<para>
410The general flow of the project specific kernel tree construction is as follows:
411<orderedlist>
412 <listitem><para>a top level kernel feature is passed to the kernel build subsystem,
413 normally this is a BSP for a particular kernel type.</para></listitem>
414
415 <listitem><para>the file that describes the top level feature is located by searching
416 system directories:</para>
417
418 <itemizedlist>
419 <listitem><para>the kernel-cache under linux/wrs/cfg/kernel-cache</para></listitem>
420<!-- <listitem><para>kernel-*-cache directories in layers</para></listitem> -->
421 <listitem><para>recipe SRC_URIs</para></listitem>
422<!-- <listitem><para>configured and default templates</para></listitem> -->
423 </itemizedlist>
424
425 <para>In a typical build a feature description of the format:
426 &lt;bsp name&gt;-&lt;kernel type&gt;.scc is the target of the search.
427 </para></listitem>
428
429 <listitem><para>once located, the feature description is compiled into a simple script
430 of actions, or an existing equivalent script which was part of the
431 shipped kernel is located.</para></listitem>
432
433 <listitem><para>extra features are appended to the top level feature description. Extra
434 features can come from the KERNEL_FEATURES variable in recipes.</para></listitem>
435
436 <listitem><para>each extra feature is located, compiled and appended to the script from
437 step #3</para></listitem>
438
439 <listitem><para>the script is executed, and a meta-series is produced. The meta-series
440 is a description of all the branches, tags, patches and configuration that
441 need to be applied to the base git repository to completely create the
442 "bsp_name-kernel_type".</para></listitem>
443
444 <listitem><para>the base repository is cloned, and the actions
445 listed in the meta-series are applied to the tree.</para></listitem>
446
447 <listitem><para>the git repository is left with the desired branch checked out and any
448 required branching, patching and tagging has been performed.</para></listitem>
449</orderedlist>
450</para>
451
452<para>
453The tree is now ready for configuration and compilation. Those two topics will
454be covered below.
455</para>
456
457<note><para>The end user generated meta-series adds to the kernel as shipped with
458 the Yocto Project release. Any add-ons and configuration data are applied
459 to the end of an existing branch. The full repository generation that
460 is found in the linux-2.6-windriver.git is the combination of all
461 supported boards and configurations.
462</para></note>
463
464<para>
465This technique is flexible and allows the seamless blending of an immutable
466history with additional deployment specific patches. Any additions to the
467kernel become an integrated part of the branches.
468</para>
469
470<!-- <note><para>It is key that feature descriptions indicate if any branches are
471 required, since the build system cannot automatically decide where a
472 BSP should branch or if that branch point needs a name with
473 significance. There is a single restriction enforced by the compilation
474 phase:
475 </para>
476 <para>A BSP must create a branch of the format &lt;bsp name&gt;-&lt;kernel type&gt;.</para>
477
478 <para>This means that all merged/support BSPs must indicate where to start
479 its branch from, with the right name, in its .scc files. The scc
480 section describes the available branching commands in more detail.
481 </para>
482</note> -->
483
484<para>
485A summary of end user tree construction activities follow:
486<itemizedlist>
487 <listitem><para>compile and link a full top-down kernel description from feature descriptions</para></listitem>
488 <listitem><para>execute the complete description to generate a meta-series</para></listitem>
489 <listitem><para>interpret the meta-series to create a customized git repository for the
490 board</para></listitem>
491 <listitem><para>migrate configuration fragments and configure the kernel</para></listitem>
492 <listitem><para>checkout the BSP branch and build</para></listitem>
493</itemizedlist>
494</para>
495 </section>
496
497 <section id='build-strategy'>
498 <title>Build Strategy</title>
499<para>
500There are some prerequisites that must be met before starting the compilation
501phase of the kernel build system:
502</para>
503<itemizedlist>
504 <listitem><para>There must be a kernel git repository indicated in the SRC_URI.</para></listitem>
505 <listitem><para>There must be a branch &lt;bsp name&gt;-&lt;kernel type&gt;.</para></listitem>
506</itemizedlist>
507
508<para>
509These are typically met by running tree construction/patching phase of the
510build system, but can be achieved by other means. Examples of alternate work
511flows such as bootstrapping a BSP are provided below.
512</para>
513<para>
514Before building a kernel it is configured by processing all of the
515configuration "fragments" specified by the scc feature descriptions. As the
516features are compiled, associated kernel configuration fragments are noted
517and recorded in the meta-series in their compilation order. The
518fragments are migrated, pre-processed and passed to the Linux Kernel
519Configuration subsystem (lkc) as raw input in the form of a .config file.
520The lkc uses its own internal dependency constraints to do the final
521processing of that information and generates the final .config that will
522be used during compilation.
523</para>
524<para>
525Kernel compilation is started, using the board's architecture and other
526relevant values from the board template, and a kernel image is produced.
527</para>
528<para>
529The other thing that you will first see once you configure a kernel is that
530it will generate a build tree that is separate from your git source tree.
531This build dir will be called "linux-&lt;BSPname&gt;-&lt;kerntype&gt;-build" where
532kerntype is one of standard kernel types. This functionality is done by making
533use of the existing support that is within the kernel.org tree by default.
534</para>
535<para>
536What this means, is that all the generated files (that includes the final
537".config" itself, all ".o" and ".a" etc) are now in this directory. Since
538the git source tree can contain any number of BSPs, all on their own branch,
539you now can easily switch between builds of BSPs as well, since each one also
540has their own separate build directory.
541</para>
542 </section>
543
544<!-- <section id='scc'>
545 <title>Series &amp; Configuration Compiler (SCC)</title>
546<para>
547In early versions of the product, kernel patches were simply listed in a flat
548file called "patches.list", and then quilt was added as a tool to help
549traverse this list, which in quilt terms was called a "series" file.
550</para>
551<para>
552Before the 2.0 release, it was already apparent that a static series file was
553too inflexible, and that the series file had to become more dynamic and rely
554on certain state (like kernel type) in order to determine whether a patch was
555to be used or not. The 2.0 release already made use of some stateful
556construction of series files, but since the delivery mechanism was unchanged
557(tar + patches + series files), most people were not aware of anything really
558different. The 3.0 release continues with this stateful construction of
559series files, but since the delivery mechanism is changed (git + branches) it
560now is more apparent to people.
561</para>
562<para>
563As was previously mentioned, scc is a "series and configuration
564compiler". Its role is to combine feature descriptions into a format that can
565be used to generate a meta-series. A meta series contains all the required
566information to construct a complete set of branches that are required to
567build a desired board and feature set. The meta series is interpreted by the
568kgit tools to create a git repository that could be built.
569</para>
570<para>
571To illustrate how scc works, a feature description must first be understood.
572A feature description is simply a small bash shell script that is executed by
573scc in a controlled environment. Each feature description describes a set of
574operations that add patches, modify existing patches or configure the
575kernel. It is key that feature descriptions can include other features, and
576hence allow the division of patches and configuration into named, reusable
577containers.
578</para>
579<para>
580Each feature description can use any of the following valid scc commands:
581<itemizedlist>
582 <listitem><para>shell constructs: bash conditionals and other utilities can be used in a feature
583 description. During compilation, the working directory is the feature
584 description itself, so any command that is "raw shell" and not from the
585 list of supported commands, can not directly modify a git repository.</para></listitem>
586
587 <listitem><para>patch &lt;relative path&gt;/&lt;patch name&gt;: outputs a patch to be included in a feature's patch set. Only the name of
588 the patch is supplied, the path is calculated from the currently set
589 patch directory, which is normally the feature directory itself.</para></listitem>
590
591 <listitem><para>patch_trigger &gt;condition&lt; &gt;action&lt; &lt;tgt&gt;: indicate that a trigger should be set to perform an action on a
592 patch.</para>
593
594<para>The conditions can be:
595
596 <itemizedlist>
597 <listitem><para>arch:&lt;comma separated arch list or "all"&gt;</para></listitem>
598 <listitem><para>plat:&lt;comma separated platform list or "all"&gt;</para></listitem>
599 </itemizedlist></para>
600<para>The action can be:
601 <itemizedlist>
602 <listitem><para>exclude: This is used in exceptional situations where a patch
603 cannot be applied for certain reasons (arch or platform).
604 When the trigger is satisfied the patch will be removed from
605 the patch list.</para></listitem>
606 <listitem><para>include: This is used to include a patch only for a specific trigger.
607 Like exclude, this should only be used when necessary.
608 It takes 1 argument, the patch to include.</para></listitem>
609 </itemizedlist></para></listitem>
610
611 <listitem><para>include &lt;feature name&gt; [after &lt;feature&gt;]: includes a feature for processing. The feature is "expanded" at the
612 position of the include directive. This means that any patches,
613 configuration or sub-includes of the feature will appear in the final
614 series before the commands that follow the include.</para>
615 <para>
616 include searches the include directories for a matching feature name,
617 include directories are passed to scc by the caller using -I &lt;path&gt; and
618 is transparent to the feature script. This means that &lt;feature name&gt; must
619 be relative to one of the search paths. For example, if
620 /opt/kernel-cache/feat/sched.scc is to be included and scc is invoked
621 with -I /opt/kernel-cache, then a feature would issue "include
622 feat/sched.scc" to include the feature.
623</para>
624<para>
625 The optional "after" directive allows a feature to modify the existing
626 order of includes and insert a feature after the named feature is
627 processed. Note: the "include foo after bar" must be issued before "bar"
628 is processed, so is normally only used by a new top level feature to
629 modify the order of features in something it is including.</para></listitem>
630
631 <listitem><para>exclude &lt;feature name&gt;: Indicates that a particular feature should *not* be included even if an
632 'include' directive is found. The exclude must be issued before the
633 include is processed, so is normally only used by a new top level feature
634 to modify the order of features in something it is including.</para></listitem>
635
636 <listitem><para>git &lt;command&gt;: Issues any git command during tree construction. Note: this command is
637 not validated/sanitized so care must be taken to not damage the
638 tree. This can be used to script branching, tagging, pulls or other git
639 operations.</para></listitem>
640
641 <listitem><para>dir &lt;directory&gt;: changes the working directory for "patch" directives. This can be used to
642 shorten a long sequence of patches by not requiring a common relative
643 directory to be issued each time.</para></listitem>
644
645 <listitem><para>kconf &lt;type&gt; &lt;fragment name&gt;: associates a kernel config frag with the feature.
646 &lt;type&gt; can be
647 "hardware" or "non-hardware" and is used by the kernel configuration
648 subsystem to audit configuration. &lt;fragment name&gt; is the name of a file
649 in the current feature directory that contains a series of kernel
650 configuration options. There is no restriction on the chosen fragment
651 name, although a suffix of ".cfg" is recommended. Multiple fragment
652 specifications are supported.</para></listitem>
653
654 <listitem><para>branch &lt;branch name&gt;: creates a branch in the tree. All subsequent patch commands will be
655 applied to the new branch and changes isolated from the rest of the
656 repository.</para></listitem>
657
658 <listitem><para>scc_leaf &lt;base feature&gt; &lt;branch name&gt;: Performs a combination feature include and branch. This is mainly a
659 convenience directive, but has significance to some build system bindings
660 as a sentinel to indicate that this intends to create a branch that is
661 valid for kernel compilation.</para></listitem>
662
663 <listitem><para>tag &lt;tag name&gt;: Tags the tree. The tag will be applied in processing order, so will
664 be after already applied patches and precede patches yet to be applied.</para></listitem>
665
666 <listitem><para>define &lt;var&gt; &lt;value&gt;: Creates a variable with a particular value that can be used in subsequent
667 feature descriptions.</para></listitem>
668</itemizedlist>
669
670</para>
671 </section> -->
672
673<!-- <section id='kgit-tools'>
674 <title>kgit Tools</title>
675<para>
676The kgit tools are responsible for constructing and maintaining the Wind
677River kernel repository. These activities include importing, exporting, and
678applying patches as well as sanity checking and branch management. From the
679developers perspective, the kgit tools are hidden and rarely require
680interactive use. But one tool in particular that warrants further description
681is "kgit-meta".
682</para>
683<para>
684kgit-meta is the actual application of feature description(s) to a kernel repo.
685In other words, it is responsible for interpreting the meta series generated
686from a scc compiled script. As a result, kgit-meta is coupled to the set of
687commands permitted in a .scc feature description (listed in the scc section).
688kgit-meta understands both the meta series format and how to use git and
689guilt to modify a base git repository. It processes a meta-series line by
690line, branching, tagging, patching and tracking changes that are made to the
691base git repository.
692</para>
693<para>
694Once kgit-meta has processed a meta-series, it leaves the repository with the
695last branch checked out, and creates the necessary guilt infrastructure to
696inspect the tree, or add to it via using guilt. As was previously mentioned,
697guilt is not required, but is provided as a convenience. Other utilities such
698as quilt, stgit, git or others can also be used to manipulate the git
699repository.
700</para>
701 </section> -->
702
703 <section id='workflow-examples'>
704 <title>Workflow Examples</title>
705
706 <para>
707As previously noted, the Yocto Project kernel has built in git/guilt
708integration, but these utilities are not the only way to work with the kernel
709repository. Yocto Project has not made changes to git, or other tools that
710invalidate alternate workflows. Additionally, the way the kernel repository
711is constructed uses only core git functionality allowing any number of tools
712or front ends to use the resulting tree.</para>
713<para>
714This section contains several workflow examples.
715</para>
716
717 <section id='change-inspection-kernel-changes-commits'>
718 <title>Change Inspection: Kernel Changes/Commits</title>
719<para>
720A common question when working with a BSP/kernel is: "What changes have been applied to this tree?"
721</para>
722<para>
723In some projects, where a collection of directories that
724contained patches to the kernel, those patches could be inspected, grep'd or
725otherwise used to get a general feeling for changes. This sort of patch
726inspection is not an efficient way to determine what has been done to the
727kernel, since there are many optional patches that are selected based on the
728kernel type and feature description, not to mention patches that are actually
729in directories that are not being searched.
730</para>
731<para>
732A more effective way to determine what has changed in the kernel is to use
733git and inspect / search the kernel tree. This is a full view of not only the
734source code modifications, but the reasoning behind the changes.
735</para>
736 <section id='what-changed-in-a-bsp'>
737 <title>What Changed in a BSP?</title>
738<para>
739These examples could continue for some time, since the Yocto Project git
740repository doesn't break existing git functionality and there are nearly
741endless permutations of those commands. Also note that unless a commit range
742is given (&lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;), kernel.org history is blended
743with Yocto Project changes
744</para>
745<literallayout class='monospaced'>
746 # full description of the changes
747 &gt; git whatchanged &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
748 &gt; eg: git whatchanged standard..common_pc-standard
749
750 # summary of the changes
751 &gt; git log &dash;&dash;pretty=oneline &dash;&dash;abbrev-commit &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
752
753 # source code changes (one combined diff)
754 &gt; git diff &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
755 &gt; git show &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
756
757 # dump individual patches per commit
758 &gt; git format-patch -o &lt;dir&gt; &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
759
760 # determine the change history of a particular file
761 &gt; git whatchanged &lt;path to file&gt;
762
763 # determine the commits which touch each line in a file
764 &gt; git blame &lt;path to file&gt;
765</literallayout>
766 </section>
767
768 <section id='show-a-particular-feature-or-branch-change'>
769 <title>Show a Particular Feature or Branch Change</title>
770<para>
771Significant features or branches are tagged in the Yocto Project tree to divide
772changes. Remember to first determine (or add) the tag of interest. Note:
773there will be many tags, since each BSP branch is tagged, kernel.org tags and
774feature tags are all present.
775</para>
776<literallayout class='monospaced'>
777 # show the changes tagged by a feature
778 &gt; git show &lt;tag&gt;
779 &gt; eg: git show yaffs2
780
781 # determine which branches contain a feature
782 &gt; git branch &dash;&dash;contains &lt;tag&gt;
783
784 # show the changes in a kernel type
785 &gt; git whatchanged wrs_base..&lt;kernel type&gt;
786 &gt; eg: git whatchanged wrs_base..standard
787</literallayout>
788<para>
789Many other comparisons can be done to isolate BSP changes, such as comparing
790to kernel.org tags (v2.6.27.18, etc), per subsystem comparisons (git
791whatchanged mm) or many other types of checks.
792</para>
793 </section>
794 </section>
795
796 <section id='development-saving-kernel-modifications'>
797 <title>Development: Saving Kernel Modifications</title>
798<para>
799Another common operation is to build a Yocto Project supplied BSP, make some
800changes, rebuild and test. Those local changes often need to be exported,
801shared or otherwise maintained.
802</para>
803<para>
804Since the Yocto Project kernel source tree is backed by git, this activity is
805greatly simplified and is much easier than in previous releases. git tracks
806file modifications, additions and deletions, which allows the developer to
807modify the code and later realize that the changes should be saved, and
808easily determine what was changed. It also provides many tools to commit,
809undo and export those modifications.
810</para>
811<para>
812There are many ways to perform this action, and the technique employed
813depends on the destination for the patches, which could be any of:
814<itemizedlist>
815 <listitem><para>bulk storage</para></listitem>
816 <listitem><para>internal sharing either through patches or using git</para></listitem>
817 <listitem><para>external submission</para></listitem>
818 <listitem><para>export for integration into another SCM</para></listitem>
819</itemizedlist>
820</para>
821<para>
822The destination of the patches also incluences the method of gathering them
823due to issues such as:
824<itemizedlist>
825 <listitem><para>bisectability</para></listitem>
826 <listitem><para>commit headers</para></listitem>
827 <listitem><para>division of subsystems for separate submission / review</para></listitem>
828</itemizedlist>
829</para>
830
831 <section id='bulk-export'>
832 <title>Bulk Export</title>
833<para>
834If patches are simply being stored outside of the kernel source repository,
835either permanently or temporarily, then there are several methods that can be
836used.
837</para>
838<para>
839Note the "bulk" in this discussion, these techniques are not appropriate for
840full integration of upstream submission, since they do not properly divide
841changes or provide an avenue for per-change commit messages. This example
842assumes that changes have not been committed incrementally during development
843and simply must be gathered and exported.
844<literallayout class='monospaced'>
845 # bulk export of ALL modifications without separation or division
846 # of the changes
847
848 &gt; git add .
849 &gt; git commit -s -a -m &gt;commit message&lt;
850 or
851 &gt; git commit -s -a # and interact with $EDITOR
852</literallayout>
853</para>
854<para>
855These operations have captured all the local changes in the project source
856tree in a single git commit, and that commit is also stored in the project's
857source tree.
858</para>
859<para>
860Once exported, those changes can then be restored manually, via a template or
861through integration with the default_kernel. Those topics are covered in
862future sections.
863</para>
864 </section>
865
866 <section id='incremental-planned-sharing'>
867 <title>Incremental/Planned Sharing</title>
868<para>
869Note: unlike the previous "bulk" section, the following examples assume that
870changes have been incrementally committed to the tree during development and
871now are being exported.
872</para>
873<para>
874During development the following commands will be of interest, but for full
875git documentation refer to the git man pages or an online resource such as
876http://github.com
877<literallayout class='monospaced'>
878 # edit a file
879 &gt; vi &gt;path&lt;/file
880 # stage the change
881 &gt; git add &gt;path&lt;/file
882 # commit the change
883 &gt; git commit -s
884 # remove a file
885 &gt; git rm &gt;path&lt;/file
886 # commit the change
887 &gt; git commit -s
888
889 ... etc.
890</literallayout>
891</para>
892<para>
893Distributed development with git is possible by having a universally agreed
894upon unique commit identifier (set by the creator of the commit) mapping to a
895specific changeset with a specific parent. This ID is created for you when
896you create a commit, and will be re-created when you amend/alter or re-apply
897a commit. As an individual in isolation, this is of no interest, but if you
898intend to share your tree with normal git push/pull operations for
899distributed development, you should consider the ramifications of changing a
900commit that you've already shared with others.
901</para>
902<para>
903Assuming that the changes have *not* been pushed upstream, or pulled into
904another repository, both the commit content and commit messages associated
905with development can be update via:
906<literallayout class='monospaced'>
907 &gt; git add &gt;path&lt;/file
908 &gt; git commit &dash;&dash;amend
909 &gt; git rebase or git rebase -i
910</literallayout>
911</para>
912<para>
913Again, assuming that the changes have *not* been pushed upstream, and that
914there are no pending works in progress (use "git status" to check) then
915commits can be reverted (undone) via:
916<literallayout class='monospaced'>
917 # remove the commit, update working tree and remove all
918 # traces of the change
919 &gt; git reset &dash;&dash;hard HEAD^
920 # remove the commit, but leave the files changed and staged for re-commit
921 &gt; git reset &dash;&dash;soft HEAD^
922 # remove the commit, leave file change, but not staged for commit
923 &gt; git reset &dash;&dash;mixed HEAD^
924</literallayout>
925</para>
926<para>
927Branches can be created, changes cherry-picked or any number of git
928operations performed until the commits are in good order for pushing upstream
929or pull requests. After a push or pull, commits are normally considered
930'permanent' and should not be modified, only incrementally changed in new
931commits. This is standard "git" workflow and Yocto Project recommends the
932kernel.org best practices.
933</para>
934<note><para>It is recommend to tag or branch before adding changes to a Yocto Project
935 BSP (or creating a new one), since the branch or tag provides a
936 reference point to facilitate locating and exporting local changes.
937</para></note>
938
939 <section id='export-internally-via-patches'>
940 <title>Export Internally Via Patches</title>
941<para>
942Committed changes can be extracted from a working directory by exporting them
943as patches. Those patches can be used for upstream submission, placed in a
944Yocto Project template for automatic kernel patching or many other common uses.
945
946<literallayout class='monospaced'>
947 # &gt;first commit&gt; can be a tag if one was created before development
948 # began. It can also be the parent branch if a branch was created
949 # before development began.
950
951 &gt; git format-patch -o &lt;dir&gt; &lt;first commit&gt;..&lt;last commit&gt;
952</literallayout>
953</para>
954
955<para>
956 In other words:
957<literallayout class='monospaced'>
958 # identify commits of interest.
959
960 # if the tree was tagged before development
961 &gt; git format-patch -o &lt;save dir&gt; &lt;tag&gt;
962
963 # if no tags are available
964 &gt; git format-patch -o &lt;save dir&gt; HEAD^ # last commit
965 &gt; git format-patch -o &lt;save dir&gt; HEAD^^ # last 2 commits
966 &gt; git whatchanged # identify last commit
967 &gt; git format-patch -o &lt;save dir&gt; &lt;commit id&gt;
968 &gt; git format-patch -o &lt;save dir&gt; &lt;rev-list&gt;
969</literallayout>
970</para>
971
972<para>
973The result is a directory with sequentially numbered patches, that when
974applied to a repository using "git am", will reproduce the original commit
975and all related information (author, date, commit log, etc) will be
976preserved. Note that new commit IDs will be generated upon reapplication,
977reflecting that the commit is now applied to an underlying commit with a
978different ID.
979</para>
980<!--<para>
981See the "template patching" example for how to use the patches to
982automatically apply to a new kernel build.
983</para> -->
984 </section>
985
986 <section id='export-internally-via-git'>
987 <title>Export Internally Via git</title>
988<para>
989Committed changes can also be exported from a working directory by pushing
990(or by making a pull request) the changes into a master repository. Those
991same change can then be pulled into a new kernel build at a later time using this command form:
992<literallayout class='monospaced'>
993 git push ssh://&lt;master server&gt;/&lt;path to repo&gt; &lt;local branch&gt;:&lt;remote branch&gt;
994</literallayout>
995For example:
996<literallayout class='monospaced'>
997 &gt; push ssh://git.mycompany.com/pub/git/kernel-2.6.27 common_pc-standard:common_pc-standard
998</literallayout>
999A pull request entails using "git request-pull" to compose an email to the
1000maintainer requesting that a branch be pulled into the master repository, see
1001http://github.com/guides/pull-requests for an example.
1002</para>
1003<para>
1004Other commands such as 'git stash' or branching can also be used to save
1005changes, but are not covered in this document.
1006</para>
1007<para>
1008See the section "importing from another SCM" for how a git push to the
1009default_kernel, can be used to automatically update the builds of all users
1010of a central git repository.
1011</para>
1012 </section>
1013 </section>
1014
1015 <section id='export-for-external-upstream-submission'>
1016 <title>Export for External (Upstream) Submission</title>
1017<para>
1018If patches are to be sent for external submission, they can be done via a
1019pull request if the patch series is large or the maintainer prefers to pull
1020changes. But commonly, patches are sent as email series for easy review and
1021integration.
1022</para>
1023<note><para>
1024Before sending patches for review ensure that you understand the
1025standard of the community in question and follow their best practices. For
1026example, kernel patches should follow standards such as:
1027<itemizedlist>
1028 <listitem><para><ulink url='http://userweb.kernel.org/~akpm/stuff/tpp.txt'></ulink></para></listitem>
1029 <listitem><para><ulink url='http://linux.yyz.us/patch-format.html'></ulink></para></listitem>
1030 <listitem><para>Documentation/SubmittingPatches (in any linux kernel source tree)</para></listitem>
1031</itemizedlist>
1032</para></note>
1033<para>
1034The messages used to commit changes are a large part of these standards, so
1035ensure that the headers for each commit have the required information. If the
1036initial commits were not properly documented or don't meet those standards
1037rebasing via git rebase -i offer an opportunity to manipulate the commits and
1038get them into the required format. Other techniques such as branching and
1039cherry picking commits are also viable options.
1040</para>
1041<para>
1042Once complete, patches are sent via email to the maintainer(s) or lists that
1043review and integrate changes. "git send-email" is commonly used to ensure
1044that patches are properly formatted for easy application and avoid mailer
1045induced patch damage.
1046</para>
1047<para>
1048An example of dumping patches for external submission follows:
1049<literallayout class='monospaced'>
1050 # dump the last 4 commits
1051 &gt; git format-patch &dash;&dash;thread -n -o ~/rr/ HEAD^^^^
1052 &gt; git send-email &dash;&dash;compose &dash;&dash;subject '[RFC 0/N] &lt;patch series summary&gt;' \
1053 &dash;&dash;to foo@yoctoproject.org &dash;&dash;to bar@yoctoproject.org \
1054 &dash;&dash;cc list@yoctoproject.org ~/rr
1055 # the editor is invoked for the 0/N patch, and when complete the entire
1056 # series is sent via email for review
1057</literallayout>
1058</para>
1059 </section>
1060
1061 <section id='export-for-import-into-other-scm'>
1062 <title>Export for Import into Other SCM</title>
1063<para>
1064Using any one of the previously discussed techniques, commits can be exported
1065as patches for import into another SCM. Note however, that if those patches
1066are manually applied to a secondary tree and then that secondary tree is
1067checked into the SCM, then it often results in lost information (like commit
1068logs) and so it is not recommended.
1069</para>
1070<para>
1071Many SCMs can directly import git commits, or can translate git patches to
1072not lose information. Those facilities are SCM dependent and should be used
1073whenever possible.
1074</para>
1075 </section>
1076 </section>
1077
1078 <section id='scm-working-with-the-yocto-project-kernel-in-another-scm'>
1079 <title>SCM: Working with the Yocto Project Kernel in Another SCM</title>
1080<para>
1081This is not the same as the exporting of patches to another SCM, but instead
1082is concerned with kernel development that is done completely in another
1083environment, but built with the Yocto Project build system. In this scenario two
1084things must happen:
1085<itemizedlist>
1086 <listitem><para>The delivered Yocto Project kernel must be exported into the second
1087 SCM.</para></listitem>
1088 <listitem><para>Development must be exported from that secondary SCM into a
1089 format that can be used by the Yocto Project build system.</para></listitem>
1090</itemizedlist>
1091</para>
1092 <section id='exporting-delivered-kernel-to-scm'>
1093 <title>Exporting Delivered Kernel to SCM</title>
1094<para>
1095Depending on the SCM it may be possible to export the entire Yocto Project
1096kernel git repository, branches and all, into a new environment. This is the
1097preferred method, since it has the most flexibility and potential to maintain
1098the meta data associated with each commit.
1099</para>
1100<para>
1101When a direct import mechanism is not available, it is still possible to
1102export a branch (or series of branches) and check them into a new
1103repository.
1104</para>
1105<para>
1106The following commands illustrate some of the steps that could be used to
1107import the common_pc-standard kernel into a secondary SCM
1108<literallayout class='monospaced'>
1109 &gt; git checkout common_pc-standard
1110 &gt; cd .. ; echo linux/.git &gt; .cvsignore
1111 &gt; cvs import -m "initial import" linux MY_COMPANY start
1112</literallayout>
1113The CVS repo could now be relocated and used in a centralized manner.
1114</para>
1115<para>
1116The following commands illustrate how two BSPs could be condensed and merged
1117into a second SCM:
1118<literallayout class='monospaced'>
1119 &gt; git checkout common_pc-standard
1120 &gt; git merge cav_ebt5800-standard
1121 # resolve any conflicts and commit them
1122 &gt; cd .. ; echo linux/.git &gt; .cvsignore
1123 &gt; cvs import -m "initial import" linux MY_COMPANY start
1124</literallayout>
1125</para>
1126 </section>
1127
1128 <section id='importing-changes-for-build'>
1129 <title>Importing Changes for Build</title>
1130<para>
1131Once development has reached a suitable point in the second development
1132environment, changes can either be exported as patches or imported into git
1133directly (if a conversion/import mechanism is available for the SCM).
1134</para>
1135<para>
1136If changes are exported as patches, they can be placed in a recipe and
1137automatically applied to the kernel during patching.
1138</para>
1139<!--<para>
1140If changes are imported directly into git, they must be propagated to the
1141wrll-linux-2.6.27/git/default_kernel bare clone of each individual build
1142to be present when the kernel is checked out.
1143</para>
1144<para>
1145The following example illustrates one variant of this workflow:
1146<literallayout class='monospaced'>
1147 # on master git repository
1148 &gt; cd linux-2.6.27
1149 &gt; git tag -d common_pc-standard-mark
1150 &gt; git pull ssh://&lt;foo&gt;@&lt;bar&gt;/pub/git/kernel-2.6.27 common_pc-standard:common_pc-standard
1151 &gt; git tag common_pc-standard-mark
1152
1153 # on each build machine (or NFS share, etc)
1154 &gt; cd wrll-linux-2.6.27/git/default_kernel
1155 &gt; git fetch ssh://&lt;foo&gt;@&lt;master server&gt;/pub/git/kernel-2.6.27
1156
1157 # in the build, perform a from-scratch build of Linux and the new changes
1158 # will be checked out and built.
1159 &gt; make linux
1160</literallayout>
1161</para> -->
1162 </section>
1163 </section>
1164
1165<!-- <section id='bsp-template-migration-from-2'>
1166 <title>BSP: Template Migration from 2.0</title>
1167<para>
1168The move to a git-backed kernel build system in 3.0 introduced a small new
1169requirement for any BSP that is not integrated into the GA release of the
1170product: branching information.
1171</para>
1172<para>
1173As was previously mentioned in the background sections, branching information
1174is always required, since the kernel build system cannot make intelligent
1175branching decisions and must rely on the developer. This branching
1176information is provided via a .scc file.
1177</para>
1178<para>
1179A BSP template in 2.0 contained build system information (config.sh, etc) and
1180kernel patching information in the 'linux' subdirectory. The same holds true
1181in 3.0, with only minor changes in the kernel patching directory.
1182The ".smudge" files are now ".scc" files and now contain a full description
1183 of the kernel branching, patching and configuration for the BSP. Where in
1184 2.0, they only contained kernel patching information.
1185</para>
1186<para>
1187The following illustrates the migration of a simple 2.0 BSP template to the
1188new 3.0 kernel build system.
1189</para>
1190<note><para>
1191Note: all operations are from the root of a customer layer.
1192</para></note>
1193<literallayout class='monospaced'>
1194 templates/
1195 `&dash;&dash; board
1196 `&dash;&dash; my_board
1197 |&dash;&dash; config.sh
1198 |&dash;&dash; include
1199 `&dash;&dash; linux
1200 `&dash;&dash; 2.6.x
1201 |&dash;&dash; knl-base.cfg
1202 |&dash;&dash; bsp.patch
1203 `&dash;&dash; my_bsp.smudge
1204
1205 &gt; mv templates/board/my_board/linux/2.6.x/* templates/board/my_board/linux
1206 &gt; rm -rf templates/board/my_board/linux/2.6.x/
1207 &gt; mv templates/board/my_board/linux/my_bsp.smudge \
1208 templates/board/my_board/linux/my_bsp-standard.scc
1209 &gt; echo "kconf hardware knl-base.cfg" &gt;&gt; \
1210 templates/board/my_board/linux/my_bsp-standard.scc
1211 &gt; vi templates/board/my_board/linux/my_bsp-standard.scc
1212 # add the following at the top of the file
1213 scc_leaf ktypes/standard my_bsp-standard
1214
1215 templates/
1216 `&dash;&dash; board
1217 `&dash;&dash; my_board
1218 |&dash;&dash; config.sh
1219 |&dash;&dash; include
1220 `&dash;&dash; linux
1221 |&dash;&dash; knl-base.cfg
1222 |&dash;&dash; bsp.patch
1223 `&dash;&dash; my_bsp-standard.scc
1224</literallayout>
1225<para>
1226That's it. Configure and build.
1227</para>
1228<note><para>There is a naming convention for the .scc file, which allows the build
1229 system to locate suitable feature descriptions for a board:
1230</para></note>
1231<literallayout class='monospaced'>
1232 &lt;bsp name&gt;-&lt;kernel type&gt;.scc
1233</literallayout>
1234<para>
1235 if this naming convention isn't followed your feature description will
1236 not be located and a build error thrown.
1237</para>
1238 </section> -->
1239
1240
1241
1242 <section id='bsp-creating'>
1243 <title>BSP: Creating</title>
1244 <para>
1245 This section provides an example for creating a BSP based on an existing, and hopefully,
1246 similar one.
1247 Follow these steps and keep in mind your particular situation and differences:
1248 <orderedlist>
1249 <listitem><para>Get a machine configuration file that matches your machine.</para>
1250 <para>You can start with something in <filename>meta/conf/machine</filename>.
1251 Or, <filename>meta-emenlow/conf/machine</filename> has an example in its own layer.</para>
1252 <para>The most up-to-date machines that are probably most similar to yours and that you might want
1253 to look at are <filename>meta/conf/machine/atom-pc.conf</filename> and
1254 <filename>meta-emenlow/conf/machine/emenlow.conf</filename>.
1255 Both of these were either just added or upgraded to use the Yocto Project kernel
1256 at <ulink url='http://git.pokylinux.org/cgit/cgit.cgi/linux-2.6-windriver/'></ulink>.
1257 The main difference between them is that "emenlow" is in its own layer.
1258 It is in its own layer because it needs extra machine-specific packages such as its
1259 own video driver and other supporting packages.
1260 The "atom-pc" is simpler and does not need any special packages - everything it needs can
1261 be specified in the configuration file.
1262 The "atom-pc" machine also supports all of Asus eee901, Acer Aspire One, Toshiba NB305,
1263 and the Intel&reg; Embedded Development Board 1-N450 with no changes.</para>
1264 <para>If you want to make minor changes to support a slightly different machine, you can
1265 create a new configuration file for it and add it alongside the others.
1266 You might consider keeping the common stuff separate and including it.</para>
1267 <para>Similarly, you can also use multiple configuration files for different machines even
1268 if you do it as a separate layer like meta-emenlow.</para>
1269 <para>As an example consider this:
1270 <itemizedlist>
1271 <listitem><para>Copy meta-emenlow</para></listitem>
1272 <listitem><para>Fix or remove anything you do not need.
1273 For this example the only thing left was the kernel directory with a linux-yocto_git.bbappend
1274 file (linux-yocto is the kernel listed in
1275 <filename>meta-crownbay/conf/machine/crownbay.conf</filename>.
1276 Finally, a new entry to the <filename>build/donf/bblayers.conf</filename> was added so the
1277 new layer could be found by Bitbake.</para></listitem>
1278 </itemizedlist>
1279 </para></listitem>
1280 <listitem><para>Get an image with a working kernel built.</para>
1281 <para>For the kernel to compile successfully, you need to create a branch in the git repository
1282 specifically named for your machine.
1283 So first create a bare clone of the Yocto Project git repository, and then create a
1284 local clone of that:
1285 <literallayout class='monospaced'>
1286 $ git clone &dash;&dash;bare git://git.pokylinux.org/linux-2.6-windriver.git
1287 linux-2.6-windriver.git
1288 $ git clone linux-2.6-windriver.git linux-2.6-windriver
1289 </literallayout>
1290 </para>
1291 <para>Now create a branch in the local clone and push it to the bare clone:
1292 <literallayout class='monospaced'>
1293 $ git checkout -b crownbay-standard origin/standard $ git push origin crownbay-standard:crownbay-standard
1294 </literallayout>
1295 </para>
1296 <para>At this point, your git tree should be set up well enough to compile.</para></listitem>
1297 <listitem><para>Point the build at the new kernel git tree.</para>
1298 <para>You can do this by commenting out the SRC_URI variable in
1299 <filename>meta/recipes-kernel/linux/linux-yocto_git.bb</filename> and using a SRC_URI
1300 that points to your new bare git tree.
1301 You should also be able to do this in <filename>linux-yocto_git.bbappend</filename> in the layer:
1302 <literallayout class='monospaced'>
1303 # To use a staged, on-disk bare clone of a Wind River Kernel, use a variant of the
1304 # below SRC_URI = "git://///path/to/kernel/default_kernel.git;fullclone=1"
1305 #
1306 SRC_URI = "git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;fullclone=1;branch=${KBRANCH};name=machine
1307\
1308 git://git.pokylinux.org/linux-2.6-windriver.git;protocol=git;noclone=1;branch=wrs_meta;name=meta"
1309 </literallayout>
1310 </para>
1311 <para>After doing that, select the machine in <filename>build/conf/local.conf</filename>:
1312 <literallayout class='monospaced'>
1313 #
1314 MACHINE ?= "crownbay"
1315 #
1316 </literallayout>
1317 </para>
1318 <para>You should now be able to build and boot an image with the new kernel:
1319 <literallayout class='monospaced'>
1320 $ bitbake poky-image-sato-live
1321 </literallayout>
1322 </para>
1323 <para>Of course, that will give you a kernel with the default config, which is probably
1324 not what you want.
1325 If you just want to set some kernel config options, you can do that by putting them in a files.
1326 For example inserting the following into some <filename>.cfg</filename> file:
1327 <literallayout class='monospaced'>
1328 CONFIG_NETDEV_1000=y
1329 CONFIG_E1000E=y
1330 </literallayout>
1331 </para>
1332 <para>And, another <filename>.cfg</filename> file would contain:
1333 <literallayout class='monospaced'>
1334 CONFIG_LOG_BUF_SHIFT=18
1335
1336 http://git.pokylinux.org/cgit/cgit.cgi/linux-2.6-windriver/
1337
1338 SRC_URI_append_crownbay = " file://some.cfg \
1339 file://other.cfg \
1340 "
1341 </literallayout>
1342 </para>
1343 <para>You could also add these directly to the git repo's wrs_meta branch as well.
1344 However, the former method is probably easier.</para></listitem>
1345 <listitem><para>If you're also adding patches to the kernel, you can do the same thing.
1346 Put your patches in the SRC_URI as well (plus .cfg for their kernel config options if needed).</para>
1347 <para>Practically speaking, to generate the patches, you'd go to the source in the build tree:
1348 <literallayout class='monospaced'>
1349 build/tmp/work/crownbay-poky-linux/linux-yocto-2.6.34+git0+d1cd5c80ee97e81e130be8c3de3965b770f320d6_0+
13500431115c9d720fee5bb105f6a7411efb4f851d26-r13/linux
1351 </literallayout>
1352 </para>
1353 <para>Then, modify the code there, using quilt to save the changes, and recompile
1354 (bitbake -c compile -f)
1355 until it works.</para></listitem>
1356 <listitem><para>Once you have the final patch from quilt, copy it to the
1357 SRC_URI location, and it should be
1358 applied the next time you do a clean build.
1359 Of course, since you have a branch for the BSP in git, it would be better to put it there instead.
1360 For example, in this case, commit the patch to the crownbay-standard branch, and during the
1361 next build it will be applied from there.</para></listitem>
1362 </orderedlist>
1363 </para>
1364 </section>
1365
1366
1367
1368
1369
1370
1371
1372
1373<!-- <section id='bsp-creating-a-new-bsp'>
1374 <title>BSP: Creating a New BSP</title>
1375<para>
1376Although it is obvious that the structure of a new BSP uses the migrated
1377directory structure from the previous example,the first question is whether
1378or not the BSP is started from scratch.
1379</para>
1380<para>
1381If Yocto Project has a similar BSP, it is often easier to clone and update,
1382rather than start from scratch. If the mainline kernel has support, it is
1383easier to branch from the -standard kernel and begin development (and not be
1384concerned with undoing existing changes). This section covers both options.
1385</para>
1386<para>
1387In almost every scenario, the LDAT build system bindings must be completed
1388before either cloning or starting a new BSP from scratch. This is simply
1389because the board template files are required to configure a project/build
1390and create the necessary environment to begin working directly with the
1391kernel. If it is desired to start immediately with kernel development and
1392then add LDAT bindings, see the "bootstrapping a BSP" section.
1393</para>
1394 <section id='creating-from-scratch'>
1395 <title>Creating the BSP from Scratch</title>
1396<para>
1397To create the BSP from scratch you need to do the following:
1398<orderedlist>
1399 <listitem><para>Create a board template for the new BSP in a layer.</para></listitem>
1400 <listitem><para>Configure a build with the board.</para></listitem>
1401 <listitem><para>Configure a kernel.</para></listitem>
1402</orderedlist>
1403</para>
1404<para>
1405Following is an example showing all three steps. You start by creating a board template for the new BSP in a layer.
1406<literallayout class='monospaced'>
1407 templates/
1408 `&dash;&dash; board
1409 `&dash;&dash; my_bsp
1410 |&dash;&dash; include
1411 |&dash;&dash; config.sh
1412 `&dash;&dash; linux
1413 |&dash;&dash; my_bsp.cfg
1414 `&dash;&dash; my_bsp-standard.scc
1415
1416 &gt; cat config.sh
1417 TARGET_BOARD="my_bsp"
1418 TARGET_LINUX_LINKS="bzImage"
1419 TARGET_SUPPORTED_KERNEL="standard"
1420 TARGET_SUPPORTED_ROOTFS="glibc_std"
1421 BANNER="This BSP is *NOT* supported"
1422 TARGET_PROCFAM="pentium4"
1423 TARGET_PLATFORMS="GPP"
1424
1425 &gt; cat include
1426 cpu/x86_32_i686
1427 karch/i386
1428
1429 &gt; cat linux/my_bsp-standard.scc
1430 scc_leaf ktypes/standard/standard.scc my_bsp-standard
1431
1432 &gt; cat linux/my_bsp.cfg
1433 CONFIG_X86=y
1434 CONFIG_SMP=y
1435 CONFIG_VT=y
1436 # etc, etc, etc
1437</literallayout>
1438</para>
1439<para>
1440Something like the following can now be added to a board build, and
1441a project can be started:
1442<literallayout class='monospaced'>
1443 &dash;&dash;enable-board=my_bsp \
1444 &dash;&dash;with-layer=custom_bsp
1445</literallayout>
1446</para>
1447<para>
1448Now you can configure a kernel:
1449<literallayout class='monospaced'>
1450 &gt; make -C build linux.config
1451</literallayout>
1452</para>
1453<para>
1454You now have a kernel tree, which is branched and has no patches, ready for
1455development.
1456</para>
1457 </section> -->
1458
1459<!-- <section id='cloning-an-existing-bsp'>
1460 <title>Cloning an Existing BSP</title>
1461<para>
1462Cloning an existing BSP from the shipped product is similar to the "from
1463scratch" option and there are two distinct ways to achieve this goal:
1464<itemizedlist>
1465 <listitem><para>Create a board template for the new BSP in a layer.</para></listitem>
1466 <listitem><para>Clone the .scc and board config.</para></listitem>
1467</itemizedlist>
1468</para>
1469<para>
1470The first method is similar to the from scratch BSP where you create a board template for the new
1471BSP. Although in this case, copying an existing board template from
1472wrll-wrlinux/templates/board would be appropriate, since we are cloning an
1473existing BSP. Edit the config.sh, include and other board options for the new
1474BSP.
1475</para>
1476<para>
1477The second method is to clone the .scc and board config.
1478To do this, in the newly created board template, create a linux subdirectory and export
1479the .scc and configuration from the source BSP in the published Yocto Project
1480kernel. During construction, all of the configuration and patches were
1481captured, so it is simply a matter of extracting them.
1482</para>
1483<para>
1484Extraction can be accomplished using four different techniques:
1485<itemizedlist>
1486 <listitem><para>Config and patches from the bare default_kernel.</para></listitem>
1487 <listitem><para>Clone default_kernel and checkout wrs_base.</para></listitem>
1488 <listitem><para>Clone default_kernel and checkout BSP branch.</para></listitem>
1489 <listitem><para>Branch from the Yocto Project BSP.</para></listitem>
1490</itemizedlist>
1491</para>
1492<para>
1493Technique 1: config and patches from the bare default_kernel
1494<literallayout class='monospaced'>
1495 &gt; cd layers/wrll-linux-2.6.27/git/default_kernel
1496 &gt; git show checkpoint_end | filterdiff -i '*common_pc*' | patch -s -p2 -d /tmp
1497
1498 # This will create two directories: cfg and patches.
1499
1500 &gt; cd /tmp/cfg/kernel-cache/bsp/common_pc/
1501
1502 # This directory contains all the patches and .scc files used to construct
1503 # the BSP in the shipped tree. Copy the patches to the new BSP template,
1504 # and add them to the .scc file created above. See "template patching" if
1505 # more details are required.
1506</literallayout>
1507</para>
1508<para>
1509Technique 2: clone default_kernel and checkout wrs_base
1510<literallayout class='monospaced'>
1511 &gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
1512 &gt; cd windriver-2.6.27
1513 &gt; git checkout wrs_base
1514 &gt; cd wrs/cfg/kernel-cache/bsp/common_pc
1515
1516# again, this directory has all the patches and .scc files used to construct
1517# the BSP
1518</literallayout>
1519</para>
1520<para>
1521Technique 3: clone default_kernel and checkout BSP branch
1522<literallayout class='monospaced'>
1523 &gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
1524 &gt; cd windriver-2.6.27
1525 &gt; git checkout common_pc-standard
1526 &gt; git whatchanged
1527 # browse patches and determine which ones are of interest, say there are
1528 # 3 patches of interest
1529 &gt; git format-patch -o &lt;path to BSP template&gt;/linux HEAD^^^
1530 # update the .scc file to add the patches, see "template patches" if
1531 # more details are required
1532</literallayout>
1533</para>
1534<para>
1535Technique #4: branch from the Yocto Project BSP
1536<note><para>This is potentially the most "different" technique, but is actually
1537 the easiest to support and leverages the infrastructure. rtcore BSPs
1538 are created in a similar manner to this.
1539</para></note>
1540</para>
1541<para>
1542In this technique the .scc file in the board template is slightly different
1543 and indicates that the BSP should branch after the base Yocto Project BSP
1544 of the correct kernel type, so to start a new BSP that inherits the
1545 kernel patches of the common_pc-standard, the following would be done:
1546<literallayout class='monospaced'>
1547 &gt; cat linux/my_bsp-standard.scc
1548 scc_leaf bsp/common_pc/common_pc-standard.scc my_bsp-standard
1549</literallayout>
1550</para>
1551<para>
1552 And only kernel configuration (not patches) need be contained in the
1553 board template.
1554</para>
1555<para>
1556 This has the advantage of automatically picking up updates to the BSP
1557 and not duplicating any patches for a similar board.
1558</para>
1559 </section> -->
1560
1561 <!-- <section id='bsp-bootstrapping'>
1562 <title>BSP: Bootstrapping</title>
1563<para>
1564The previous examples created the board templates and configured a build
1565before beginning work on a new BSP. It is also possible for advanced users to
1566simply treat the Yocto Project git repository as an upstream source and begin
1567BSP development directly on the repository. This is the closest match to how
1568the kernel community at large would operate.
1569</para>
1570<para>
1571Two techniques exist to accomplish this:
1572</para>
1573<para>
1574Technique 1: upstream workflow
1575<literallayout class='monospaced'>
1576 &gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
1577 &gt; cd windriver-2.6.27
1578 &gt; git checkout -b my_bsp-standard common_pc-standard
1579
1580 # edit files, import patches, generally do BSP development
1581
1582 # at this point we can create the BSP template, and export the kernel
1583 # changes using one of the techniques discussed in that section. For
1584 # example, It is possible to push these changes, directly into the
1585 # default_kernel and never directly manipulate or export patch files
1586</literallayout>
1587</para>
1588<para>
1589Technique 2: Yocto Project kernel build workflow
1590</para>
1591<para>
1592 Create the BSP branch from the appropriate kernel type
1593<literallayout class='monospaced'>
1594 &gt; cd linux
1595 # the naming convention for auto-build is &lt;bsp&gt;-&lt;kernel type&gt;
1596 &gt; git checkout -b my_bsp-standard standard
1597</literallayout>
1598</para>
1599<para>
1600Make changes, import patches, etc.
1601<literallayout class='monospaced'>
1602 &gt; ../../host-cross/bin/guilt init
1603 # 'wrs/patches/my_bsp-standard' has now been created to
1604 # manage the branches patches
1605
1606 # option 1: edit files, guilt import
1607 &gt; ../../host-cross/bin/guilt new extra-version.patch
1608 &gt; vi Makefile
1609 &gt; ../../host-cross/bin/guilt refresh
1610 # add a header
1611 &gt; ../../host-cross/bin/guilt header -e
1612 # describe the patch using best practices, like the example below:
1613
1614 &dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt; cut here
1615 From: Bruce Ashfield &lt;bruce.ashfield@windriver.com&gt;
1616
1617 Adds an extra version to the kernel
1618
1619 Modify the main EXTRAVERSION to show our bsp name
1620
1621 Signed-off-by: Bruce Ashfield &lt;bruce.ashfield@windriver.com&gt;
1622 &dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt; cut here
1623
1624 # option 2: import patches
1625 &gt; git am &lt;patch&gt;
1626 or
1627 &gt; git apply &lt;patch&gt;
1628 &gt; git add &lt;files&gt;
1629 &gt; git commit -s
1630
1631 # configure the board, save relevant options
1632 &gt; make ARCH=&lt;arch&gt; menuconfig
1633
1634 # save the cfg changes for reconfiguration
1635 &gt; mkdir wrs/cfg/&lt;cache&gt;/my_bsp
1636 &gt; vi wrs/cfg/&lt;cache&gt;/my_bsp/my_bsp.cfg
1637
1638 # classify the patches
1639 &gt; ../../host-cross/bin/kgit classify create &lt;kernel-foo-cache&gt;/my_bsp/my_bsp
1640 # test build
1641 &gt; cd ..
1642 &gt; make linux TARGET_BOARD=my_bsp kprofile=my_bsp use_current_branch=1
1643</literallayout>
1644</para>
1645<para>
1646 Assuming the patches have been exported to the correct location, Future
1647 builds will now find the board, apply the patches to the base tree and make
1648 the relevant branches and structures and the special build options are no
1649 longer required.
1650</para>
1651 </section>
1652 </section> -->
1653
1654<!-- <section id='patching'>
1655 <title>Patching</title>
1656<para>
1657The most common way to apply patches to the kernel is via a template.
1658However, for more advanced applications (such as the sharing of patches between
1659multiple sub-features) it is possible to patch the kernel-cache.
1660This section covers both scenarios.
1661</para>
1662 <section id='patching-template'>
1663 <title>Patching: Template</title>
1664<para>
1665kernel
1666templates follow the same rules as any LDAT template. A directory should be
1667created in a recognized template location, with a 'linux' subdirectory. The
1668'linux' directory triggers LDAT to pass the dir as a potential patch location
1669to the kernel build system. Any .scc files found in that directory, will be
1670automatically appended to the end of the BSP branch (for the configured
1671board).
1672</para>
1673<para>
1674This behavior is essentially the same since previous product
1675releases. The only exception is the use of ".scc", which allows kernel
1676configuration AND patches to be applied in a template.
1677</para>
1678<note><para>
1679If creating a full template is not required, a .scc file can be placed at
1680the top of the build, along with configuration and patches. The build
1681system will pickup the .scc and add it onto the patch list automatically
1682</para></note>
1683<para>
1684As an example, consider a simple template to update a BP:
1685<literallayout class='monospaced'>
1686 &gt; cat templates/feature/extra_version/linux/extra_version.scc
1687 patch 0001-extraversion-add-Wind-River-identifier.patch
1688</literallayout>
1689</para>
1690<para>
1691To illustrate how the previous template patch was created, the following
1692steps were performed:
1693<literallayout class='monospaced'>
1694 &gt; cd &lt;board build&gt;/build/linux
1695 &gt; vi Makefile
1696 # modify EXTRAVERSION to have a unique string
1697 &gt; git commit -s -m "extraversion: add Yocto Project identifier" Makefile
1698 &gt; git format-patch -o &lt;path to layer&gt;/templates/feature/extra_version/linux/
1699 &gt; echo "patch 0001-extraversion-add-Wind-River-identifier.patch" &gt; \
1700 &lt;path to layer&gt;/templates/feature/extra_version/linux/extra_version.scc
1701</literallayout>
1702</para>
1703<para>
1704This next example creates a template with a linux subdirectory, just as we
1705 always have for previous releases.
1706<literallayout class='monospaced'>
1707 &gt; mkdir templates/features/my_feature/linux
1708</literallayout>
1709</para>
1710<para>
1711 In that directory place your feature description, your
1712 patch and configuration (if required).
1713<literallayout class='monospaced'>
1714 &gt; ls templates/features/my_feature/linux
1715
1716 version.patch
1717 my_feature.scc
1718 my_feature.cfg
1719</literallayout>
1720</para>
1721<para>
1722 The .scc file describes the patches, configuration and
1723 where in the patch order the feature should be inserted.
1724<literallayout class='monospaced'>
1725 patch version.patch
1726 kconf non-hardware my_feature.cfg
1727</literallayout>
1728</para>
1729<para>
1730 Configure your build with the new template
1731<literallayout class='monospaced'>
1732 &dash;&dash;with-template=features/my_feature
1733</literallayout>
1734</para>
1735<para>
1736Build the kernel
1737<literallayout class='monospaced'>
1738 &gt; make linux
1739</literallayout>
1740</para>
1741 </section>
1742
1743 <section id='patching-kernel-cache'>
1744 <title>Patching: Kernel Cache</title>
1745<para>
1746As previously mentioned, this example is included for completeness, and is for more advanced
1747applications (such as the sharing of patches between multiple sub-features).
1748Most patching should be done via templates, since that interface is
1749guaranteed not to change and the kernel-cache interface carries no such
1750guarantee.
1751</para>
1752<para>
1753At the top of a layer, create a kernel cache. The build system will recognize
1754any directory of the name 'kernel-*-cache' as a kernel cache.
1755<literallayout class='monospaced'>
1756 &gt; cd &lt;my layer&gt;
1757 &gt;mkdir kernel-temp-cache
1758</literallayout>
1759</para>
1760<para>
1761Make a directory with the BSP
1762<literallayout class='monospaced'>
1763 &gt; mkdir kernel-temp-cache
1764 &gt; mkdir kernel-temp-cache/my_feat
1765</literallayout>
1766</para>
1767<para>
1768Create the feature files as they were in technique #1
1769<literallayout class='monospaced'>
1770 &gt; echo "patch my_patch.path" &gt; kernel-temp-cache/my_feat/my_feature.scc
1771</literallayout>
1772</para>
1773<para>
1774Configure the build with the feature added to the kernel type
1775<literallayout class='monospaced'>
1776 &dash;&dash;with-kernel=standard+my_feat/my_feature.scc
1777</literallayout>
1778</para>
1779<para>
1780Build the kernel
1781<literallayout class='monospaced'>
1782 &gt; make linux
1783</literallayout>
1784</para>
1785 </section>
1786 </section>
1787
1788 <section id='bsp-updating-patches-and-configuration'>
1789 <title>BSP: Updating Patches and Configuration</title>
1790<para>
1791As was described in the "template patching" example, it is simple
1792to add patches to a BSP via a template, but often, it is desirable
1793to experiment and test patches before committing them to a template.
1794You can do this by modifying the BSP source.
1795</para>
1796<para>
1797Start as follows:
1798<literallayout class='monospaced'>
1799 &gt; cd linux
1800 &gt; git checkout &lt;bspname&gt;-&lt;kernel name&gt;
1801
1802 &gt; git am &lt;patch&gt;
1803</literallayout>
1804</para>
1805<para>
1806Or you can do this:
1807<literallayout class='monospaced'>
1808 &gt; kgit-import -t patch &lt;patch&gt;
1809
1810 &gt; cd ..
1811 &gt; make linux
1812</literallayout>
1813</para>
1814<para>
1815For details on conflict resolution and patch application, see the
1816git manual, or other suitable online references.
1817<literallayout class='monospaced'>
1818 &gt; git am &lt;mbox&gt;
1819 # conflict
1820 &gt; git apply &dash;&dash;reject .git/rebase-apply/0001
1821 # resolve conflict
1822 &gt; git am &dash;&dash;resolved (or git am &dash;&dash;skip, git am &dash;&dash;abort)
1823 # continue until complete
1824</literallayout>
1825</para>
1826<para>
1827Here is another example:
1828<literallayout class='monospaced'>
1829 # merge the patches
1830 # 1) single patch
1831 &gt; git am &lt;mbox&gt;
1832 &gt; git apply &lt;patch&lt;
1833 &gt; kgit import -t patch &lt;patch&gt;
1834
1835 # 2) multiple patches
1836 &gt; git am &lt;mbox&gt;
1837 &gt; kgit import -t dir &lt;dir&gt;
1838
1839 # if kgit -t dir is used, a patch resolution cycle such
1840 # as this can be used:
1841
1842 &gt; kgit import -t dir &lt;dir&gt;
1843 # locate rejects and resolve
1844 # options:
1845 &gt; wiggle &dash;&dash;replace &lt;path to file&gt; &lt;path to reject&gt;
1846 &gt; guilt refresh
1847 or
1848 &gt; # manual resolution
1849 &gt; git add &lt;files&gt;
1850 &gt; git commit -s
1851 or
1852 &gt; git apply &dash;&dash;reject .git/rebase-apply/0001
1853 &gt; git add &lt;files&gt;
1854 &gt; git am &dash;&dash;resolved
1855 or
1856 &gt; # merge tool of choice
1857
1858 # continue series:
1859
1860 &gt; kgit import -t dir &lt;dir&gt;
1861 or
1862 &gt; git am &dash;&dash;continue
1863</literallayout>
1864</para>
1865<para>
1866Once all the patches have been tested and are satisfactory, they
1867should be exported via the techniques described in "saving kernel
1868modifications."
1869</para>
1870<para>
1871Once the kernel has been patched and configured for a BSP, it's
1872configuration commonly needs to be modified. This can be done by
1873running [menu|x]config on the kernel tree, or working with
1874configuration fragments.
1875</para>
1876<para>
1877Using menuconfig, the operation is as follows:
1878<literallayout class='monospaced'>
1879 &gt; make linux.menuconfig
1880 &gt; make linux.rebuild
1881</literallayout>
1882</para>
1883<para>
1884Once complete, the changes are in linux-&lt;bsp&gt;-&lt;kernel type&gt;-build/.config.
1885To permanently save these changes, compare the .config before and after the
1886menuconfig, and place those changes in a configuration fragment in the
1887template of your choice.
1888</para>
1889<para>
1890Using configuration fragments, the operation is as follows (using the
1891si_is8620 as an example BSP):
1892<literallayout class='monospaced'>
1893 &gt; vi linux/wrs/cfg/kernel-cache/bsp/si_is8620/si_is8620.cfg
1894 &gt; make linux.reconfig
1895 &gt; make linux.rebuild
1896</literallayout>
1897</para>
1898<para>
1899The modified configuration fragment can simply be copied out of the
1900linux/wrs/.. directory and placed in the appropriate template for future
1901application.
1902</para>
1903 </section>
1904
1905 <section id='tools-guilt'>
1906 <title>Tools: guilt</title>
1907<para>
1908Yocto Project has guilt integrated as a kernel tool; therefore users that are
1909familiar with quilt may wish to use this tool to pop, push and refresh
1910their patches. Note: guilt should only be used for local operations, once
1911a set of changes has been pushed or pulled, they should no longer be popped
1912or refresh by guilt, since popping, refreshing and re-pushing patches
1913changes their commit IDs and creating non-fast forward branches.
1914</para>
1915<para>
1916The following example illustrates how to add patches a Yocto Project
1917BSP branch via guilt:
1918<literallayout class='monospaced'>
1919 &gt; cd build/linux
1920 &gt; git checkout common_pc-standard
1921 &gt; guilt new extra.patch
1922 # edit files, make changes, etc
1923 &gt; guilt refresh
1924 &gt; guilt top
1925 extra.patch
1926
1927 # export that patch to an external location
1928 &gt; kgit export -p top /tmp
1929</literallayout>
1930</para>
1931<para>
1932Other guilt operations of interest are:
1933<literallayout class='monospaced'>
1934 > guilt push, guilt push -a
1935 > guilt pop
1936 > guilt applied, guilt unapplied
1937 > guilt top
1938 > guilt refresh
1939 > guilt header -e
1940 > guilt next
1941</literallayout>
1942</para>
1943<note><para>
1944Guilt only uses git commands and git plumbing to perform its operations,
1945anything that guilt does can also be done using git directly. It is provided
1946as a convenience utility, but is not required and the developer can use whatever
1947tools or workflow they wish.
1948</para></note>
1949<para>
1950The following builds from the above instructions to show how guilt can be
1951used to assist in getting your BSP kernel patches ready. You should follow
1952the above instructions up to and including 'make linux.config'. In this
1953example I will create a new commit (patch) from scratch and import another
1954fictitious patch from some external public git tree (ie, a commit with full
1955message, signoff etc.). Please ensure you have host-cross/bin in your path.
1956<literallayout class='monospaced'>
1957 %> cd linux
1958 %> guilt-init
1959 %> guilt-new -m fill_me_in_please first_one.patch
1960 %> touch somefile.txt
1961 %> guilt-add somefile.txt
1962 %> guilt-header -e
1963 %> guilt-refresh
1964 %> guilt-import path_to_some_patch/patch_filename
1965 %> guilt-push
1966</literallayout>
1967</para>
1968<para>
1969Here are a few notes about the above:
1970<itemizedlist>
1971 <listitem><para>guilt-header -e &dash;&dash; this will open editing of the patch header in
1972 EDITOR. As with a git commit the first line is the short log and
1973 should be just that short and concise message about the commit. Follow
1974 the short log with lines of text that will be the long description but
1975 note Do not put a blank line after the short log. As usual you will
1976 want to follow this with a blank line and then a signoff line.</para></listitem>
1977
1978 <listitem><para>The last line in the example above has 2 dots on the end. If you
1979 don't add the 2 periods on the end guilt will think you are sending
1980 just one patch. The wrong one!</para></listitem>
1981
1982 <listitem><para>The advantage to using guilt over not using guilt is that if you have a
1983 review comment in the first patch (first_one.patch in the case of this
1984 example) it is very easy to use guilt to pop the other patches off
1985 allowing you to make the necessary changes without having to use more
1986 inventive git type strategies.</para></listitem>
1987</itemizedlist>
1988</para>
1989 </section>
1990
1991 <section id='tools-scc-file-example'>
1992 <title>Tools: scc File Example</title>
1993<para>
1994This section provides some scc file examples: leaf node, 'normal' mode, and transforms.
1995</para>
1996 <section id='leaf-node'>
1997 <title>Leaf Node</title>
1998<para>
1999The following example is a BSP branch with no child branches - a leaf on the tree.
2000<literallayout class='monospaced'>
2001 # these are optional, but allow standalone tree construction
2002 define WRS_BOARD &lt;name&gt;
2003 define WRS_KERNEL &lt;kern type&gt;
2004 define WRS_ARCH &lt;arch&gt;
2005
2006 scc_leaf ktypes/standard common_pc-standard
2007 # ^ ^
2008 # +&dash;&dash; parent + branch name
2009
2010 include common_pc.scc
2011 # ^
2012 # +&dash;&dash;&dash; include another feature
2013</literallayout>
2014</para>
2015 </section>
2016
2017 <section id='normal-mode'>
2018 <title>'Normal' Mode</title>
2019<para>
2020Here is an example of 'normal' mode:
2021<literallayout class='monospaced'>
2022 # +&dash;&dash;&dash;&dash; name of file to read
2023 # v
2024 kconf hardware common_pc.cfg
2025 # ^ ^
2026 # | +&dash;&dash; 'type: hardware or non-hardware
2027 # |
2028 # +&dash;&dash;&dash; kernel config
2029
2030 # patches
2031 patch 0002-atl2-add-atl2-driver.patch
2032 patch 0003-net-remove-LLTX-in-atl2-driver.patch
2033 patch 0004-net-add-net-poll-support-for-atl2-driver.patch
2034</literallayout>
2035</para>
2036
2037 </section>
2038
2039 <section id='transforms'>
2040 <title>Transforms</title>
2041<para>
2042This section shows an example of transforms:
2043<literallayout class='monospaced'>
2044 # either of the next two options will trigger an 'auto'
2045 # branch from existing ones, since they change the commit
2046 # order and hence must construct their own branch
2047
2048 # this changes the order of future includes, if the
2049 # passed feature is detected, the first feature is
2050 # included AFTER it
2051 include features/rt/rt.scc after features/kgdb/kgdb
2052 # this also changes the order of existing branches
2053 # this prevents the named feature from ever being
2054 # included
2055 exclude features/dynamic_ftrace/dynamic_ftrace.scc
2056
2057 # inherit the standard kernel
2058 include ktypes/standard/standard
2059
2060
2061 # LTT supplies this, so we don't want the sub-chunk from RT.
2062 patch_trigger arch:all exclude ftrace-upstream-tracepoints.patch
2063 # ...but we still want the one unique tracepoint it added.
2064 patch tracepoint-add-for-sched_resched_task.patch
2065
2066 # these will change the named patches in the series into
2067 # &lt;patch name&gt;.patch.&lt;feature name&gt;
2068 # where the substituted patch is in this directory
2069 patch_trigger arch:all ctx_mod dynamic_printk.patch
2070 patch_trigger arch:all ctx_mod 0001-Implement-futex-macros-for-ARM.patch
2071 # unconditionally exclude a patch
2072 patch_trigger arch:all exclude ftrace-fix-ARM-crash.patch
2073</literallayout>
2074</para>
2075 </section>
2076 </section> -->
2077
2078 <section id='tip-dirty-string'>
2079 <title>"-dirty" String</title>
2080<para>
2081If kernel images are being built with -dirty on the end of the version
2082string, this simply means that there are modification in the source
2083directory that haven't been committed.
2084<literallayout class='monospaced'>
2085 &gt; git status
2086</literallayout>
2087</para>
2088<para>
2089The above git command will indicate modified, removed or added files. Those changes should
2090be committed to the tree (even if they will never be saved, or exported
2091for future use) and the kernel rebuilt.
2092</para>
2093<para>
2094To brute force pickup and commit all such pending changes enter the following:
2095<literallayout class='monospaced'>
2096 &gt; git add .
2097 &gt; git commit -s -a -m "getting rid of -dirty"
2098</literallayout>
2099</para>
2100<para>
2101And then rebuild the kernel
2102</para>
2103 </section>
2104
2105 <section id='kernel-transition-kernel-layer'>
2106 <title>Kernel: Transition Kernel Layer</title>
2107<para>
2108In order to temporarily use a different base kernel in Yocto Project
2109Linux 3.0 you need to do the following:
2110<orderedlist>
2111 <listitem><para>Create a custom kernel layer.</para></listitem>
2112 <listitem><para>Create a git repository of the transition kernel.</para></listitem>
2113</orderedlist>
2114</para>
2115<para>
2116Once those requirements are met multiple boards and kernels can
2117be built. The cost of setup is only paid once and then additional
2118BSPs and options can be added.
2119</para>
2120<para>
2121This creates a transition kernel layer to evaluate functionality
2122of some other kernel with the goal of easing transition to an
2123integrated and validated Yocto Project kernel.
2124</para>
2125<!--<para>
2126The next few sections describe the process:
2127</para> -->
2128 <!-- <section id='creating-a-custom-kernel-layer'>
2129 <title>Creating a Custom Kernel Layer</title>
2130<para>
2131The custom kernel layer must have the following minimum
2132elements:
2133<itemizedlist>
2134 <listitem><para>An include of the shipped Yocto Project kernel layer.</para></listitem>
2135 <listitem><para>A kernel-cache with an override of the standard kernel type.</para></listitem>
2136</itemizedlist>
2137</para>
2138<para>
2139This allows the inheritance of the kernel build infrastructure,
2140while overriding the list of patches that should be applied to
2141the base kernel.
2142</para>
2143<para>
2144The kernel layer can optionally include an override to the base
2145Yocto Project Linux BSP to inhibit the application of BSP specific
2146patches. If a custom BSP is being used, this is not required.
2147</para>
2148 </section> -->
2149
2150 <!-- <section id='git-repo-of-the-transition-kernel'>
2151 <title>git Repo of the Transition Kernel</title>
2152<para>
2153The kernel build system requires a base kernel repository to
2154seed the build process. This repository must be found in the
2155same layer as the build infrastructure (i.e wrll-linux-2.6.27)
2156in the 'git' subdir, with the name 'default_kernel'
2157</para>
2158<para>Since Yocto Project Linux ships with a default_kernel
2159(the validated Yocto Project kernel) in the wrll-linux-2.6.27
2160kernel layer, that must be removed and replaced with the
2161transition kernel.
2162</para>
2163<para>If the Yocto Project install cannot be directly modified
2164with the new default kernel, then the path to the transition
2165kernel layer's 'git' subdir must be passed to the build
2166process via:
2167<programlisting>
2168linux_GIT_BASE=&lt;absolute path to layer&gt;/git
2169</programlisting>
2170</para>
2171<para>
2172If the transition kernel has not been delivered via git,
2173then a git repo should be created, and bare cloned into
2174place. Creating this repository is as simple as:
2175<literallayout class='monospaced'>
2176 &gt; tar zxvf temp_kernel.tgz
2177 &gt; cd temp_kernel
2178 &gt; git init
2179 &gt; git add .
2180 &gt; git commit -a -m "Transition kernel baseline"
2181
2182 'temp_kernel' can now be cloned into place via:
2183
2184 &gt; cd &lt;path to git base&gt;/git
2185 &gt; git clone &dash;&dash;bare &lt;path to temp_kernel/temp_kernel default_kernel
2186</literallayout>
2187</para>
2188 </section> -->
2189
2190 <!-- <section id='building-the-kernel'>
2191 <title>Building the Kernel</title>
2192<para>
2193Once these prerequisites have been met, the kernel can be
2194built with:
2195<literallayout class='monospaced'>
2196 &gt; make linux
2197</literallayout>
2198</para>
2199<para>
2200The new base kernel will be cloned into place and have any patches
2201indicated in the transition kernel's cache (or templates) applied.
2202The kernel build will detect the non-Yocto Project base repo and
2203use the HEAD of the tree for the build.
2204</para>
2205 </section> -->
2206
2207 <!-- <section id='example'>
2208 <title>Example</title>
2209<para>
2210This example creates a kernel layer to build the latest
2211kernel.org tree as the 'common_pc' BSP.
2212<literallayout class='monospaced'>
2213 &gt; cd &lt;path to layers&gt;
2214 &gt; mkdir wrll-linux-my_version
2215 &gt; cd wrll-linux-my_version
2216 &gt; echo "wrll-linux-2.6.27" &gt; include
2217 &gt; mkdir -p kernel-cache/ktypes/standard
2218 &gt; mkdir -p kernel-cache/bsp/common_pc
2219 &gt; echo "v2.6.29" &gt; kernel-cache/kver
2220 &gt; echo "branch common_pc-standard" &gt; kernel-cache/bsp/common_pc/common_pc.scc
2221 &gt; echo "kconf hardware common_pc.cfg" &gt;&gt; kernel-cache/bsp/common_pc/common_pc.scc
2222 &gt; echo "CONFIG_FOO=y" &gt; kernel-cache/bsp/common_pc/common_pc.cfg
2223 &gt; mkdir git
2224 &gt; cd git
2225 &gt; git clone &dash;&dash;bare git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git default_kernel
2226</literallayout>
2227</para>
2228<para>
2229Configure a build to use the new layer. This means that:
2230<literallayout class='monospaced'>
2231 &dash;&dash;enable-kernel-version=my_version
2232</literallayout>
2233</para>
2234<para>
2235Should be used to override the shipped default.
2236</para>
2237<para>
2238To build the kernel:
2239<literallayout class='monospaced'>
2240 &gt; cd build
2241 &gt; make linux_GIT_BASE=&lt;layer path&gt;/wrll-linux-my_version/git linux
2242</literallayout>
2243</para>
2244<para>
2245If this is to build without some user intervention (passing of the
2246GIT_BASE), you must do the clone into the wrll-linux-2.6.27/git directory.
2247</para>
2248<note><para>Unless you define valid "hardware.kcf" and "non-hardware.kcf" some
2249non fatal warnings will be seen. They can be fixed by populating these
2250files in the kernel-cache with valid hardware and non hardware config
2251options.
2252</para></note>
2253 </section> -->
2254 </section>
2255 </section>
2256
2257
2258
2259
2260
2261<!-- <itemizedlist>
2262 <listitem><para>Introduction to this section.</para></listitem>
2263 <listitem><para>Constructing a project-specific kernel tree.</para></listitem>
2264 <listitem><para>Building the kernel.</para></listitem>
2265 <listitem><para>Seeing what has changed.</para></listitem>
2266 <listitem><para>Seeing what has changed in a particular branch.</para></listitem>
2267 <listitem><para>Modifying the kernel.</para></listitem>
2268 <listitem><para>Saving modifications.</para></listitem>
2269 <listitem><para>Storing patches outside of the kernel source repository (bulk export).</para></listitem>
2270 <listitem><para>Working with incremental changes.</para></listitem>
2271 <listitem><para>Extracting commited changes from a working directory (exporting internally through
2272 patches.</para></listitem>
2273 <listitem><para>Pushing commited changes.</para></listitem>
2274 <listitem><para>Exporting for external (upstream) submission.</para></listitem>
2275 <listitem><para>Exporting for import into another Source Control Manager (SCM).</para></listitem>
2276 <listitem><para>Working with the Yocto Project kernel in another SCM.</para>
2277 <itemizedlist>
2278 <listitem><para>Exporting the delivered kernel to an SCM.</para></listitem>
2279 <listitem><para>Importing changed for the build.</para></listitem>
2280 </itemizedlist></listitem>
2281 <listitem><para>Migrating templates from version 2.0.</para></listitem>
2282 <listitem><para>Creating a new Board Support Package (BSP).</para>
2283 <itemizedlist>
2284 <listitem><para>Creating from scratch.</para></listitem>
2285 <listitem><para>Cloning.</para></listitem>
2286 </itemizedlist></listitem>
2287 <listitem><para>BSP bootstrapping.</para></listitem>
2288 <listitem><para>Applying patches to the kernel through a template.</para></listitem>
2289 <listitem><para>Applying patches to the kernel without using a template.</para></listitem>
2290 <listitem><para>Updating patches and configurations for a BSP.</para></listitem>
2291 <listitem><para>Using guilt to add and export patches.</para></listitem>
2292 <listitem><para>Using scc.</para></listitem>
2293 <listitem><para>Building a 'dirty' image.</para></listitem>
2294 <listitem><para>Temporarily using a different base kernel.</para></listitem>
2295 <listitem><para>Creating a custom kernel layer.</para></listitem>
2296 <listitem><para>Creating the git repository of the transition kernel.</para></listitem>
2297 </itemizedlist> -->
2298
2299
2300</section>
2301
2302</chapter>
2303<!--
2304vim: expandtab tw=80 ts=4
2305-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-09 16:53:05 +0000