summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
Diffstat
-rw-r--r--documentation/kernel-manual/yocto-project-kernal-manual-customization.xsl8
-rw-r--r--documentation/kernel-manual/yocto-project-kernal-manual.xml2175
2 files changed, 0 insertions, 2183 deletions
diff --git a/documentation/kernel-manual/yocto-project-kernal-manual-customization.xsl b/documentation/kernel-manual/yocto-project-kernal-manual-customization.xsl
deleted file mode 100644
index 8e6ea34dd4..0000000000
--- a/documentation/kernel-manual/yocto-project-kernal-manual-customization.xsl
+++ /dev/null
@@ -1,8 +0,0 @@
1<?xml version='1.0'?>
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3
4 <xsl:import href="http://docbook.sourceforge.net/release/xsl/current/xhtml/docbook.xsl" />
5
6 <xsl:param name="generate.toc" select="'article nop'"></xsl:param>
7
8</xsl:stylesheet>
diff --git a/documentation/kernel-manual/yocto-project-kernal-manual.xml b/documentation/kernel-manual/yocto-project-kernal-manual.xml
deleted file mode 100644
index b1693500fc..0000000000
--- a/documentation/kernel-manual/yocto-project-kernal-manual.xml
+++ /dev/null
@@ -1,2175 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<article id='intro'>
5 <imagedata fileref="figures/yocto-project-transp.png" width="6in" depth="1in" align="right" scale="25" />
6
7<section id='fake-title'>
8 <title>Yocto Project Kernel Architecture and Use Manual</title>
9</section>
10
11<section id='introduction'>
12 <title>Introduction</title>
13 <para>
14 Yocto Project presents the kernel as a fully patched, history-clean git
15 repository.
16 The git tree represents the selected features, board support,
17 and configurations extensively tested by Yocto Project.
18 The Yocto Project kernel allows the end user to leverage community
19 best practices to seamlessly manage the development, build and debug cycles.
20 </para>
21 <para>
22 This manual describes the Yocto Project kernel by providing information
23 on its history, organization, benefits, and use.
24 The manual consists of two sections:
25 <itemizedlist>
26 <listitem><para>Concepts - Describes concepts behind the kernel.
27 You will understand how the kernel is organized and why it is organized in
28 the way it is. You will understand the benefits of the kernel's organization
29 and the mechanisms used to work with the kernel and how to apply it in your
30 design process.</para></listitem>
31 <listitem><para>Using the Kernel - Describes best practices and "how-to" information
32 that lets you put the kernel to practical use. Some examples are "How to Build a
33 Project Specific Tree", "How to Examine Changes in a Branch", and "Saving Kernel
34 Modifications."</para></listitem>
35 </itemizedlist>
36 </para>
37 <para>
38 For more information on the kernel, see the following links:
39 <itemizedlist>
40 <listitem><para><ulink url='http://ldn.linuxfoundation.org/book/1-a-guide-kernel-development-process'></ulink></para></listitem>
41 <listitem><para><ulink url='http://userweb.kernel.org/~akpm/stuff/tpp.txt'></ulink></para></listitem>
42 <listitem><para><ulink url='http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=Documentation/HOWTO;hb=HEAD'></ulink></para></listitem>
43 </itemizedlist>
44 <para>
45 You can find more information on Yocto Project by visiting the website at
46 <ulink url='http://www.yoctoproject.org'></ulink>.
47 </para>
48 </para>
49</section>
50
51<section id='concepts'>
52 <title>Concepts</title>
53 <para>
54 This section provides conceptual information about the Yocto Project kernel:
55 <itemizedlist>
56 <listitem><para>Kernel Goals</para></listitem>
57 <listitem><para>Yocto Project Kernel Development and Maintenance Overview</para></listitem>
58 <listitem><para>Kernel Architecture</para></listitem>
59 <listitem><para>Kernel Tools</para></listitem>
60 </itemizedlist>
61 </para>
62 <section id='kernel-goals'>
63 <title>Kernel Goals</title>
64 <para>
65 The complexity of embedded kernel design has increased dramatically.
66 Whether it is managing multiple implementations of a particular feature or tuning and
67 optimizing board specific features, flexibility and maintainability are key concerns.
68 The Yocto Project Linux kernel is presented with the embedded
69 developer's needs in mind and has evolved to assist in these key concerns.
70 For example, prior methods such as applying hundreds of patches to an extracted
71 tarball have been replaced with proven techniques that allow easy inspection,
72 bisection and analysis of changes.
73 Application of these techniques also creates a platform for performing integration and
74 collaboration with the thousands of upstream development projects.
75 </para>
76 <para>
77 With all these considerations in mind, the Yocto Project kernel and development team
78 strives to attain these goals:
79 <itemizedlist>
80 <listitem><para>Allow the end user to leverage community best practices to seamlessly
81 manage the development, build and debug cycles.</para></listitem>
82 <listitem><para>Create a platform for performing integration and collaboration with the
83 thousands of upstream development projects that exist.</para></listitem>
84 <listitem><para>Provide mechanisms that support many different work flows, front-ends and
85 management techniques.</para></listitem>
86 <listitem><para>Deliver the most up-to-date kernel possible while still ensuring that
87 the baseline kernel is the the most stable official release.</para></listitem>
88 <listitem><para>Include major technological features as part of Yocto Project's up-rev
89 strategy.</para></listitem>
90 <listitem><para>Present a git tree, that just like the upstream kernel.org tree, has a
91 clear and continuous history.</para></listitem>
92 <listitem><para>Deliver a key set of supported kernel types, where each type is tailored
93 to a specific use case (i.g. networking, consumer, devices, and so forth).</para></listitem>
94 <listitem><para>Employ a git branching strategy that from a customer's point of view
95 results in a linear path from the baseline kernel.org, through a select group of features and
96 ends with their BSP-specific commits.</para></listitem>
97 </itemizedlist>
98 </para>
99 </section>
100
101 <section id='kernel-big-picture'>
102 <title>Yocto Project Kernel Development and Maintenance Overview</title>
103 <para>
104 Yocto Project kernel, like other kernels, is based off the Linux kernel release
105 from <ulink url='http://www.kernel.org'></ulink>.
106 At the beginning of our major development cycle, we choose our Yocto Project kernel
107 based on factors like release timing, the anticipated release timing of "final" (i.e. non "rc")
108 upstream kernel.org versions, and Yocto Project feature requirements.
109 Typically this will be a kernel that is in the
110 final stages of development by the community (i.e. still in the release
111 candidate or "rc" phase) and not yet a final release.
112 But by being in the final stages of external development, we know that the
113 kernel.org final release will clearly land within the early stages of
114 the Yocto Project development window.
115 </para>
116 <para>
117 This balance allows us to deliver the most up-to-date kernel
118 as possible, while still ensuring that we have a stable official release as
119 our baseline kernel version.
120 </para>
121 <para>
122 The following figure represents the overall place the Yocto Project kernel fills.
123 </para>
124 <para>
125 <imagedata fileref="figures/kernel-big-picture.png" width="6in" depth="4in" align="center" scale="100" />
126 </para>
127 <para>
128 In the figure the ultimate source for the Yocto Project kernel is a released kernel
129 from kernel.org.
130 In addition to a foundational kernel from kernel.org the commercially released
131 Yocto Project kernel contains a mix of important new mainline
132 developments, non-mainline developments, Board Support Package (BSP) developments,
133 and custom features.
134 These additions result in a commercially released Yocto Project kernel that caters
135 to specific embedded designer needs for targeted hardware.
136 </para>
137 <para>
138 Once a Yocto Project kernel is officially released the Yocto Project team goes into
139 their next development cycle, or "uprev" cycle.
140 It is important to note that the most sustainable and stable way
141 to include feature development upstream is through a kernel uprev process.
142 Back-porting of hundreds of individual fixes and minor features from various
143 kernel versions is not sustainable and can easily compromise quality.
144 During the uprev cycle, the Yocto Project team uses an ongoing analysis of
145 kernel development, BSP support, and release timing to select the best
146 possible kernel.org version.
147 The team continually monitors community kernel
148 development to look for significant features of interest.
149 The illustration depicts this by showing the team looking back to kernel.org for new features,
150 BSP features, and significant bug fixes.
151 The team does consider back-porting large features if they have a significant advantage.
152 User or community demand can also trigger a back-port or creation of new
153 functionality in the Yocto Project baseline kernel during the uprev cycle.
154 </para>
155 <para>
156 Generally speaking, every new kernel both adds features and introduces new bugs.
157 These consequences are the basic properties of upstream kernel development and are
158 managed by the Yocto Project team's kernel strategy.
159 It is the Yocto Project team's policy to not back-port minor features to the released kernel.
160 They only consider back-porting significant technological jumps - and, that is done
161 after a complete gap analysis.
162 The reason for this policy is that simply back-porting any small to medium sized change
163 from an evolving kernel can easily create mismatches, incompatibilities and very
164 subtle errors.
165 </para>
166 <para>
167 These policies result in both a stable and a cutting
168 edge kernel that mixes forward ports of existing features and significant and critical
169 new functionality.
170 Forward porting functionality in the Yocto Project kernel can be thought of as a
171 "micro uprev."
172 The many “micro uprevs” produce a kernel version with a mix of
173 important new mainline, non-mainline, BSP developments and feature integrations.
174 This kernel gives insight into new features and allows focused
175 amounts of testing to be done on the kernel, which prevents
176 surprises when selecting the next major uprev.
177 The quality of these cutting edge kernels is evolving and the kernels are used in very special
178 cases for BSP and feature development.
179 </para>
180 </section>
181
182 <section id='kernel-architecture'>
183 <title>Kernel Architecture</title>
184 <para>
185 This section describes the architecture of the Yocto Project kernel and provides information
186 on the mechanisms used to achieve that architecture.
187 </para>
188
189 <section id='architecture-overview'>
190 <title>Overview</title>
191 <para>
192 As mentioned earlier, a key goal of Yocto Project is to present the developer with
193 a kernel that has a clear and continuous history that is visible to the user.
194 The architecture and mechanisms used achieve that goal in a manner similar to the
195 upstream kernel.org.
196
197 </para>
198 <para>
199 You can think of the Yocto Project kernel as consisting of a baseline kernel with
200 added features logically structured on top of the baseline.
201 The features are tagged and organized by way of a branching strategy implemented by the
202 source code manager (SCM) git.
203 The result is that the user has the ability to see the added features and
204 the commits that make up those features.
205 In addition to being able to see added features, the user can also view the history of what
206 made up the baseline kernel as well.
207 </para>
208 <para>
209 The following illustration shows the conceptual Yocto Project kernel.
210 </para>
211 <para>
212 <imagedata fileref="figures/kernel-architecture-overview.png" width="6in" depth="4in" align="center" scale="100" />
213 </para>
214 <para>
215 In the illustration, the "kernel.org Branch Point" marks the specific spot (or release) from
216 which the Yocto Project kernel is created. From this point "up" in the tree features and
217 differences are organized and tagged.
218 </para>
219 <para>
220 The "Yocto Project Baseline Kernel" contains functionality that is common to every kernel
221 type and BSP that is organized further up the tree. Placing these common features in the
222 tree this way means features don't have to be duplicated along individual branches of the
223 structure.
224 </para>
225 <para>
226 From the Yocto Project Baseline Kernel branch points represent specific functionality
227 for individual BSPs as well as real-time kernels.
228 The illustration represents this through three BSP-specific branches and a real-time
229 kernel branch.
230 Each branch represents some unique functionality for the BSP or a real-time kernel.
231 </para>
232 <para>
233 The real-time kernel branch has common features for all real-time kernels and contains
234 more branches for individual BSP-specific real-time kernels.
235 The illustration shows three branches as an example.
236 Each branch points the way to specific, unique features for a respective real-time
237 kernel as they apply to a given BSP.
238 </para>
239 <para>
240 The resulting tree structure presents a clear path of markers (or branches) to the user
241 that for all practical purposes is the kernel needed for any given set of requirements.
242 </para>
243 </section>
244
245 <section id='branching-and-workflow'>
246 <title>Branching Strategy and Workflow</title>
247 <para>
248 The Yocto Project team creates kernel branches at points where functionality is
249 no longer shared and thus, needs to be isolated.
250 For example, board-specific incompatibilities would require different functionality
251 and would require a branch to separate the features.
252 Likewise, for specific kernel features the same branching strategy is used.
253 This branching strategy results in a tree that has features organized to be specific
254 for particular functionality, single kernel types, or a subset of kernel types.
255 This strategy results in not having to store the same feature twice internally in the
256 tree.
257 Rather we store the unique differences required to apply the feature onto the kernel type
258 in question.
259 </para>
260 <para>
261 BSP-specific code additions are handled in a similar manner to kernel-specific additions.
262 Some BSPs only make sense given certain kernel types.
263 So, for these types, we create branches off the end of that kernel type for all
264 of the BSPs that are supported on that kernel type.
265 From the perspective of the tools that create the BSP branch, the BSP is really no
266 different than a feature.
267 Consequently, the same branching strategy applies to BSPs as it does to features.
268 So again, rather than store the BSP twice, only the unique differences for the BSP across
269 the supported multiple kernels are uniquely stored.
270 </para>
271 <para>
272 While this strategy results in a tree with a significant number of branches, it is
273 important to realize that from the customer's point of view, there is a linear
274 path that travels from the baseline kernel.org, through a select group of features and
275 ends with their BSP-specific commits.
276 In other words, the divisions of the kernel are transparent and are not relevant
277 to the developer on a day-to-day basis.
278 From the customer's perspective, this is the "master" branch.
279 They do not need not be aware of the existence of any other branches at all.
280 Of course there is value in the existence of these branches
281 in the tree, should a person decide to explore them.
282 For example, a comparison between two BSPs at either the commit level or at the line-by-line
283 code diff level is now a trivial operation.
284 </para>
285 <para>
286 Working with the kernel as a structured tree follows recognized community best practices.
287 In particular, the kernel as shipped with the product should be
288 considered an 'upstream source' and viewed as a series of
289 historical and documented modifications (commits).
290 These modifications represent the development and stabilization done
291 by the Yocto Project kernel development team.
292 </para>
293 <para>
294 Because commits only change at significant release points in the product life cycle,
295 developers can work on a branch created
296 from the last relevant commit in the shipped Yocto Project kernel.
297 As mentioned previously, the structure is transparent to the user
298 because the kernel tree is left in this state after cloning and building the kernel.
299 </para>
300 </section>
301
302 <section id='source-code-manager-git'>
303 <title>Source Code Manager - git</title>
304 <para>
305 The Source Code Manager (SCM) is git and it is the obvious mechanism for meeting the
306 previously mentioned goals.
307 Not only is it the SCM for kernel.org but git continues to grow in popularity and
308 supports many different work flows, front-ends and management techniques.
309 </para>
310 <note><para>
311 It should be noted that you can use as much, or as little, of what git has to offer
312 as is appropriate to your project.
313 </para></note>
314 </section>
315 </section>
316
317 <section id='kernel-tools'>
318 <title>Kernel Tools</title>
319 <para>
320Since most standard workflows involve moving forward with an existing tree by
321continuing to add and alter the underlying baseline, the tools that manage
322Yocto Project's kernel construction are largely hidden from the developer to
323present a simplified view of the kernel for ease of use.
324</para>
325<para>
326The fundamental properties of the tools that manage and construct the
327kernel are:
328<itemizedlist>
329 <listitem><para>the ability to group patches into named, reusable features</para></listitem>
330 <listitem><para>to allow top down control of included features</para></listitem>
331 <listitem><para>the binding of kernel configuration to kernel patches/features</para></listitem>
332 <listitem><para>the presentation of a seamless git repository that blends Yocto Project value with the kernel.org history and development</para></listitem>
333</itemizedlist>
334</para>
335<para>
336The tools that construct a kernel tree will be discussed later in this
337document. The following tools form the foundation of the Yocto Project
338kernel toolkit:
339<itemizedlist>
340 <listitem><para>git : distributed revision control system created by Linus Torvalds</para></listitem>
341 <listitem><para>guilt: quilt on top of git</para></listitem>
342 <listitem><para>*cfg : kernel configuration management and classification</para></listitem>
343 <listitem><para>kgit*: Yocto Project kernel tree creation and management tools</para></listitem>
344 <listitem><para>scc : series &amp; configuration compiler</para></listitem>
345</itemizedlist>
346</para>
347 </section>
348</section>
349
350
351
352
353<!-- <section id='concepts2'>
354 <title>Kernel Concepts</title>
355 <itemizedlist>
356 <listitem><para>What tools and commands are used with the kernel.</para></listitem>
357 <listitem><para>Source Control Manager (SCM).</para></listitem>
358 <listitem><para>What are some workflows that you can apply using the kernel.</para></listitem>
359 </itemizedlist>
360</section> -->
361
362<section id='actions'>
363 <title>How to get things accomplished with the kernel</title>
364 <para>
365 This section describes how to accomplish tasks involving the kernel's tree structure.
366 The information covers the following:
367 <itemizedlist>
368 <listitem><para>Tree construction</para></listitem>
369 <listitem><para>Build strategies</para></listitem>
370 <listitem><para>Series &amp; Configuration Compiler</para></listitem>
371 <listitem><para>kgit</para></listitem>
372 <listitem><para>Workflow examples</para></listitem>
373 <listitem><para>Source Code Manager (SCM)</para></listitem>
374 <listitem><para>Board Support Package (BSP) template migration</para></listitem>
375 <listitem><para>BSP creation</para></listitem>
376 <listitem><para>Patching</para></listitem>
377 <listitem><para>Updating BSP patches and configuration</para></listitem>
378 <listitem><para>guilt</para></listitem>
379 <listitem><para>scc file example</para></listitem>
380 <listitem><para>"dirty" string</para></listitem>
381 <listitem><para>Transition kernel layer</para></listitem>
382 </itemizedlist>
383 </para>
384
385 <section id='tree-construction'>
386 <title>Tree Construction</title>
387 <para>
388The Yocto Project kernel repository, as shipped with the product, is created by
389compiling and executing the set of feature descriptions for every BSP/feature
390in the product. Those feature descriptions list all necessary patches,
391configuration, branching, tagging and feature divisions found in the kernel.
392</para>
393<para>
394The files used to describe all the valid features and BSPs in the Yocto Project
395kernel can be found in any clone of the kernel git tree. The directory
396wrs/cfg/kernel-cache/ is a snapshot of all the kernel configuration and
397feature descriptions (.scc) that were used to build the kernel repository.
398It should however be noted, that browsing the snapshot of feature
399descriptions and patches is not an effective way to determine what is in a
400particular kernel branch. Using git directly to get insight into the changes
401in a branch is more efficient and a more flexible way to inspect changes to
402the kernel. Examples of using git to inspect kernel commits are in the
403following sections.
404</para>
405<para>
406As a reminder, it is envisioned that a ground up reconstruction of the
407complete kernel tree is an action only taken by Yocto Project staff during an
408active development cycle. When an end user creates a project, it takes
409advantage of this complete tree in order to efficiently place a git tree
410within their project.
411</para>
412<para>
413The general flow of the project specific kernel tree construction is as follows:
414<orderedlist>
415 <listitem><para>a top level kernel feature is passed to the kernel build subsystem,
416 normally this is a BSP for a particular kernel type.</para></listitem>
417
418 <listitem><para>the file that describes the top level feature is located by searching
419 system directories:</para>
420
421 <itemizedlist>
422 <listitem><para>the kernel-cache under linux/wrs/cfg/kernel-cache</para></listitem>
423 <listitem><para>kernel-*-cache directories in layers</para></listitem>
424 <listitem><para>configured and default templates</para></listitem>
425 </itemizedlist>
426
427 <para>In a typical build a feature description of the format:
428 &lt;bsp name&gt;-&lt;kernel type&gt;.scc is the target of the search.
429 </para></listitem>
430
431 <listitem><para>once located, the feature description is compiled into a simple script
432 of actions, or an existing equivalent script which was part of the
433 shipped kernel is located.</para></listitem>
434
435 <listitem><para>extra features are appended to the top level feature description. Extra
436 features can come from the command line, the configure script or
437 templates.</para></listitem>
438
439 <listitem><para>each extra feature is located, compiled and appended to the script from
440 step #3</para></listitem>
441
442 <listitem><para>the script is executed, and a meta-series is produced. The meta-series
443 is a description of all the branches, tags, patches and configuration that
444 need to be applied to the base git repository to completely create the
445 "bsp_name-kernel_type".</para></listitem>
446
447 <listitem><para>the base repository (normally kernel.org) is cloned, and the actions
448 listed in the meta-series are applied to the tree.</para></listitem>
449
450 <listitem><para>the git repository is left with the desired branch checked out and any
451 required branching, patching and tagging has been performed.</para></listitem>
452</orderedlist>
453</para>
454
455<para>
456The tree is now ready for configuration and compilation. Those two topics will
457be covered below.
458</para>
459
460<note><para>The end user generated meta-series adds to the kernel as shipped with
461 the Yocto Project release. Any add-ons and configuration data are applied
462 to the end of an existing branch. The full repository generation that
463 is found in the linux-2.6-windriver.git is the combination of all
464 supported boards and configurations.
465</para></note>
466
467<para>
468This technique is flexible and allows the seamless blending of an immutable
469history with additional deployment specific patches. Any additions to the
470kernel become an integrated part of the branches.
471</para>
472
473<note><para>It is key that feature descriptions indicate if any branches are
474 required, since the build system cannot automatically decide where a
475 BSP should branch or if that branch point needs a name with
476 significance. There is a single restriction enforced by the compilation
477 phase:
478 </para>
479 <para>A BSP must create a branch of the format &lt;bsp name&gt;-&lt;kernel type&gt;.</para>
480
481 <para>This means that all merged/support BSPs must indicate where to start
482 its branch from, with the right name, in its .scc files. The scc
483 section describes the available branching commands in more detail.
484 </para>
485</note>
486
487<para>
488A summary of end user tree construction activities follow:
489<itemizedlist>
490 <listitem><para>compile and link a full top-down kernel description from feature descriptions</para></listitem>
491 <listitem><para>execute the complete description to generate a meta-series</para></listitem>
492 <listitem><para>interpret the meta-series to create a customized git repository for the
493 board</para></listitem>
494 <listitem><para>migrate configuration fragments and configure the kernel</para></listitem>
495 <listitem><para>checkout the BSP branch and build</para></listitem>
496</itemizedlist>
497</para>
498 </section>
499
500 <section id='build-strategy'>
501 <title>Build Strategy</title>
502<para>
503There are some prerequisites that must be met before starting the compilation
504phase of the kernel build system:
505</para>
506<itemizedlist>
507 <listitem><para>There must be a kernel git repository indicated in the SRC_URI.</para></listitem>
508 <listitem><para>There must be a branch &lt;bsp name&gt;-&lt;kernel type&gt;.</para></listitem>
509</itemizedlist>
510
511<para>
512These are typically met by running tree construction/patching phase of the
513build system, but can be achieved by other means. Examples of alternate work
514flows such as bootstrapping a BSP are provided below.
515</para>
516<para>
517Before building a kernel it is configured by processing all of the
518configuration "fragments" specified by the scc feature descriptions. As the
519features are compiled, associated kernel configuration fragments are noted
520and recorded in the meta-series in their compilation order. The
521fragments are migrated, pre-processed and passed to the Linux Kernel
522Configuration subsystem (lkc) as raw input in the form of a .config file.
523The lkc uses its own internal dependency constraints to do the final
524processing of that information and generates the final .config that will
525be used during compilation.
526</para>
527<para>
528Kernel compilation is started, using the board's architecture and other
529relevant values from the board template, and a kernel image is produced.
530</para>
531<para>
532The other thing that you will first see once you configure a kernel is that
533it will generate a build tree that is separate from your git source tree.
534This build dir will be called "linux-&lt;BSPname&gt;-&lt;kerntype&gt;-build" where
535kerntype is one of standard, cg``
536e, etc. This functionality is done by making
537use of the existing support that is within the kernel.org tree by default.
538</para>
539<para>
540What this means, is that all the generated files (that includes the final
541".config" itself, all ".o" and ".a" etc) are now in this directory. Since
542the git source tree can contain any number of BSPs, all on their own branch,
543you now can easily switch between builds of BSPs as well, since each one also
544has their own separate build directory.
545</para>
546 </section>
547
548 <section id='scc'>
549 <title>Series &amp; Configuration Compiler (SCC)</title>
550<para>
551In early versions of the product, kernel patches were simply listed in a flat
552file called "patches.list", and then quilt was added as a tool to help
553traverse this list, which in quilt terms was called a "series" file.
554</para>
555<para>
556Before the 2.0 release, it was already apparent that a static series file was
557too inflexible, and that the series file had to become more dynamic and rely
558on certain state (like kernel type) in order to determine whether a patch was
559to be used or not. The 2.0 release already made use of some stateful
560construction of series files, but since the delivery mechanism was unchanged
561(tar + patches + series files), most people were not aware of anything really
562different. The 3.0 release continues with this stateful construction of
563series files, but since the delivery mechanism is changed (git + branches) it
564now is more apparent to people.
565</para>
566<para>
567As was previously mentioned, scc is a "series and configuration
568compiler". Its role is to combine feature descriptions into a format that can
569be used to generate a meta-series. A meta series contains all the required
570information to construct a complete set of branches that are required to
571build a desired board and feature set. The meta series is interpreted by the
572kgit tools to create a git repository that could be built.
573</para>
574<para>
575To illustrate how scc works, a feature description must first be understood.
576A feature description is simply a small bash shell script that is executed by
577scc in a controlled environment. Each feature description describes a set of
578operations that add patches, modify existing patches or configure the
579kernel. It is key that feature descriptions can include other features, and
580hence allow the division of patches and configuration into named, reusable
581containers.
582</para>
583<para>
584Each feature description can use any of the following valid scc commands:
585<itemizedlist>
586 <listitem><para>shell constructs: bash conditionals and other utilities can be used in a feature
587 description. During compilation, the working directory is the feature
588 description itself, so any command that is "raw shell" and not from the
589 list of supported commands, can not directly modify a git repository.</para></listitem>
590
591 <listitem><para>patch &lt;relative path&gt;/&lt;patch name&gt;: outputs a patch to be included in a feature's patch set. Only the name of
592 the patch is supplied, the path is calculated from the currently set
593 patch directory, which is normally the feature directory itself.</para></listitem>
594
595 <listitem><para>patch_trigger &gt;condition&lt; &gt;action&lt; &lt;tgt&gt;: indicate that a trigger should be set to perform an action on a
596 patch.</para>
597
598<para>The conditions can be:
599
600 <itemizedlist>
601 <listitem><para>arch:&lt;comma separated arch list or "all"&gt;</para></listitem>
602 <listitem><para>plat:&lt;comma separated platform list or "all"&gt;</para></listitem>
603 </itemizedlist></para>
604<para>The action can be:
605 <itemizedlist>
606 <listitem><para>exclude: This is used in exceptional situations where a patch
607 cannot be applied for certain reasons (arch or platform).
608 When the trigger is satisfied the patch will be removed from
609 the patch list.</para></listitem>
610 <listitem><para>include: This is used to include a patch only for a specific trigger.
611 Like exclude, this should only be used when necessary.
612 It takes 1 argument, the patch to include.</para></listitem>
613 </itemizedlist></para></listitem>
614
615 <listitem><para>include &lt;feature name&gt; [after &lt;feature&gt;]: includes a feature for processing. The feature is "expanded" at the
616 position of the include directive. This means that any patches,
617 configuration or sub-includes of the feature will appear in the final
618 series before the commands that follow the include.</para>
619 <para>
620 include searches the include directories for a matching feature name,
621 include directories are passed to scc by the caller using -I &lt;path&gt; and
622 is transparent to the feature script. This means that &lt;feature name&gt; must
623 be relative to one of the search paths. For example, if
624 /opt/kernel-cache/feat/sched.scc is to be included and scc is invoked
625 with -I /opt/kernel-cache, then a feature would issue "include
626 feat/sched.scc" to include the feature.
627</para>
628<para>
629 The optional "after" directive allows a feature to modify the existing
630 order of includes and insert a feature after the named feature is
631 processed. Note: the "include foo after bar" must be issued before "bar"
632 is processed, so is normally only used by a new top level feature to
633 modify the order of features in something it is including.</para></listitem>
634
635 <listitem><para>exclude &lt;feature name&gt;: Indicates that a particular feature should *not* be included even if an
636 'include' directive is found. The exclude must be issued before the
637 include is processed, so is normally only used by a new top level feature
638 to modify the order of features in something it is including.</para></listitem>
639
640 <listitem><para>git &lt;command&gt;: Issues any git command during tree construction. Note: this command is
641 not validated/sanitized so care must be taken to not damage the
642 tree. This can be used to script branching, tagging, pulls or other git
643 operations.</para></listitem>
644
645 <listitem><para>dir &lt;directory&gt;: changes the working directory for "patch" directives. This can be used to
646 shorten a long sequence of patches by not requiring a common relative
647 directory to be issued each time.</para></listitem>
648
649 <listitem><para>kconf &lt;type&gt; &lt;fragment name&gt;: associates a kernel config frag with the feature.
650 &lt;type&gt; can be
651 "hardware" or "non-hardware" and is used by the kernel configuration
652 subsystem to audit configuration. &lt;fragment name&gt; is the name of a file
653 in the current feature directory that contains a series of kernel
654 configuration options. There is no restriction on the chosen fragment
655 name, although a suffix of ".cfg" is recommended. Multiple fragment
656 specifications are supported.</para></listitem>
657
658 <listitem><para>branch &lt;branch name&gt;: creates a branch in the tree. All subsequent patch commands will be
659 applied to the new branch and changes isolated from the rest of the
660 repository.</para></listitem>
661
662 <listitem><para>scc_leaf &lt;base feature&gt; &lt;branch name&gt;: Performs a combination feature include and branch. This is mainly a
663 convenience directive, but has significance to some build system bindings
664 as a sentinel to indicate that this intends to create a branch that is
665 valid for kernel compilation.</para></listitem>
666
667 <listitem><para>tag &lt;tag name&gt;: Tags the tree. The tag will be applied in processing order, so will
668 be after already applied patches and precede patches yet to be applied.</para></listitem>
669
670 <listitem><para>define &lt;var&gt; &lt;value&gt;: Creates a variable with a particular value that can be used in subsequent
671 feature descriptions.</para></listitem>
672</itemizedlist>
673
674</para>
675 </section>
676
677 <section id='kgit-tools'>
678 <title>kgit Tools</title>
679<para>
680The kgit tools are responsible for constructing and maintaining the Wind
681River kernel repository. These activities include importing, exporting, and
682applying patches as well as sanity checking and branch management. From the
683developers perspective, the kgit tools are hidden and rarely require
684interactive use. But one tool in particular that warrants further description
685is "kgit-meta".
686</para>
687<para>
688kgit-meta is the actual application of feature description(s) to a kernel repo.
689In other words, it is responsible for interpreting the meta series generated
690from a scc compiled script. As a result, kgit-meta is coupled to the set of
691commands permitted in a .scc feature description (listed in the scc section).
692kgit-meta understands both the meta series format and how to use git and
693guilt to modify a base git repository. It processes a meta-series line by
694line, branching, tagging, patching and tracking changes that are made to the
695base git repository.
696</para>
697<para>
698Once kgit-meta has processed a meta-series, it leaves the repository with the
699last branch checked out, and creates the necessary guilt infrastructure to
700inspect the tree, or add to it via using guilt. As was previously mentioned,
701guilt is not required, but is provided as a convenience. Other utilities such
702as quilt, stgit, git or others can also be used to manipulate the git
703repository.
704</para>
705 </section>
706
707 <section id='workflow-examples'>
708 <title>Workflow Examples</title>
709
710 <para>
711As previously noted, the Yocto Project kernel has built in git/guilt
712integration, but these utilities are not the only way to work with the kernel
713repository. Yocto Project has not made changes to git, or other tools that
714invalidate alternate workflows. Additionally, the way the kernel repository
715is constructed uses only core git functionality allowing any number of tools
716or front ends to use the resulting tree.</para>
717<para>
718This section contains several workflow examples.
719</para>
720
721 <section id='change-inspection-kernel-changes-commits'>
722 <title>Change Inspection: Kernel Changes/Commits</title>
723<para>
724A common question when working with a BSP/kernel is: "What changes have been applied to this tree?"
725</para>
726<para>
727In previous Yocto Project releases, there were a collection of directories that
728contained patches to the kernel, those patches could be inspected, grep'd or
729otherwise used to get a general feeling for changes. This sort of patch
730inspection is not an efficient way to determine what has been done to the
731kernel, since there are many optional patches that are selected based on the
732kernel type and feature description, not to mention patches that are actually
733in directories that are not being searched.
734</para>
735<para>
736A more effective way to determine what has changed in the kernel is to use
737git and inspect / search the kernel tree. This is a full view of not only the
738source code modifications, but the reasoning behind the changes.
739</para>
740 <section id='what-changed-in-a-bsp'>
741 <title>What Changed in a BSP?</title>
742<para>
743These examples could continue for some time, since the Yocto Project git
744repository doesn't break existing git functionality and there are nearly
745endless permutations of those commands. Also note that unless a commit range
746is given (&lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;), kernel.org history is blended
747with Yocto Project changes
748</para>
749<literallayout class='monospaced'>
750 # full description of the changes
751 &gt; git whatchanged &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
752 &gt; eg: git whatchanged standard..common_pc-standard
753
754 # summary of the changes
755 &gt; git log &dash;&dash;pretty=oneline &dash;&dash;abbrev-commit &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
756
757 # source code changes (one combined diff)
758 &gt; git diff &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
759 &gt; git show &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
760
761 # dump individual patches per commit
762 &gt; git format-patch -o &lt;dir&gt; &lt;kernel type&gt;..&lt;bsp&gt;-&lt;kernel type&gt;
763
764 # determine the change history of a particular file
765 &gt; git whatchanged &lt;path to file&gt;
766
767 # determine the commits which touch each line in a file
768 &gt; git blame &lt;path to file&gt;
769</literallayout>
770 </section>
771
772 <section id='show-a-particular-feature-or-branch-change'>
773 <title>Show a Particular Feature or Branch Change</title>
774<para>
775Significant features or branches are tagged in the Yocto Project tree to divide
776changes. Remember to first determine (or add) the tag of interest. Note:
777there will be many tags, since each BSP branch is tagged, kernel.org tags and
778feature tags are all present.
779</para>
780<literallayout class='monospaced'>
781 # show the changes tagged by a feature
782 &gt; git show &lt;tag&gt;
783 &gt; eg: git show yaffs2
784
785 # determine which branches contain a feature
786 &gt; git branch &dash;&dash;contains &lt;tag&gt;
787
788 # show the changes in a kernel type
789 &gt; git whatchanged wrs_base..&lt;kernel type&gt;
790 &gt; eg: git whatchanged wrs_base..standard
791</literallayout>
792<para>
793Many other comparisons can be done to isolate BSP changes, such as comparing
794to kernel.org tags (v2.6.27.18, etc), per subsystem comparisons (git
795whatchanged mm) or many other types of checks.
796</para>
797 </section>
798 </section>
799
800 <section id='development-saving-kernel-modifications'>
801 <title>Development: Saving Kernel Modifications</title>
802<para>
803Another common operation is to build a Yocto Project supplied BSP, make some
804changes, rebuild and test. Those local changes often need to be exported,
805shared or otherwise maintained.
806</para>
807<para>
808Since the Yocto Project kernel source tree is backed by git, this activity is
809greatly simplified and is much easier than in previous releases. git tracks
810file modifications, additions and deletions, which allows the developer to
811modify the code and later realize that the changes should be saved, and
812easily determine what was changed. It also provides many tools to commit,
813undo and export those modifications.
814</para>
815<para>
816There are many ways to perform this action, and the technique employed
817depends on the destination for the patches, which could be any of:
818<itemizedlist>
819 <listitem><para>bulk storage</para></listitem>
820 <listitem><para>internal sharing either through patches or using git</para></listitem>
821 <listitem><para>external submission</para></listitem>
822 <listitem><para>export for integration into another SCM</para></listitem>
823</itemizedlist>
824</para>
825<para>
826The destination of the patches also incluences the method of gathering them
827due to issues such as:
828<itemizedlist>
829 <listitem><para>bisectability</para></listitem>
830 <listitem><para>commit headers</para></listitem>
831 <listitem><para>division of subsystems for separate submission / review</para></listitem>
832</itemizedlist>
833</para>
834
835 <section id='bulk-export'>
836 <title>Bulk Export</title>
837<para>
838If patches are simply being stored outside of the kernel source repository,
839either permanently or temporarily, then there are several methods that can be
840used.
841</para>
842<para>
843Note the "bulk" in this discussion, these techniques are not appropriate for
844full integration of upstream submission, since they do not properly divide
845changes or provide an avenue for per-change commit messages. This example
846assumes that changes have not been committed incrementally during development
847and simply must be gathered and exported.
848<literallayout class='monospaced'>
849 # bulk export of ALL modifications without separation or division
850 # of the changes
851
852 &gt; git add .
853 &gt; git commit -s -a -m &gt;commit message&lt;
854 or
855 &gt; git commit -s -a # and interact with $EDITOR
856</literallayout>
857</para>
858<para>
859These operations have captured all the local changes in the project source
860tree in a single git commit, and that commit is also stored in the project's
861source tree.
862</para>
863<para>
864Once exported, those changes can then be restored manually, via a template or
865through integration with the default_kernel. Those topics are covered in
866future sections.
867</para>
868 </section>
869
870 <section id='incremental-planned-sharing'>
871 <title>Incremental/Planned Sharing</title>
872<para>
873Note: unlike the previous "bulk" section, the following examples assume that
874changes have been incrementally committed to the tree during development and
875now are being exported.
876</para>
877<para>
878During development the following commands will be of interest, but for full
879git documentation refer to the git man pages or an online resource such as
880http://github.com
881<literallayout class='monospaced'>
882 # edit a file
883 &gt; vi &gt;path&lt;/file
884 # stage the change
885 &gt; git add &gt;path&lt;/file
886 # commit the change
887 &gt; git commit -s
888 # remove a file
889 &gt; git rm &gt;path&lt;/file
890 # commit the change
891 &gt; git commit -s
892
893 ... etc.
894</literallayout>
895</para>
896<para>
897Distributed development with git is possible by having a universally agreed
898upon unique commit identifier (set by the creator of the commit) mapping to a
899specific changeset with a specific parent. This ID is created for you when
900you create a commit, and will be re-created when you amend/alter or re-apply
901a commit. As an individual in isolation, this is of no interest, but if you
902intend to share your tree with normal git push/pull operations for
903distributed development, you should consider the ramifications of changing a
904commit that you've already shared with others.
905</para>
906<para>
907Assuming that the changes have *not* been pushed upstream, or pulled into
908another repository, both the commit content and commit messages associated
909with development can be update via:
910<literallayout class='monospaced'>
911 &gt; git add &gt;path&lt;/file
912 &gt; git commit &dash;&dash;amend
913 &gt; git rebase or git rebase -i
914</literallayout>
915</para>
916<para>
917Again, assuming that the changes have *not* been pushed upstream, and that
918there are no pending works in progress (use "git status" to check) then
919commits can be reverted (undone) via:
920<literallayout class='monospaced'>
921 # remove the commit, update working tree and remove all
922 # traces of the change
923 &gt; git reset &dash;&dash;hard HEAD^
924 # remove the commit, but leave the files changed and staged for re-commit
925 &gt; git reset &dash;&dash;soft HEAD^
926 # remove the commit, leave file change, but not staged for commit
927 &gt; git reset &dash;&dash;mixed HEAD^
928</literallayout>
929</para>
930<para>
931Branches can be created, changes cherry-picked or any number of git
932operations performed until the commits are in good order for pushing upstream
933or pull requests. After a push or pull, commits are normally considered
934'permanent' and should not be modified, only incrementally changed in new
935commits. This is standard "git" workflow and Yocto Project recommends the
936kernel.org best practices.
937</para>
938<note><para>It is recommend to tag or branch before adding changes to a Yocto Project
939 BSP (or creating a new one), since the branch or tag provides a
940 reference point to facilitate locating and exporting local changes.
941</para></note>
942
943 <section id='export-internally-via-patches'>
944 <title>Export Internally Via Patches</title>
945<para>
946Committed changes can be extracted from a working directory by exporting them
947as patches. Those patches can be used for upstream submission, placed in a
948Yocto Project template for automatic kernel patching or many other common uses.
949
950<literallayout class='monospaced'>
951 # &gt;first commit&gt; can be a tag if one was created before development
952 # began. It can also be the parent branch if a branch was created
953 # before development began.
954
955 &gt; git format-patch -o &lt;dir&gt; &lt;first commit&gt;..&lt;last commit&gt;
956</literallayout>
957</para>
958
959<para>
960 In other words:
961<literallayout class='monospaced'>
962 # identify commits of interest.
963
964 # if the tree was tagged before development
965 &gt; git format-patch -o &lt;save dir&gt; &lt;tag&gt;
966
967 # if no tags are available
968 &gt; git format-patch -o &lt;save dir&gt; HEAD^ # last commit
969 &gt; git format-patch -o &lt;save dir&gt; HEAD^^ # last 2 commits
970 &gt; git whatchanged # identify last commit
971 &gt; git format-patch -o &lt;save dir&gt; &lt;commit id&gt;
972 &gt; git format-patch -o &lt;save dir&gt; &lt;rev-list&gt;
973</literallayout>
974</para>
975
976<para>
977The result is a directory with sequentially numbered patches, that when
978applied to a repository using "git am", will reproduce the original commit
979and all related information (author, date, commit log, etc) will be
980preserved. Note that new commit IDs will be generated upon reapplication,
981reflecting that the commit is now applied to an underlying commit with a
982different ID.
983</para>
984<para>
985See the "template patching" example for how to use the patches to
986automatically apply to a new kernel build.
987</para>
988 </section>
989
990 <section id='export-internally-via-git'>
991 <title>Export Internally Via git</title>
992<para>
993Committed changes can also be exported from a working directory by pushing
994(or by making a pull request) the changes into a master repository. Those
995same change can then be pulled into a new kernel build at a later time using this command form:
996<literallayout class='monospaced'>
997 git push ssh://&lt;master server&gt;/&lt;path to repo&gt; &lt;local branch&gt;:&lt;remote branch&gt;
998</literallayout>
999For example:
1000<literallayout class='monospaced'>
1001 &gt; push ssh://openlinux.windriver.com/pub/git/kernel-2.6.27 common_pc-standard:common_pc-standard
1002</literallayout>
1003A pull request entails using "git request-pull" to compose an email to the
1004maintainer requesting that a branch be pulled into the master repository, see
1005http://github.com/guides/pull-requests for an example.
1006</para>
1007<para>
1008Other commands such as 'git stash' or branching can also be used to save
1009changes, but are not covered in this document.
1010</para>
1011<para>
1012See the section "importing from another SCM" for how a git push to the
1013default_kernel, can be used to automatically update the builds of all users
1014of a central git repository.
1015</para>
1016 </section>
1017 </section>
1018
1019 <section id='export-for-external-upstream-submission'>
1020 <title>Export for External (Upstream) Submission</title>
1021<para>
1022If patches are to be sent for external submission, they can be done via a
1023pull request if the patch series is large or the maintainer prefers to pull
1024changes. But commonly, patches are sent as email series for easy review and
1025integration.
1026</para>
1027<note><para>
1028Before sending patches for review ensure that you understand the
1029standard of the community in question and follow their best practices. For
1030example, kernel patches should follow standards such as:
1031<itemizedlist>
1032 <listitem><para><ulink url='http://userweb.kernel.org/~akpm/stuff/tpp.txt'></ulink></para></listitem>
1033 <listitem><para><ulink url='http://linux.yyz.us/patch-format.html'></ulink></para></listitem>
1034 <listitem><para>Documentation/SubmittingPatches (in any linux kernel source tree)</para></listitem>
1035</itemizedlist>
1036</para></note>
1037<para>
1038The messages used to commit changes are a large part of these standards, so
1039ensure that the headers for each commit have the required information. If the
1040initial commits were not properly documented or don't meet those standards
1041rebasing via git rebase -i offer an opportunity to manipulate the commits and
1042get them into the required format. Other techniques such as branching and
1043cherry picking commits are also viable options.
1044</para>
1045<para>
1046Once complete, patches are sent via email to the maintainer(s) or lists that
1047review and integrate changes. "git send-email" is commonly used to ensure
1048that patches are properly formatted for easy application and avoid mailer
1049induced patch damage.
1050</para>
1051<para>
1052An example of dumping patches for external submission follows:
1053<literallayout class='monospaced'>
1054 # dump the last 4 commits
1055 &gt; git format-patch &dash;&dash;thread -n -o ~/rr/ HEAD^^^^
1056 &gt; git send-email &dash;&dash;compose &dash;&dash;subject '[RFC 0/N] &lt;patch series summary&gt;' \
1057 &dash;&dash;to foo@yoctoproject.org &dash;&dash;to bar@yoctoproject.org \
1058 &dash;&dash;cc list@yoctoproject.org ~/rr
1059 # the editor is invoked for the 0/N patch, and when complete the entire
1060 # series is sent via email for review
1061</literallayout>
1062</para>
1063 </section>
1064
1065 <section id='export-for-import-into-other-scm'>
1066 <title>Export for Import into Other SCM</title>
1067<para>
1068Using any one of the previously discussed techniques, commits can be exported
1069as patches for import into another SCM. Note however, that if those patches
1070are manually applied to a secondary tree and then that secondary tree is
1071checked into the SCM, then it often results in lost information (like commit
1072logs) and so it is not recommended.
1073</para>
1074<para>
1075Many SCMs can directly import git commits, or can translate git patches to
1076not lose information. Those facilities are SCM dependent and should be used
1077whenever possible.
1078</para>
1079 </section>
1080 </section>
1081
1082 <section id='scm-working-with-the-yocto-project-kernel-in-another-scm'>
1083 <title>SCM: Working with the Yocto Project Kernel in Another SCM</title>
1084<para>
1085This is not the same as the exporting of patches to another SCM, but instead
1086is concerned with kernel development that is done completely in another
1087environment, but built with the Yocto Project build system. In this scenario two
1088things must happen:
1089<itemizedlist>
1090 <listitem><para>The delivered Yocto Project kernel must be exported into the second
1091 SCM.</para></listitem>
1092 <listitem><para>Development must be exported from that secondary SCM into a
1093 format that can be used by the Yocto Project build system.</para></listitem>
1094</itemizedlist>
1095</para>
1096 <section id='exporting-delivered-kernel-to-scm'>
1097 <title>Exporting Delivered Kernel to SCM</title>
1098<para>
1099Depending on the SCM it may be possible to export the entire Yocto Project
1100kernel git repository, branches and all, into a new environment. This is the
1101preferred method, since it has the most flexibility and potential to maintain
1102the meta data associated with each commit.
1103</para>
1104<para>
1105When a direct import mechanism is not available, it is still possible to
1106export a branch (or series of branches) and check them into a new
1107repository.
1108</para>
1109<para>
1110The following commands illustrate some of the steps that could be used to
1111import the common_pc-standard kernel into a secondary SCM
1112<literallayout class='monospaced'>
1113 &gt; git checkout common_pc-standard
1114 &gt; cd .. ; echo linux/.git &gt; .cvsignore
1115 &gt; cvs import -m "initial import" linux MY_COMPANY start
1116</literallayout>
1117The CVS repo could now be relocated and used in a centralized manner.
1118</para>
1119<para>
1120The following commands illustrate how two BSPs could be condensed and merged
1121into a second SCM:
1122<literallayout class='monospaced'>
1123 &gt; git checkout common_pc-standard
1124 &gt; git merge cav_ebt5800-standard
1125 # resolve any conflicts and commit them
1126 &gt; cd .. ; echo linux/.git &gt; .cvsignore
1127 &gt; cvs import -m "initial import" linux MY_COMPANY start
1128</literallayout>
1129</para>
1130 </section>
1131
1132 <section id='importing-changes-for-build'>
1133 <title>Importing Changes for Build</title>
1134<para>
1135Once development has reached a suitable point in the second development
1136environment, changes can either be exported as patches or imported into git
1137directly (if a conversion/import mechanism is available for the SCM).
1138</para>
1139If changes are exported as patches, they can be placed in a template and
1140automatically applied to the kernel during patching. See the template patch
1141example for details.
1142<para>
1143</para>
1144If changes are imported directly into git, they must be propagated to the
1145wrll-linux-2.6.27/git/default_kernel bare clone of each individual build
1146to be present when the kernel is checked out.
1147<para>
1148The following example illustrates one variant of this workflow:
1149<literallayout class='monospaced'>
1150 # on master git repository
1151 &gt; cd linux-2.6.27
1152 &gt; git tag -d common_pc-standard-mark
1153 &gt; git pull ssh://&lt;foo&gt;@&lt;bar&gt;/pub/git/kernel-2.6.27 common_pc-standard:common_pc-standard
1154 &gt; git tag common_pc-standard-mark
1155
1156 # on each build machine (or NFS share, etc)
1157 &gt; cd wrll-linux-2.6.27/git/default_kernel
1158 &gt; git fetch ssh://&lt;foo&gt;@&lt;master server&gt;/pub/git/kernel-2.6.27
1159
1160 # in the build, perform a from-scratch build of Linux and the new changes
1161 # will be checked out and built.
1162 &gt; make linux
1163</literallayout>
1164</para>
1165 </section>
1166 </section>
1167
1168 <section id='bsp-template-migration-from-2'>
1169 <title>BSP: Template Migration from 2.0</title>
1170<para>
1171The move to a git-backed kernel build system in 3.0 introduced a small new
1172requirement for any BSP that is not integrated into the GA release of the
1173product: branching information.
1174</para>
1175<para>
1176As was previously mentioned in the background sections, branching information
1177is always required, since the kernel build system cannot make intelligent
1178branching decisions and must rely on the developer. This branching
1179information is provided via a .scc file.
1180</para>
1181<para>
1182A BSP template in 2.0 contained build system information (config.sh, etc) and
1183kernel patching information in the 'linux' subdirectory. The same holds true
1184in 3.0, with only minor changes in the kernel patching directory.
1185The ".smudge" files are now ".scc" files and now contain a full description
1186 of the kernel branching, patching and configuration for the BSP. Where in
1187 2.0, they only contained kernel patching information.
1188</para>
1189<para>
1190The following illustrates the migration of a simple 2.0 BSP template to the
1191new 3.0 kernel build system.
1192</para>
1193<note><para>
1194Note: all operations are from the root of a customer layer.
1195</para></note>
1196<literallayout class='monospaced'>
1197 templates/
1198 `&dash;&dash; board
1199 `&dash;&dash; my_board
1200 |&dash;&dash; config.sh
1201 |&dash;&dash; include
1202 `&dash;&dash; linux
1203 `&dash;&dash; 2.6.x
1204 |&dash;&dash; knl-base.cfg
1205 |&dash;&dash; bsp.patch
1206 `&dash;&dash; my_bsp.smudge
1207
1208 &gt; mv templates/board/my_board/linux/2.6.x/* templates/board/my_board/linux
1209 &gt; rm -rf templates/board/my_board/linux/2.6.x/
1210 &gt; mv templates/board/my_board/linux/my_bsp.smudge \
1211 templates/board/my_board/linux/my_bsp-standard.scc
1212 &gt; echo "kconf hardware knl-base.cfg" &gt;&gt; \
1213 templates/board/my_board/linux/my_bsp-standard.scc
1214 &gt; vi templates/board/my_board/linux/my_bsp-standard.scc
1215 # add the following at the top of the file
1216 scc_leaf ktypes/standard my_bsp-standard
1217
1218 templates/
1219 `&dash;&dash; board
1220 `&dash;&dash; my_board
1221 |&dash;&dash; config.sh
1222 |&dash;&dash; include
1223 `&dash;&dash; linux
1224 |&dash;&dash; knl-base.cfg
1225 |&dash;&dash; bsp.patch
1226 `&dash;&dash; my_bsp-standard.scc
1227</literallayout>
1228<para>
1229That's it. Configure and build.
1230</para>
1231<note><para>There is a naming convention for the .scc file, which allows the build
1232 system to locate suitable feature descriptions for a board:
1233</para></note>
1234<literallayout class='monospaced'>
1235 &lt;bsp name&gt;-&lt;kernel type&gt;.scc
1236</literallayout>
1237<para>
1238 if this naming convention isn't followed your feature description will
1239 not be located and a build error thrown.
1240</para>
1241 </section>
1242
1243 <section id='bsp-creating-a-new-bsp'>
1244 <title>BSP: Creating a New BSP</title>
1245<para>
1246Although it is obvious that the structure of a new BSP uses the migrated
1247directory structure from the previous example,the first question is whether
1248or not the BSP is started from scratch.
1249</para>
1250<para>
1251If Yocto Project has a similar BSP, it is often easier to clone and update,
1252rather than start from scratch. If the mainline kernel has support, it is
1253easier to branch from the -standard kernel and begin development (and not be
1254concerned with undoing existing changes). This section covers both options.
1255</para>
1256<para>
1257In almost every scenario, the LDAT build system bindings must be completed
1258before either cloning or starting a new BSP from scratch. This is simply
1259because the board template files are required to configure a project/build
1260and create the necessary environment to begin working directly with the
1261kernel. If it is desired to start immediately with kernel development and
1262then add LDAT bindings, see the "bootstrapping a BSP" section.
1263</para>
1264 <section id='creating-from-scratch'>
1265 <title>Creating the BSP from Scratch</title>
1266<para>
1267To create the BSP from scratch you need to do the following:
1268<orderedlist>
1269 <listitem><para>Create a board template for the new BSP in a layer.</para></listitem>
1270 <listitem><para>Configure a build with the board.</para></listitem>
1271 <listitem><para>Configure a kernel.</para></listitem>
1272</orderedlist>
1273</para>
1274<para>
1275Following is an example showing all three steps. You start by creating a board template for the new BSP in a layer.
1276<literallayout class='monospaced'>
1277 templates/
1278 `&dash;&dash; board
1279 `&dash;&dash; my_bsp
1280 |&dash;&dash; include
1281 |&dash;&dash; config.sh
1282 `&dash;&dash; linux
1283 |&dash;&dash; my_bsp.cfg
1284 `&dash;&dash; my_bsp-standard.scc
1285
1286 &gt; cat config.sh
1287 TARGET_BOARD="my_bsp"
1288 TARGET_LINUX_LINKS="bzImage"
1289 TARGET_SUPPORTED_KERNEL="standard"
1290 TARGET_SUPPORTED_ROOTFS="glibc_std"
1291 BANNER="This BSP is *NOT* supported"
1292 TARGET_PROCFAM="pentium4"
1293 TARGET_PLATFORMS="GPP"
1294
1295 &gt; cat include
1296 cpu/x86_32_i686
1297 karch/i386
1298
1299 &gt; cat linux/my_bsp-standard.scc
1300 scc_leaf ktypes/standard/standard.scc my_bsp-standard
1301
1302 &gt; cat linux/my_bsp.cfg
1303 CONFIG_X86=y
1304 CONFIG_SMP=y
1305 CONFIG_VT=y
1306 # etc, etc, etc
1307</literallayout>
1308</para>
1309<para>
1310Something like the following can now be added to a board build, and
1311a project can be started:
1312<literallayout class='monospaced'>
1313 &dash;&dash;enable-board=my_bsp \
1314 &dash;&dash;with-layer=custom_bsp
1315</literallayout>
1316</para>
1317<para>
1318Now you can configure a kernel:
1319<literallayout class='monospaced'>
1320 &gt; make -C build linux.config
1321</literallayout>
1322</para>
1323<para>
1324You now have a kernel tree, which is branched and has no patches, ready for
1325development.
1326</para>
1327 </section>
1328
1329 <section id='cloning-an-existing-bsp'>
1330 <title>Cloning an Existing BSP</title>
1331<para>
1332Cloning an existing BSP from the shipped product is similar to the "from
1333scratch" option and there are two distinct ways to achieve this goal:
1334<itemizedlist>
1335 <listitem><para>Create a board template for the new BSP in a layer.</para></listitem>
1336 <listitem><para>Clone the .scc and board config.</para></listitem>
1337</itemizedlist>
1338</para>
1339<para>
1340The first method is similar to the from scratch BSP where you create a board template for the new
1341BSP. Although in this case, copying an existing board template from
1342wrll-wrlinux/templates/board would be appropriate, since we are cloning an
1343existing BSP. Edit the config.sh, include and other board options for the new
1344BSP.
1345</para>
1346<para>
1347The second method is to clone the .scc and board config.
1348To do this, in the newly created board template, create a linux subdirectory and export
1349the .scc and configuration from the source BSP in the published Yocto Project
1350kernel. During construction, all of the configuration and patches were
1351captured, so it is simply a matter of extracting them.
1352</para>
1353<para>
1354Extraction can be accomplished using four different techniques:
1355<itemizedlist>
1356 <listitem><para>Config and patches from the bare default_kernel.</para></listitem>
1357 <listitem><para>Clone default_kernel and checkout wrs_base.</para></listitem>
1358 <listitem><para>Clone default_kernel and checkout BSP branch.</para></listitem>
1359 <listitem><para>Branch from the Yocto Project BSP.</para></listitem>
1360</itemizedlist>
1361</para>
1362<para>
1363Technique 1: config and patches from the bare default_kernel
1364<literallayout class='monospaced'>
1365 &gt; cd layers/wrll-linux-2.6.27/git/default_kernel
1366 &gt; git show checkpoint_end | filterdiff -i '*common_pc*' | patch -s -p2 -d /tmp
1367
1368 # This will create two directories: cfg and patches.
1369
1370 &gt; cd /tmp/cfg/kernel-cache/bsp/common_pc/
1371
1372 # This directory contains all the patches and .scc files used to construct
1373 # the BSP in the shipped tree. Copy the patches to the new BSP template,
1374 # and add them to the .scc file created above. See "template patching" if
1375 # more details are required.
1376</literallayout>
1377</para>
1378<para>
1379Technique 2: clone default_kernel and checkout wrs_base
1380<literallayout class='monospaced'>
1381 &gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
1382 &gt; cd windriver-2.6.27
1383 &gt; git checkout wrs_base
1384 &gt; cd wrs/cfg/kernel-cache/bsp/common_pc
1385
1386# again, this directory has all the patches and .scc files used to construct
1387# the BSP
1388</literallayout>
1389</para>
1390<para>
1391Technique 3: clone default_kernel and checkout BSP branch
1392<literallayout class='monospaced'>
1393 &gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
1394 &gt; cd windriver-2.6.27
1395 &gt; git checkout common_pc-standard
1396 &gt; git whatchanged
1397 # browse patches and determine which ones are of interest, say there are
1398 # 3 patches of interest
1399 &gt; git format-patch -o &lt;path to BSP template&gt;/linux HEAD^^^
1400 # update the .scc file to add the patches, see "template patches" if
1401 # more details are required
1402</literallayout>
1403</para>
1404<para>
1405Technique #4: branch from the Yocto Project BSP
1406<note><para>This is potentially the most "different" technique, but is actually
1407 the easiest to support and leverages the infrastructure. rtcore BSPs
1408 are created in a similar manner to this.
1409</para></note>
1410</para>
1411<para>
1412In this technique the .scc file in the board template is slightly different
1413 and indicates that the BSP should branch after the base Yocto Project BSP
1414 of the correct kernel type, so to start a new BSP that inherits the
1415 kernel patches of the common_pc-standard, the following would be done:
1416<literallayout class='monospaced'>
1417 &gt; cat linux/my_bsp-standard.scc
1418 scc_leaf bsp/common_pc/common_pc-standard.scc my_bsp-standard
1419</literallayout>
1420</para>
1421<para>
1422 And only kernel configuration (not patches) need be contained in the
1423 board template.
1424</para>
1425<para>
1426 This has the advantage of automatically picking up updates to the BSP
1427 and not duplicating any patches for a similar board.
1428</para>
1429 </section>
1430
1431 <section id='bsp-bootstrapping'>
1432 <title>BSP: Bootstrapping</title>
1433<para>
1434The previous examples created the board templates and configured a build
1435before beginning work on a new BSP. It is also possible for advanced users to
1436simply treat the Yocto Project git repository as an upstream source and begin
1437BSP development directly on the repository. This is the closest match to how
1438the kernel community at large would operate.
1439</para>
1440<para>
1441Two techniques exist to accomplish this:
1442</para>
1443<para>
1444Technique 1: upstream workflow
1445<literallayout class='monospaced'>
1446 &gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
1447 &gt; cd windriver-2.6.27
1448 &gt; git checkout -b my_bsp-standard common_pc-standard
1449
1450 # edit files, import patches, generally do BSP development
1451
1452 # at this point we can create the BSP template, and export the kernel
1453 # changes using one of the techniques discussed in that section. For
1454 # example, It is possible to push these changes, directly into the
1455 # default_kernel and never directly manipulate or export patch files
1456</literallayout>
1457</para>
1458<para>
1459Technique 2: Yocto Project kernel build workflow
1460</para>
1461<para>
1462 Create the BSP branch from the appropriate kernel type
1463<literallayout class='monospaced'>
1464 &gt; cd linux
1465 # the naming convention for auto-build is &lt;bsp&gt;-&lt;kernel type&gt;
1466 &gt; git checkout -b my_bsp-standard standard
1467</literallayout>
1468</para>
1469<para>
1470Make changes, import patches, etc.
1471<literallayout class='monospaced'>
1472 &gt; ../../host-cross/bin/guilt init
1473 # 'wrs/patches/my_bsp-standard' has now been created to
1474 # manage the branches patches
1475
1476 # option 1: edit files, guilt import
1477 &gt; ../../host-cross/bin/guilt new extra-version.patch
1478 &gt; vi Makefile
1479 &gt; ../../host-cross/bin/guilt refresh
1480 # add a header
1481 &gt; ../../host-cross/bin/guilt header -e
1482 # describe the patch using best practices, like the example below:
1483
1484 &dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt; cut here
1485 From: Bruce Ashfield &lt;bruce.ashfield@windriver.com&gt;
1486
1487 Adds an extra version to the kernel
1488
1489 Modify the main EXTRAVERSION to show our bsp name
1490
1491 Signed-off-by: Bruce Ashfield &lt;bruce.ashfield@windriver.com&gt;
1492 &dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt; cut here
1493
1494 # option 2: import patches
1495 &gt; git am &lt;patch&gt;
1496 or
1497 &gt; git apply &lt;patch&gt;
1498 &gt; git add &lt;files&gt;
1499 &gt; git commit -s
1500
1501 # configure the board, save relevant options
1502 &gt; make ARCH=&lt;arch&gt; menuconfig
1503
1504 # save the cfg changes for reconfiguration
1505 &gt; mkdir wrs/cfg/&lt;cache&gt;/my_bsp
1506 &gt; vi wrs/cfg/&lt;cache&gt;/my_bsp/my_bsp.cfg
1507
1508 # classify the patches
1509 &gt; ../../host-cross/bin/kgit classify create &lt;kernel-foo-cache&gt;/my_bsp/my_bsp
1510 # test build
1511 &gt; cd ..
1512 &gt; make linux TARGET_BOARD=my_bsp kprofile=my_bsp use_current_branch=1
1513</literallayout>
1514</para>
1515<para>
1516 Assuming the patches have been exported to the correct location, Future
1517 builds will now find the board, apply the patches to the base tree and make
1518 the relevant branches and structures and the special build options are no
1519 longer required.
1520</para>
1521 </section>
1522 </section>
1523
1524 <section id='patching'>
1525 <title>Patching</title>
1526<para>
1527The most common way to apply patches to the kernel is via a template.
1528However, for more advanced applications (such as the sharing of patches between
1529multiple sub-features) it is possible to patch the kernel-cache.
1530This section covers both scenarios.
1531</para>
1532 <section id='patching-template'>
1533 <title>Patching: Template</title>
1534<para>
1535kernel
1536templates follow the same rules as any LDAT template. A directory should be
1537created in a recognized template location, with a 'linux' subdirectory. The
1538'linux' directory triggers LDAT to pass the dir as a potential patch location
1539to the kernel build system. Any .scc files found in that directory, will be
1540automatically appended to the end of the BSP branch (for the configured
1541board).
1542</para>
1543<para>
1544This behavior is essentially the same since previous product
1545releases. The only exception is the use of ".scc", which allows kernel
1546configuration AND patches to be applied in a template.
1547</para>
1548<note><para>
1549If creating a full template is not required, a .scc file can be placed at
1550the top of the build, along with configuration and patches. The build
1551system will pickup the .scc and add it onto the patch list automatically
1552</para></note>
1553<para>
1554As an example, consider a simple template to update a BP:
1555<literallayout class='monospaced'>
1556 &gt; cat templates/feature/extra_version/linux/extra_version.scc
1557 patch 0001-extraversion-add-Wind-River-identifier.patch
1558</literallayout>
1559</para>
1560<para>
1561To illustrate how the previous template patch was created, the following
1562steps were performed:
1563<literallayout class='monospaced'>
1564 &gt; cd &lt;board build&gt;/build/linux
1565 &gt; vi Makefile
1566 # modify EXTRAVERSION to have a unique string
1567 &gt; git commit -s -m "extraversion: add Yocto Project identifier" Makefile
1568 &gt; git format-patch -o &lt;path to layer&gt;/templates/feature/extra_version/linux/
1569 &gt; echo "patch 0001-extraversion-add-Wind-River-identifier.patch" &gt; \
1570 &lt;path to layer&gt;/templates/feature/extra_version/linux/extra_version.scc
1571</literallayout>
1572</para>
1573<para>
1574This next example creates a template with a linux subdirectory, just as we
1575 always have for previous releases.
1576<literallayout class='monospaced'>
1577 &gt; mkdir templates/features/my_feature/linux
1578</literallayout>
1579</para>
1580<para>
1581 In that directory place your feature description, your
1582 patch and configuration (if required).
1583<literallayout class='monospaced'>
1584 &gt; ls templates/features/my_feature/linux
1585
1586 version.patch
1587 my_feature.scc
1588 my_feature.cfg
1589</literallayout>
1590</para>
1591<para>
1592 The .scc file describes the patches, configuration and
1593 where in the patch order the feature should be inserted.
1594<literallayout class='monospaced'>
1595 patch version.patch
1596 kconf non-hardware my_feature.cfg
1597</literallayout>
1598</para>
1599<para>
1600 Configure your build with the new template
1601<literallayout class='monospaced'>
1602 &dash;&dash;with-template=features/my_feature
1603</literallayout>
1604</para>
1605<para>
1606Build the kernel
1607<literallayout class='monospaced'>
1608 &gt; make linux
1609</literallayout>
1610</para>
1611 </section>
1612
1613 <section id='patching-kernel-cache'>
1614 <title>Patching: Kernel Cache</title>
1615<para>
1616As previously mentioned, this example is included for completeness, and is for more advanced
1617applications (such as the sharing of patches between multiple sub-features).
1618Most patching should be done via templates, since that interface is
1619guaranteed not to change and the kernel-cache interface carries no such
1620guarantee.
1621</para>
1622<para>
1623At the top of a layer, create a kernel cache. The build system will recognize
1624any directory of the name 'kernel-*-cache' as a kernel cache.
1625<literallayout class='monospaced'>
1626 &gt; cd &lt;my layer&gt;
1627 &gt;mkdir kernel-temp-cache
1628</literallayout>
1629</para>
1630<para>
1631Make a directory with the BSP
1632<literallayout class='monospaced'>
1633 &gt; mkdir kernel-temp-cache
1634 &gt; mkdir kernel-temp-cache/my_feat
1635</literallayout>
1636</para>
1637<para>
1638Create the feature files as they were in technique #1
1639<literallayout class='monospaced'>
1640 &gt; echo "patch my_patch.path" &gt; kernel-temp-cache/my_feat/my_feature.scc
1641</literallayout>
1642</para>
1643<para>
1644Configure the build with the feature added to the kernel type
1645<literallayout class='monospaced'>
1646 &dash;&dash;with-kernel=standard+my_feat/my_feature.scc
1647</literallayout>
1648</para>
1649<para>
1650Build the kernel
1651<literallayout class='monospaced'>
1652 &gt; make linux
1653</literallayout>
1654</para>
1655 </section>
1656 </section>
1657
1658 <section id='bsp-updating-patches-and-configuration'>
1659 <title>BSP: Updating Patches and Configuration</title>
1660<para>
1661As was described in the "template patching" example, it is simple
1662to add patches to a BSP via a template, but often, it is desirable
1663to experiment and test patches before committing them to a template.
1664You can do this by modifying the BSP source.
1665</para>
1666<para>
1667Start as follows:
1668<literallayout class='monospaced'>
1669 &gt; cd linux
1670 &gt; git checkout &lt;bspname&gt;-&lt;kernel name&gt;
1671
1672 &gt; git am &lt;patch&gt;
1673</literallayout>
1674</para>
1675<para>
1676Or you can do this:
1677<literallayout class='monospaced'>
1678 &gt; kgit-import -t patch &lt;patch&gt;
1679
1680 &gt; cd ..
1681 &gt; make linux
1682</literallayout>
1683</para>
1684<para>
1685For details on conflict resolution and patch application, see the
1686git manual, or other suitable online references.
1687<literallayout class='monospaced'>
1688 &gt; git am &lt;mbox&gt;
1689 # conflict
1690 &gt; git apply &dash;&dash;reject .git/rebase-apply/0001
1691 # resolve conflict
1692 &gt; git am &dash;&dash;resolved (or git am &dash;&dash;skip, git am &dash;&dash;abort)
1693 # continue until complete
1694</literallayout>
1695</para>
1696<para>
1697Here is another example:
1698<literallayout class='monospaced'>
1699 # merge the patches
1700 # 1) single patch
1701 &gt; git am &lt;mbox&gt;
1702 &gt; git apply &lt;patch&lt;
1703 &gt; kgit import -t patch &lt;patch&gt;
1704
1705 # 2) multiple patches
1706 &gt; git am &lt;mbox&gt;
1707 &gt; kgit import -t dir &lt;dir&gt;
1708
1709 # if kgit -t dir is used, a patch resolution cycle such
1710 # as this can be used:
1711
1712 &gt; kgit import -t dir &lt;dir&gt;
1713 # locate rejects and resolve
1714 # options:
1715 &gt; wiggle &dash;&dash;replace &lt;path to file&gt; &lt;path to reject&gt;
1716 &gt; guilt refresh
1717 or
1718 &gt; # manual resolution
1719 &gt; git add &lt;files&gt;
1720 &gt; git commit -s
1721 or
1722 &gt; git apply &dash;&dash;reject .git/rebase-apply/0001
1723 &gt; git add &lt;files&gt;
1724 &gt; git am &dash;&dash;resolved
1725 or
1726 &gt; # merge tool of choice
1727
1728 # continue series:
1729
1730 &gt; kgit import -t dir &lt;dir&gt;
1731 or
1732 &gt; git am &dash;&dash;continue
1733</literallayout>
1734</para>
1735<para>
1736Once all the patches have been tested and are satisfactory, they
1737should be exported via the techniques described in "saving kernel
1738modifications."
1739</para>
1740<para>
1741Once the kernel has been patched and configured for a BSP, it's
1742configuration commonly needs to be modified. This can be done by
1743running [menu|x]config on the kernel tree, or working with
1744configuration fragments.
1745</para>
1746<para>
1747Using menuconfig, the operation is as follows:
1748<literallayout class='monospaced'>
1749 &gt; make linux.menuconfig
1750 &gt; make linux.rebuild
1751</literallayout>
1752</para>
1753<para>
1754Once complete, the changes are in linux-&lt;bsp&gt;-&lt;kernel type&gt;-build/.config.
1755To permanently save these changes, compare the .config before and after the
1756menuconfig, and place those changes in a configuration fragment in the
1757template of your choice.
1758</para>
1759<para>
1760Using configuration fragments, the operation is as follows (using the
1761si_is8620 as an example BSP):
1762<literallayout class='monospaced'>
1763 &gt; vi linux/wrs/cfg/kernel-cache/bsp/si_is8620/si_is8620.cfg
1764 &gt; make linux.reconfig
1765 &gt; make linux.rebuild
1766</literallayout>
1767</para>
1768<para>
1769The modified configuration fragment can simply be copied out of the
1770linux/wrs/.. directory and placed in the appropriate template for future
1771application.
1772</para>
1773 </section>
1774
1775 <section id='tools-guilt'>
1776 <title>Tools: guilt</title>
1777<para>
1778Yocto Project has guilt integrated as a kernel tool; therefore users that are
1779familiar with quilt may wish to use this tool to pop, push and refresh
1780their patches. Note: guilt should only be used for local operations, once
1781a set of changes has been pushed or pulled, they should no longer be popped
1782or refresh by guilt, since popping, refreshing and re-pushing patches
1783changes their commit IDs and creating non-fast forward branches.
1784</para>
1785<para>
1786The following example illustrates how to add patches a Yocto Project
1787BSP branch via guilt:
1788<literallayout class='monospaced'>
1789 &gt; cd build/linux
1790 &gt; git checkout common_pc-standard
1791 &gt; guilt new extra.patch
1792 # edit files, make changes, etc
1793 &gt; guilt refresh
1794 &gt; guilt top
1795 extra.patch
1796
1797 # export that patch to an external location
1798 &gt; kgit export -p top /tmp
1799</literallayout>
1800</para>
1801<para>
1802Other guilt operations of interest are:
1803<literallayout class='monospaced'>
1804 > guilt push, guilt push -a
1805 > guilt pop
1806 > guilt applied, guilt unapplied
1807 > guilt top
1808 > guilt refresh
1809 > guilt header -e
1810 > guilt next
1811</literallayout>
1812</para>
1813<note><para>
1814Guilt only uses git commands and git plumbing to perform its operations,
1815anything that guilt does can also be done using git directly. It is provided
1816as a convenience utility, but is not required and the developer can use whatever
1817tools or workflow they wish.
1818</para></note>
1819<para>
1820The following builds from the above instructions to show how guilt can be
1821used to assist in getting your BSP kernel patches ready. You should follow
1822the above instructions up to and including 'make linux.config'. In this
1823example I will create a new commit (patch) from scratch and import another
1824fictitious patch from some external public git tree (ie, a commit with full
1825message, signoff etc.). Please ensure you have host-cross/bin in your path.
1826<literallayout class='monospaced'>
1827 %> cd linux
1828 %> guilt-init
1829 %> guilt-new -m fill_me_in_please first_one.patch
1830 %> touch somefile.txt
1831 %> guilt-add somefile.txt
1832 %> guilt-header -e
1833 %> guilt-refresh
1834 %> guilt-import path_to_some_patch/patch_filename
1835 %> guilt-push
1836</literallayout>
1837</para>
1838<para>
1839Here are a few notes about the above:
1840<itemizedlist>
1841 <listitem><para>guilt-header -e &dash;&dash; this will open editing of the patch header in
1842 EDITOR. As with a git commit the first line is the short log and
1843 should be just that short and concise message about the commit. Follow
1844 the short log with lines of text that will be the long description but
1845 note Do not put a blank line after the short log. As usual you will
1846 want to follow this with a blank line and then a signoff line.</para></listitem>
1847
1848 <listitem><para>The last line in the example above has 2 dots on the end. If you
1849 don't add the 2 periods on the end guilt will think you are sending
1850 just one patch. The wrong one!</para></listitem>
1851
1852 <listitem><para>The advantage to using guilt over not using guilt is that if you have a
1853 review comment in the first patch (first_one.patch in the case of this
1854 example) it is very easy to use guilt to pop the other patches off
1855 allowing you to make the necessary changes without having to use more
1856 inventive git type strategies.</para></listitem>
1857</itemizedlist>
1858</para>
1859 </section>
1860
1861 <section id='tools-scc-file-example'>
1862 <title>Tools: scc File Example</title>
1863<para>
1864This section provides some scc file examples: leaf node, 'normal' mode, and transforms.
1865</para>
1866 <section id='leaf-node'>
1867 <title>Leaf Node</title>
1868<para>
1869The following example is a BSP branch with no child branches - a leaf on the tree.
1870<literallayout class='monospaced'>
1871 # these are optional, but allow standalone tree construction
1872 define WRS_BOARD &lt;name&gt;
1873 define WRS_KERNEL &lt;kern type&gt;
1874 define WRS_ARCH &lt;arch&gt;
1875
1876 scc_leaf ktypes/standard common_pc-standard
1877 # ^ ^
1878 # +&dash;&dash; parent + branch name
1879
1880 include common_pc.scc
1881 # ^
1882 # +&dash;&dash;&dash; include another feature
1883</literallayout>
1884</para>
1885 </section>
1886
1887 <section id='normal-mode'>
1888 <title>'Normal' Mode</title>
1889<para>
1890Here is an example of 'normal' mode:
1891<literallayout class='monospaced'>
1892 # +&dash;&dash;&dash;&dash; name of file to read
1893 # v
1894 kconf hardware common_pc.cfg
1895 # ^ ^
1896 # | +&dash;&dash; 'type: hardware or non-hardware
1897 # |
1898 # +&dash;&dash;&dash; kernel config
1899
1900 # patches
1901 patch 0002-atl2-add-atl2-driver.patch
1902 patch 0003-net-remove-LLTX-in-atl2-driver.patch
1903 patch 0004-net-add-net-poll-support-for-atl2-driver.patch
1904</literallayout>
1905</para>
1906
1907 </section>
1908
1909 <section id='transforms'>
1910 <title>Transforms</title>
1911<para>
1912This section shows an example of transforms:
1913<literallayout class='monospaced'>
1914 # either of the next two options will trigger an 'auto'
1915 # branch from existing ones, since they change the commit
1916 # order and hence must construct their own branch
1917
1918 # this changes the order of future includes, if the
1919 # passed feature is detected, the first feature is
1920 # included AFTER it
1921 include features/rt/rt.scc after features/kgdb/kgdb
1922 # this also changes the order of existing branches
1923 # this prevents the named feature from ever being
1924 # included
1925 exclude features/dynamic_ftrace/dynamic_ftrace.scc
1926
1927 # inherit the standard kernel
1928 include ktypes/standard/standard
1929
1930
1931 # LTT supplies this, so we don't want the sub-chunk from RT.
1932 patch_trigger arch:all exclude ftrace-upstream-tracepoints.patch
1933 # ...but we still want the one unique tracepoint it added.
1934 patch tracepoint-add-for-sched_resched_task.patch
1935
1936 # these will change the named patches in the series into
1937 # &lt;patch name&gt;.patch.&lt;feature name&gt;
1938 # where the substituted patch is in this directory
1939 patch_trigger arch:all ctx_mod dynamic_printk.patch
1940 patch_trigger arch:all ctx_mod 0001-Implement-futex-macros-for-ARM.patch
1941 # unconditionally exclude a patch
1942 patch_trigger arch:all exclude ftrace-fix-ARM-crash.patch
1943</literallayout>
1944</para>
1945 </section>
1946 </section>
1947
1948 <section id='tip-dirty-string'>
1949 <title>"-dirty" String</title>
1950<para>
1951If kernel images are being built with -dirty on the end of the version
1952string, this simply means that there are modification in the source
1953directory that haven't been committed.
1954<literallayout class='monospaced'>
1955 &gt; git status
1956</literallayout>
1957</para>
1958<para>
1959The above git command will indicate modified, removed or added files. Those changes should
1960be committed to the tree (even if they will never be saved, or exported
1961for future use) and the kernel rebuilt.
1962</para>
1963<para>
1964To brute force pickup and commit all such pending changes enter the following:
1965<literallayout class='monospaced'>
1966 &gt; git add .
1967 &gt; git commit -s -a -m "getting rid of -dirty"
1968</literallayout>
1969</para>
1970<para>
1971And then rebuild the kernel
1972</para>
1973 </section>
1974
1975 <section id='kernel-transition-kernel-layer'>
1976 <title>Kernel: Transition Kernel Layer</title>
1977<para>
1978In order to temporarily use a different base kernel in Yocto Project
1979Linux 3.0 you need to do the following:
1980<orderedlist>
1981 <listitem><para>Create a custom kernel layer.</para></listitem>
1982 <listitem><para>Create a git repository of the transition kernel.</para></listitem>
1983</orderedlist>
1984</para>
1985<para>
1986Once those requirements are met multiple boards and kernels can
1987be built. The cost of setup is only paid once and then additional
1988BSPs and options can be added.
1989</para>
1990<para>
1991This creates a transition kernel layer to evaluate functionality
1992of some other kernel with the goal of easing transition to an
1993integrated and validated Yocto Project kernel.
1994</para>
1995<para>
1996The next few sections describe the process:
1997</para>
1998 <section id='creating-a-custom-kernel-layer'>
1999 <title>Creating a Custom Kernel Layer</title>
2000<para>
2001The custom kernel layer must have the following minimum
2002elements:
2003<itemizedlist>
2004 <listitem><para>An include of the shipped Yocto Project kernel layer.</para></listitem>
2005 <listitem><para>A kernel-cache with an override of the standard kernel type.</para></listitem>
2006</itemizedlist>
2007</para>
2008<para>
2009This allows the inheritance of the kernel build infrastructure,
2010while overriding the list of patches that should be applied to
2011the base kernel.
2012</para>
2013<para>
2014The kernel layer can optionally include an override to the base
2015Yocto Project Linux BSP to inhibit the application of BSP specific
2016patches. If a custom BSP is being used, this is not required.
2017</para>
2018 </section>
2019
2020 <section id='git-repo-of-the-transition-kernel'>
2021 <title>git Repo of the Transition Kernel</title>
2022<para>
2023The kernel build system requires a base kernel repository to
2024seed the build process. This repository must be found in the
2025same layer as the build infrastructure (i.e wrll-linux-2.6.27)
2026in the 'git' subdir, with the name 'default_kernel'
2027</para>
2028<para>Since Yocto Project Linux ships with a default_kernel
2029(the validated Yocto Project kernel) in the wrll-linux-2.6.27
2030kernel layer, that must be removed and replaced with the
2031transition kernel.
2032</para>
2033<para>If the Yocto Project install cannot be directly modified
2034with the new default kernel, then the path to the transition
2035kernel layer's 'git' subdir must be passed to the build
2036process via:
2037<programlisting>
2038linux_GIT_BASE=&lt;absolute path to layer&gt;/git
2039</programlisting>
2040</para>
2041<para>
2042If the transition kernel has not been delivered via git,
2043then a git repo should be created, and bare cloned into
2044place. Creating this repository is as simple as:
2045<literallayout class='monospaced'>
2046 &gt; tar zxvf temp_kernel.tgz
2047 &gt; cd temp_kernel
2048 &gt; git init
2049 &gt; git add .
2050 &gt; git commit -a -m "Transition kernel baseline"
2051
2052 'temp_kernel' can now be cloned into place via:
2053
2054 &gt; cd &lt;path to git base&gt;/git
2055 &gt; git clone &dash;&dash;bare &lt;path to temp_kernel/temp_kernel default_kernel
2056</literallayout>
2057</para>
2058 </section>
2059
2060 <section id='building-the-kernel'>
2061 <title>Building the Kernel</title>
2062<para>
2063Once these prerequisites have been met, the kernel can be
2064built with:
2065<literallayout class='monospaced'>
2066 &gt; make linux
2067</literallayout>
2068</para>
2069<para>
2070The new base kernel will be cloned into place and have any patches
2071indicated in the transition kernel's cache (or templates) applied.
2072The kernel build will detect the non-Yocto Project base repo and
2073use the HEAD of the tree for the build.
2074</para>
2075 </section>
2076
2077 <section id='example'>
2078 <title>Example</title>
2079<para>
2080This example creates a kernel layer to build the latest
2081kernel.org tree as the 'common_pc' BSP.
2082<literallayout class='monospaced'>
2083 &gt; cd &lt;path to layers&gt;
2084 &gt; mkdir wrll-linux-my_version
2085 &gt; cd wrll-linux-my_version
2086 &gt; echo "wrll-linux-2.6.27" &gt; include
2087 &gt; mkdir -p kernel-cache/ktypes/standard
2088 &gt; mkdir -p kernel-cache/bsp/common_pc
2089 &gt; echo "v2.6.29" &gt; kernel-cache/kver
2090 &gt; echo "branch common_pc-standard" &gt; kernel-cache/bsp/common_pc/common_pc.scc
2091 &gt; echo "kconf hardware common_pc.cfg" &gt;&gt; kernel-cache/bsp/common_pc/common_pc.scc
2092 &gt; echo "CONFIG_FOO=y" &gt; kernel-cache/bsp/common_pc/common_pc.cfg
2093 &gt; mkdir git
2094 &gt; cd git
2095 &gt; git clone &dash;&dash;bare git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git default_kernel
2096</literallayout>
2097</para>
2098<para>
2099Configure a build to use the new layer. This means that:
2100<literallayout class='monospaced'>
2101 &dash;&dash;enable-kernel-version=my_version
2102</literallayout>
2103</para>
2104<para>
2105Should be used to override the shipped default.
2106</para>
2107<para>
2108To build the kernel:
2109<literallayout class='monospaced'>
2110 &gt; cd build
2111 &gt; make linux_GIT_BASE=&lt;layer path&gt;/wrll-linux-my_version/git linux
2112</literallayout>
2113</para>
2114<para>
2115If this is to build without some user intervention (passing of the
2116GIT_BASE), you must do the clone into the wrll-linux-2.6.27/git directory.
2117</para>
2118<note><para>Unless you define valid "hardware.kcf" and "non-hardware.kcf" some
2119non fatal warnings will be seen. They can be fixed by populating these
2120files in the kernel-cache with valid hardware and non hardware config
2121options.
2122</para></note>
2123 </section>
2124 </section>
2125 </section>
2126
2127
2128
2129
2130
2131<!-- <itemizedlist>
2132 <listitem><para>Introduction to this section.</para></listitem>
2133 <listitem><para>Constructing a project-specific kernel tree.</para></listitem>
2134 <listitem><para>Building the kernel.</para></listitem>
2135 <listitem><para>Seeing what has changed.</para></listitem>
2136 <listitem><para>Seeing what has changed in a particular branch.</para></listitem>
2137 <listitem><para>Modifying the kernel.</para></listitem>
2138 <listitem><para>Saving modifications.</para></listitem>
2139 <listitem><para>Storing patches outside of the kernel source repository (bulk export).</para></listitem>
2140 <listitem><para>Working with incremental changes.</para></listitem>
2141 <listitem><para>Extracting commited changes from a working directory (exporting internally through
2142 patches.</para></listitem>
2143 <listitem><para>Pushing commited changes.</para></listitem>
2144 <listitem><para>Exporting for external (upstream) submission.</para></listitem>
2145 <listitem><para>Exporting for import into another Source Control Manager (SCM).</para></listitem>
2146 <listitem><para>Working with the Yocto Project kernel in another SCM.</para>
2147 <itemizedlist>
2148 <listitem><para>Exporting the delivered kernel to an SCM.</para></listitem>
2149 <listitem><para>Importing changed for the build.</para></listitem>
2150 </itemizedlist></listitem>
2151 <listitem><para>Migrating templates from version 2.0.</para></listitem>
2152 <listitem><para>Creating a new Board Support Package (BSP).</para>
2153 <itemizedlist>
2154 <listitem><para>Creating from scratch.</para></listitem>
2155 <listitem><para>Cloning.</para></listitem>
2156 </itemizedlist></listitem>
2157 <listitem><para>BSP bootstrapping.</para></listitem>
2158 <listitem><para>Applying patches to the kernel through a template.</para></listitem>
2159 <listitem><para>Applying patches to the kernel without using a template.</para></listitem>
2160 <listitem><para>Updating patches and configurations for a BSP.</para></listitem>
2161 <listitem><para>Using guilt to add and export patches.</para></listitem>
2162 <listitem><para>Using scc.</para></listitem>
2163 <listitem><para>Building a 'dirty' image.</para></listitem>
2164 <listitem><para>Temporarily using a different base kernel.</para></listitem>
2165 <listitem><para>Creating a custom kernel layer.</para></listitem>
2166 <listitem><para>Creating the git repository of the transition kernel.</para></listitem>
2167 </itemizedlist> -->
2168
2169
2170</section>
2171
2172</article>
2173<!--
2174vim: expandtab tw=80 ts=4
2175-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-18 05:57:48 +0000