summaryrefslogtreecommitdiffstats
path: root/documentation
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2012-12-26 14:09:53 -0600
committerRichard Purdie <richard.purdie@linuxfoundation.org>2013-01-16 15:59:09 +0000
commitea6d5be3ce49a6e64eb8d5fc62e72dc26cc6cb97 (patch)
tree3c9e9b3da6bb5116e08421972b19e79a44b58b7c /documentation
parent46b0fc3ec6b4090056560935d35aef26aa3c8ac5 (diff)
downloadpoky-ea6d5be3ce49a6e64eb8d5fc62e72dc26cc6cb97.tar.gz
kernel-dev: First edits to chapter 3 intro and section 3.1.
(From yocto-docs rev: bb640ebcfd99eaf3944711edc813a4bd39aa3753) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation')
-rw-r--r--documentation/kernel-dev/kernel-dev-advanced.xml1154
1 files changed, 252 insertions, 902 deletions
diff --git a/documentation/kernel-dev/kernel-dev-advanced.xml b/documentation/kernel-dev/kernel-dev-advanced.xml
index 9d9aef6d06..27c082a940 100644
--- a/documentation/kernel-dev/kernel-dev-advanced.xml
+++ b/documentation/kernel-dev/kernel-dev-advanced.xml
@@ -2,916 +2,266 @@
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4 4
5<chapter id='kernel-how-to'> 5<chapter id='kernel-dev-advanced'>
6 6
7<title>Working with the Yocto Project Kernel</title> 7<title>Working with Advanced Metadata</title>
8 8
9 <para>
10 In addition to configuration fragments and patches, the
11 Yocto Project kernel tools support rich metadata that you can
12 use to define complex policies and BSP support.
13 The purpose of the metadata and the tools to manage it, known as
14 the kern-tools (<filename>kern-tools-native_git.bb</filename>), is
15 to assist in managing the complexity of the configuration and sources
16 in support of multiple Board Support Packages (BSPs) and Linux kernel
17 types.
18 </para>
9 19
10<section id='actions-org'>
11 <title>Introduction</title>
12 <para> 20 <para>
13 This chapter describes how to accomplish tasks involving a kernel's tree structure. 21 In particular, the kernel tools allow you to specify only what you
14 The information is designed to help the developer that wants to modify the Yocto 22 must, and nothing more.
15 Project kernel and contribute changes upstream to the Yocto Project. 23 Where a complete Linux kernel <filename>.config</filename> includes
16 The information covers the following: 24 all the automatically selected <filename>CONFIG</filename> options,
17 <itemizedlist> 25 the configuration fragments only need to contain the highest level
18 <listitem><para>Tree construction</para></listitem> 26 visible <filename>CONFIG</filename> options as presented by the Linux
19 <listitem><para>Build strategies</para></listitem> 27 kernel <filename>menuconfig</filename> system.
20 <listitem><para>Workflow examples</para></listitem> 28 This reduces your maintenance effort and allows you
21 </itemizedlist> 29 to further separate your configuration in ways that make sense for
30 your project.
31 A common split is policy and hardware.
32 For example, all your kernels might support
33 the <filename>proc</filename> and <filename>sys</filename> filesystems,
34 but only specific boards will require sound, USB, or specific drivers.
35 Specifying these individually allows you to aggregate them
36 together as needed, but maintain them in only one place.
37 Similar logic applies to source changes.
38 </para>
39
40 <para>
41 Original Text:
42 <literallayout class='monospaced'>
43In addition to configuration fragments and patches, the Yocto Project kernel
44tools support rich metadata which you can use to define complex policies and
45BSP support. The purpose of the metadata and the tools to manage it, known as
46the kern-tools (kern-tools-native_git.bb), is to assist in managing the
47complexity of the configuration and sources in support of multiple BSPs and
48Linux kernel types.
49
50In particular, the kernel tools allow you to specify only what you must, and
51nothing more. Where a complete Linux kernel .config includes all the
52automatically selected CONFIG options, the configuration fragments only need to
53contain the highest level visible CONFIG options as presented by the Linux
54kernel menuconfig system. This reduces your maintenance effort and allows you
55to further separate your configuration in ways that make sense for your project.
56A common split is policy and hardware. For example, all your kernels may support
57the proc and sys filesystems, but only specific boards will require sound, usb,
58or specific drivers. Specifying these individually allows you to aggregate them
59together as needed, but maintain them in only one place. Similar logic applies
60to source changes.
61 </literallayout>
62 </para>
63
64<section id='using-metadata-in-a-recipe'>
65 <title>Using Metadata in a Recipe</title>
66
67 <para>
68 The metadata provided with any linux-yocto style Linux kernel sources
69 must define a BSP that corresponds to the definition laid out in the
70 recipe.
71 A BSP consists of an aggregation of kernel policy and hardware specific
72 feature enablement.
73 This can be influenced from within the recipe.
74 </para>
75
76 <para>
77 Every linux-yocto style recipe must define the following variable:
78 <literallayout class='monospaced'>
79 KMACHINE
80 </literallayout>
81 <filename>KMACHINE</filename> is typically set to the same value as
82 used within the recipe-space BSP definition, such as "routerstationpro"
83 or "fri2".
84 However, multiple BSPs can reuse the same <filename>KMACHINE</filename>
85 name if they are built using the same BSP description.
86 See section 3.3.5 for more information.
87 The <filename>meta-intel</filename> "fri2" and "fri2-noemgd" are good
88 examples of such a situation where each specifies
89 <filename>KMACHINE</filename> as "fri2".
90 </para>
91
92 <para>
93 They may optionally define the following variables:
94 <literallayout class='monospaced'>
95 KBRANCH
96 KERNEL_FEATURES
97 KBRANCH_DEFAULT
98 LINUX_KERNEL_TYPE
99 </literallayout>
100 <filename>KBRANCH_DEFAULT</filename> defines the default source branch
101 within the Linux kernel source repository to be used to build the
102 Linux kernel.
103 It is used as the default value for <filename>KBRANCH</filename> which
104 may define an alternate branch, typically with a machine override,
105 such as:
106 <literallayout class='monospaced'>
107 KBRANCH_fri2 = "standard/fri2"
108 </literallayout>
109 Unless you specify otherwise, <filename>KBRANCH_DEFAULT</filename>
110 is initialized to "master".
111 </para>
112
113 <para>
114 <filename>LINUX_KERNEL_TYPE</filename> defines the kernel type to be
115 used in assembling the configuration and defaults to "standard"
116 if you do not specify otherwise.
117 Together with <filename>KMACHINE</filename>, this defines the search
118 arguments used by the Yocto Project Linux kernel tools to find the
119 appropriate description within the metadata with which to build out
120 the sources and configuration.
121 The linux-yocto recipes define "standard", "tiny", and "preempt-rt"
122 kernel types.
123 See section 3.3.4 for more inforation on kernel types.
22 </para> 124 </para>
23</section>
24 125
25 <section id='tree-construction'> 126 <para>
26 <title>Tree Construction</title> 127 During the build, the kern-tools will search for the BSP description
27 <para> 128 file that most closely matches the <filename>KMACHINE</filename>
28 This section describes construction of the Yocto Project kernel source repositories 129 and <filename>LINUX_KERNEL_TYPE</filename> passed in from the
29 as accomplished by the Yocto Project team to create kernel repositories. 130 recipe.
30 These kernel repositories are found under the heading "Yocto Linux Kernel" at 131 It will use the first BSP description it finds matching both variables.
31 <ulink url='&YOCTO_GIT_URL;/cgit.cgi'>&YOCTO_GIT_URL;/cgit.cgi</ulink> 132 Failing that it will issue a warning such as the following:
32 and can be shipped as part of a Yocto Project release.
33 The team creates these repositories by
34 compiling and executing the set of feature descriptions for every BSP/feature
35 in the product.
36 Those feature descriptions list all necessary patches,
37 configuration, branching, tagging and feature divisions found in a kernel.
38 Thus, the Yocto Project kernel repository (or tree) is built.
39 </para>
40 <para>
41 The existence of this tree allows you to access and clone a particular
42 Yocto Project kernel repository and use it to build images based on their configurations
43 and features.
44 </para>
45 <para>
46 You can find the files used to describe all the valid features and BSPs
47 in the Yocto Project kernel in any clone of the Yocto Project kernel source repository
48 Git tree.
49 For example, the following command clones the Yocto Project baseline kernel that
50 branched off of <filename>linux.org</filename> version 3.4:
51 <literallayout class='monospaced'>
52 $ git clone git://git.yoctoproject.org/linux-yocto-3.4
53 </literallayout>
54 For another example of how to set up a local Git repository of the Yocto Project
55 kernel files, see the
56 "<ulink url='&YOCTO_DOCS_DEV_URL;#local-kernel-files'>Yocto Project Kernel</ulink>" bulleted
57 item in the Yocto Project Development Manual.
58 </para>
59 <para>
60 Once you have cloned the kernel Git repository on your local machine, you can
61 switch to the <filename>meta</filename> branch within the repository.
62 Here is an example that assumes the local Git repository for the kernel is in
63 a top-level directory named <filename>linux-yocto-3.4</filename>:
64 <literallayout class='monospaced'>
65 $ cd ~/linux-yocto-3.4
66 $ git checkout -b meta origin/meta
67 </literallayout>
68 Once you have checked out and switched to the <filename>meta</filename> branch,
69 you can see a snapshot of all the kernel configuration and feature descriptions that are
70 used to build that particular kernel repository.
71 These descriptions are in the form of <filename>.scc</filename> files.
72 </para>
73 <para>
74 You should realize, however, that browsing your local kernel repository
75 for feature descriptions and patches is not an effective way to determine what is in a
76 particular kernel branch.
77 Instead, you should use Git directly to discover the changes in a branch.
78 Using Git is an efficient and flexible way to inspect changes to the kernel.
79 For examples showing how to use Git to inspect kernel commits, see the following sections
80 in this chapter.
81 <note>
82 Ground up reconstruction of the complete kernel tree is an action only taken by the
83 Yocto Project team during an active development cycle.
84 When you create a clone of the kernel Git repository, you are simply making it
85 efficiently available for building and development.
86 </note>
87 </para>
88 <para>
89 The following steps describe what happens when the Yocto Project Team constructs
90 the Yocto Project kernel source Git repository (or tree) found at
91 <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink> given the
92 introduction of a new top-level kernel feature or BSP.
93 These are the actions that effectively create the tree
94 that includes the new feature, patch or BSP:
95 <orderedlist>
96 <listitem><para>A top-level kernel feature is passed to the kernel build subsystem.
97 Normally, this feature is a BSP for a particular kernel type.</para></listitem>
98 <listitem><para>The file that describes the top-level feature is located by searching
99 these system directories:
100 <itemizedlist>
101 <listitem><para>The in-tree kernel-cache directories, which are located
102 in <filename>meta/cfg/kernel-cache</filename></para></listitem>
103 <listitem><para>Areas pointed to by <filename>SRC_URI</filename> statements
104 found in recipes</para></listitem>
105 </itemizedlist>
106 For a typical build, the target of the search is a
107 feature description in an <filename>.scc</filename> file
108 whose name follows this format:
109 <literallayout class='monospaced'>
110 &lt;bsp_name&gt;-&lt;kernel_type&gt;.scc
111 </literallayout>
112 </para></listitem>
113 <listitem><para>Once located, the feature description is either compiled into a simple script
114 of actions, or into an existing equivalent script that is already part of the
115 shipped kernel.</para></listitem>
116 <listitem><para>Extra features are appended to the top-level feature description.
117 These features can come from the
118 <ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES'><filename>KERNEL_FEATURES</filename></ulink>
119 variable in recipes.</para></listitem>
120 <listitem><para>Each extra feature is located, compiled and appended to the script
121 as described in step three.</para></listitem>
122 <listitem><para>The script is executed to produce a series of <filename>meta-*</filename>
123 directories.
124 These directories are descriptions of all the branches, tags, patches and configurations that
125 need to be applied to the base Git repository to completely create the
126 source (build) branch for the new BSP or feature.</para></listitem>
127 <listitem><para>The base repository is cloned, and the actions
128 listed in the <filename>meta-*</filename> directories are applied to the
129 tree.</para></listitem>
130 <listitem><para>The Git repository is left with the desired branch checked out and any
131 required branching, patching and tagging has been performed.</para></listitem>
132 </orderedlist>
133 </para>
134 <para>
135 The kernel tree is now ready for developer consumption to be locally cloned,
136 configured, and built into a Yocto Project kernel specific to some target hardware.
137 <note><para>The generated <filename>meta-*</filename> directories add to the kernel
138 as shipped with the Yocto Project release.
139 Any add-ons and configuration data are applied to the end of an existing branch.
140 The full repository generation that is found in the
141 official Yocto Project kernel repositories at
142 <ulink url='&YOCTO_GIT_URL;/cgit.cgi'>http://git.yoctoproject.org/cgit.cgi</ulink>
143 is the combination of all supported boards and configurations.</para>
144 <para>The technique the Yocto Project team uses is flexible and allows for seamless
145 blending of an immutable history with additional patches specific to a
146 deployment.
147 Any additions to the kernel become an integrated part of the branches.</para>
148 </note>
149 </para>
150 </section>
151
152 <section id='build-strategy'>
153 <title>Build Strategy</title>
154 <para>
155 Once a local Git repository of the Yocto Project kernel exists on a development system,
156 you can consider the compilation phase of kernel development - building a kernel image.
157 Some prerequisites exist that are validated by the build process before compilation
158 starts:
159 </para>
160
161 <itemizedlist>
162 <listitem><para>The
163 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> points
164 to the kernel Git repository.</para></listitem>
165 <listitem><para>A BSP build branch exists.
166 This branch has the following form:
167 <literallayout class='monospaced'>
168 &lt;kernel_type&gt;/&lt;bsp_name&gt;
169 </literallayout></para></listitem>
170 </itemizedlist>
171
172 <para>
173 The OpenEmbedded build system makes sure these conditions exist before attempting compilation.
174 Other means, however, do exist, such as as bootstrapping a BSP, see
175 the "<link linkend='workflow-examples'>Workflow Examples</link>".
176 </para>
177
178 <para>
179 Before building a kernel, the build process verifies the tree
180 and configures the kernel by processing all of the
181 configuration "fragments" specified by feature descriptions in the <filename>.scc</filename>
182 files.
183 As the features are compiled, associated kernel configuration fragments are noted
184 and recorded in the <filename>meta-*</filename> series of directories in their compilation order.
185 The fragments are migrated, pre-processed and passed to the Linux Kernel
186 Configuration subsystem (<filename>lkc</filename>) as raw input in the form
187 of a <filename>.config</filename> file.
188 The <filename>lkc</filename> uses its own internal dependency constraints to do the final
189 processing of that information and generates the final <filename>.config</filename> file
190 that is used during compilation.
191 </para>
192
193 <para>
194 Using the board's architecture and other relevant values from the board's template,
195 kernel compilation is started and a kernel image is produced.
196 </para>
197
198 <para>
199 The other thing that you notice once you configure a kernel is that
200 the build process generates a build tree that is separate from your kernel's local Git
201 source repository tree.
202 This build tree has a name that uses the following form, where
203 <filename>${MACHINE}</filename> is the metadata name of the machine (BSP) and "kernel_type" is one
204 of the Yocto Project supported kernel types (e.g. "standard"):
205 <literallayout class='monospaced'> 133 <literallayout class='monospaced'>
206 linux-${MACHINE}-&lt;kernel_type&gt;-build 134 WARNING: Can't find any BSP hardware or required configuration fragments.
135 WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and
136 meta/cfg/broken/fri2-broken/required_frags.txt in directory:
137 meta/cfg/broken/fri2-broken
207 </literallayout> 138 </literallayout>
208 </para> 139 In this example, <filename>KMACHINE</filename> was set to "fri2-broken"
209 140 and <filename>LINUX_KERNEL_TYPE</filename> was set to "broken".
210 <para> 141 </para>
211 The existing support in the <filename>kernel.org</filename> tree achieves this 142
212 default functionality. 143 <para>
213 </para> 144 It will then search first for the <filename>KMACHINE</filename> and
214 145 then for the <filename>LINUX_KERNEL_TYPE</filename>.
215 <para> 146 If it cannot find a partial match, it will use the
216 This behavior means that all the generated files for a particular machine or BSP are now in 147 sources from the <filename>KBRANCH</filename> and any configuration
217 the build tree directory. 148 specified in the <filename>SRC_URI</filename>.
218 The files include the final <filename>.config</filename> file, all the <filename>.o</filename> 149 </para>
219 files, the <filename>.a</filename> files, and so forth. 150
220 Since each machine or BSP has its own separate build directory in its own separate branch 151 <para>
221 of the Git repository, you can easily switch between different builds. 152 <filename>KERNEL_FEATURES</filename> can be used to include features
222 </para> 153 (configuration fragments, patches, or both) that are not already
223 </section> 154 included by the <filename>KMACHINE</filename> and
224 155 <filename>LINUX_KERNEL_TYPE</filename> combination.
225 <section id='workflow-examples'> 156 To include a feature specified as "features/netfilter.scc" for example,
226 <title>Workflow Examples</title> 157 specify:
227 158 <literallayout class='monospaced'>
228 <para> 159 KERNEL_FEATURES += "features/netfilter.scc"
229 As previously noted, the Yocto Project kernel has built-in Git integration. 160 </literallayout>
230 However, these utilities are not the only way to work with the kernel repository. 161 To include a feature called "cfg/sound.scc" just for the
231 The Yocto Project has not made changes to Git or to other tools that 162 <filename>qemux86</filename> machine, specify:
232 would invalidate alternate workflows. 163 <literallayout class='monospaced'>
233 Additionally, the way the kernel repository is constructed results in using 164 KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc"
234 only core Git functionality, thus allowing any number of tools or front ends to use the 165 </literallayout>
235 resulting tree. 166 The value of the entries in <filename>KERNEL_FEATURES</filename>
236 </para> 167 are dependent on their location within the metadata itself.
237 168 The examples here are taken from the
238 <para> 169 <filename>linux-yocto-3.4</filename> repository where "features"
239 This section contains several workflow examples. 170 and "cfg" are subdirectories of the <filename>metadata</filename>
240 Many of the examples use Git commands. 171 directory.
241 You can find Git documentation at 172 For details, see section 3.3.
242 <ulink url='http://git-scm.com/documentation'></ulink>. 173 <note>
243 You can find a simple overview of using Git with the Yocto Project in the 174 The processing of the these variables has evolved some between the
244 "<ulink url='&YOCTO_DOCS_DEV_URL;#git'>Git</ulink>" 175 0.9 and 1.3 releases of the Yocto Project and associated
245 section of the Yocto Project Development Manual. 176 kern-tools sources.
246 </para> 177 The descriptions in this section are accurate for 1.3 and later
247 178 releases of the Yocto Project.
248 <section id='change-inspection-kernel-changes-commits'> 179 </note>
249 <title>Change Inspection: Changes/Commits</title> 180 </para>
250 181
251 <para> 182 <para>
252 A common question when working with a kernel is: 183 Original Text.
253 "What changes have been applied to this tree?" 184 <literallayout class='monospaced'>
254 </para> 185The metadata provided with any linux-yocto style Linux kernel sources must
255 186define a BSP that corresponds to the definition laid out in the recipe. A BSP
256 <para> 187consists of an aggregation of kernel policy and hardware specific feature
257 In projects that have a collection of directories that 188enablement. This can be influenced from within the recipe.
258 contain patches to the kernel, it is possible to inspect or "grep" the contents 189
259 of the directories to get a general feel for the changes. 190Every linux-yocto style recipe must define the following variables:
260 This sort of patch inspection is not an efficient way to determine what has been 191
261 done to the kernel. 192 KMACHINE
262 The reason it is inefficient is because there are many optional patches that are 193
263 selected based on the kernel type and the feature description. 194KMACHINE is typically set to the same value as used within the recipe-space BSP
264 Additionally, patches could exist in directories that are not included in the search. 195definition, such as "routerstationpro" or "fri2". However, multiple BSPs can
265 </para> 196reuse the same KMACHINE name if they are built using the same BSP description
266 197(see 3.3.5). The meta-intel "fri2" and "fri2-noemgd" are good examples of such
267 <para> 198a situation where each specifies KMACHINE as "fri2".
268 A more efficient way to determine what has changed in the branch is to use 199
269 Git and inspect or search the kernel tree. 200They may optionally define the following variables:
270 This method gives you a full view of not only the source code modifications, 201 KBRANCH
271 but also provides the reasons for the changes. 202 KERNEL_FEATURES
272 </para> 203 KBRANCH_DEFAULT
273 204 LINUX_KERNEL_TYPE
274 <section id='what-changed-in-a-kernel'> 205
275 <title>What Changed in a Kernel?</title> 206KBRANCH_DEFAULT defines the default source branch within the Linux kernel source
276 207repository to be used to build the Linux kernel. It is used as the default value
277 <para> 208for KBRANCH which may define an alternate branch, typically with a machine
278 Following are a few examples that show how to use Git commands to examine changes. 209override, such as:
279 Because Git repositories in the Yocto Project do not break existing Git 210
280 functionality, and because there exists many permutations of these types of 211KBRANCH_fri2 = "standard/fri2"
281 Git commands, many methods exist by which you can discover changes. 212
282 <note> 213Unless you specify otherwise, KBRANCH_DEFAULT is initialized to "master".
283 In the following examples, unless you provide a commit range, 214
284 <filename>kernel.org</filename> history is blended with Yocto Project 215LINUX_KERNEL_TYPE defines the kernel type to be used in assembling the
285 kernel changes. 216configuration and defaults to "standard" if you do not specify otherwise.
286 You can form ranges by using branch names from the kernel tree as the 217Together with KMACHINE, this defines the search arguments used by the Yocto
287 upper and lower commit markers with the Git commands. 218Project Linux kernel tools to find the appropriate description within the
288 You can see the branch names through the web interface to the 219metadata with which to build out the sources and configuration. The linux-yocto
289 Yocto Project source repositories at 220recipes define "standard", "tiny", and "preempt-rt" kernel types. See 3.3.4 for
290 <ulink url='http://git.yoctoproject.org/cgit.cgi'></ulink>. 221more inforation on kernel types.
291 For example, the branch names for the <filename>linux-yocto-3.4</filename> 222
292 kernel repository can be seen at 223During the build, the kern-tools will search for the BSP description file that
293 <ulink url='http://git.yoctoproject.org/cgit.cgi/linux-yocto-3.4/refs/heads'></ulink>. 224most closely matches the KMACHINE and LINUX_KERNEL_TYPE passed in from the
294 </note> 225recipe. It will use the first BSP description it finds matching both variables.
295 To see a full range of the changes, use the 226Failing that it will issue a warning such as the following:
296 <filename>git whatchanged</filename> command and specify a commit range 227
297 for the branch (<filename>&lt;commit&gt;..&lt;commit&gt;</filename>). 228 WARNING: Can't find any BSP hardware or required configuration fragments.
298 </para> 229 WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and
299 230 meta/cfg/broken/fri2-broken/required_frags.txt in directory:
300 <para> 231 meta/cfg/broken/fri2-broken
301 Here is an example that looks at what has changed in the 232
302 <filename>emenlow</filename> branch of the 233 In this example KMACHINE was set to "fri2-broken" and LINUX_KERNEL_TYPE
303 <filename>linux-yocto-3.4</filename> kernel. 234 was set to "broken".
304 The lower commit range is the commit associated with the 235
305 <filename>standard/base</filename> branch, while 236It will then search first for the KMACHINE and then
306 the upper commit range is the commit associated with the 237for the LINUX_KERNEL_TYPE. If it cannot find a partial match, it will use the
307 <filename>standard/emenlow</filename> branch. 238sources from the KBRANCH and any configuration specified in the SRC_URI.
308 <literallayout class='monospaced'> 239
309 $ git whatchanged origin/standard/base..origin/standard/emenlow 240KERNEL_FEATURES can be used to include features (configuration fragments,
310 </literallayout> 241patches, or both) that are not already included by the KMACHINE and
311 </para> 242LINUX_KERNEL_TYPE combination. To include a feature specified as
312 243"features/netfilter.scc" for example, specify:
313 <para> 244
314 To see a summary of changes use the <filename>git log</filename> command. 245KERNEL_FEATURES += "features/netfilter.scc"
315 Here is an example using the same branches: 246
316 <literallayout class='monospaced'> 247To include a feature called "cfg/sound.scc" just for the qemux86 machine,
317 $ git log --oneline origin/standard/base..origin/standard/emenlow 248specify:
318 </literallayout> 249
319 The <filename>git log</filename> output might be more useful than 250KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc"
320 the <filename>git whatchanged</filename> as you get 251
321 a short, one-line summary of each change and not the entire commit. 252The value of the entries in KERNEL_FEATURES are dependent on their location
322 </para> 253within the metadata itself. The examples here are taken from the
323 254linux-yocto-3.4 repository where "features" and "cfg" are subdirectories of the
324 <para> 255metadata directory. For details, see 3.3.
325 If you want to see code differences associated with all the changes, use 256
326 the <filename>git diff</filename> command. 257 Note: The processing of the these variables has evolved some between the
327 Here is an example: 258 0.9 and 1.3 releases of the Yocto Project and associated
328 <literallayout class='monospaced'> 259 kern-tools sources. The above is accurate for 1.3 and later
329 $ git diff origin/standard/base..origin/standard/emenlow 260 releases of the Yocto Project.
330 </literallayout> 261 </literallayout>
331 </para> 262 </para>
332 263</section>
333 <para> 264
334 You can see the commit log messages and the text differences using the
335 <filename>git show</filename> command:
336 Here is an example:
337 <literallayout class='monospaced'>
338 $ git show origin/standard/base..origin/standard/emenlow
339 </literallayout>
340 </para>
341
342 <para>
343 You can create individual patches for each change by using the
344 <filename>git format-patch</filename> command.
345 Here is an example that that creates patch files for each commit and
346 places them in your <filename>Documents</filename> directory:
347 <literallayout class='monospaced'>
348 $ git format-patch -o $HOME/Documents origin/standard/base..origin/standard/emenlow
349 </literallayout>
350 </para>
351 </section>
352
353 <section id='show-a-particular-feature-or-branch-change'>
354 <title>Show a Particular Feature or Branch Change</title>
355
356 <para>
357 Developers use tags in the Yocto Project kernel tree to divide changes for significant
358 features or branches.
359 Once you know a particular tag, you can use Git commands
360 to show changes associated with the tag and find the branches that contain
361 the feature.
362 <note>
363 Because BSP branch, <filename>kernel.org</filename>, and feature tags are all
364 present, there could be many tags.
365 </note>
366 The <filename>git show &lt;tag&gt;</filename> command shows changes that are tagged by
367 a feature.
368 Here is an example that shows changes tagged by the <filename>systemtap</filename>
369 feature:
370 <literallayout class='monospaced'>
371 $ git show systemtap
372 </literallayout>
373 You can use the <filename>git branch --contains &lt;tag&gt;</filename> command
374 to show the branches that contain a particular feature.
375 This command shows the branches that contain the <filename>systemtap</filename>
376 feature:
377 <literallayout class='monospaced'>
378 $ git branch --contains systemtap
379 </literallayout>
380 </para>
381
382 <para>
383 You can use many other comparisons to isolate BSP and kernel changes.
384 For example, you can compare against <filename>kernel.org</filename> tags
385 such as the <filename>v3.4</filename> tag.
386 </para>
387 </section>
388 </section>
389
390 <section id='development-saving-kernel-modifications'>
391 <title>Development: Saving Kernel Modifications</title>
392
393 <para>
394 Another common operation is to build a BSP supplied by the Yocto Project, make some
395 changes, rebuild, and then test.
396 Those local changes often need to be exported, shared or otherwise maintained.
397 </para>
398
399 <para>
400 Since the Yocto Project kernel source tree is backed by Git, this activity is
401 much easier as compared to with previous releases.
402 Because Git tracks file modifications, additions and deletions, it is easy
403 to modify the code and later realize that you need to save the changes.
404 It is also easy to determine what has changed.
405 This method also provides many tools to commit, undo and export those modifications.
406 </para>
407
408 <para>
409 This section and its sub-sections, describe general application of Git's
410 <filename>push</filename> and <filename>pull</filename> commands, which are used to
411 get your changes upstream or source your code from an upstream repository.
412 The Yocto Project provides scripts that help you work in a collaborative development
413 environment.
414 For information on these scripts, see the
415 "<ulink url='&YOCTO_DOCS_DEV_URL;#pushing-a-change-upstream'>Using Scripts to Push a Change
416 Upstream and Request a Pull</ulink>" and
417 "<ulink url='&YOCTO_DOCS_DEV_URL;#submitting-a-patch'>Using Email to Submit a Patch</ulink>"
418 sections in the Yocto Project Development Manual.
419 </para>
420
421 <para>
422 There are many ways to save kernel modifications.
423 The technique employed
424 depends on the destination for the patches:
425
426 <itemizedlist>
427 <listitem><para>Bulk storage</para></listitem>
428 <listitem><para>Internal sharing either through patches or by using Git</para></listitem>
429 <listitem><para>External submissions</para></listitem>
430 <listitem><para>Exporting for integration into another Source Code
431 Manager (SCM)</para></listitem>
432 </itemizedlist>
433 </para>
434
435 <para>
436 Because of the following list of issues, the destination of the patches also influences
437 the method for gathering them:
438
439 <itemizedlist>
440 <listitem><para>Bisectability</para></listitem>
441 <listitem><para>Commit headers</para></listitem>
442 <listitem><para>Division of subsystems for separate submission or review</para></listitem>
443 </itemizedlist>
444 </para>
445
446 <section id='bulk-export'>
447 <title>Bulk Export</title>
448
449 <para>
450 This section describes how you can "bulk" export changes that have not
451 been separated or divided.
452 This situation works well when you are simply storing patches outside of the kernel
453 source repository, either permanently or temporarily, and you are not committing
454 incremental changes during development.
455 <note>
456 This technique is not appropriate for full integration of upstream submission
457 because changes are not properly divided and do not provide an avenue for per-change
458 commit messages.
459 Therefore, this example assumes that changes have not been committed incrementally
460 during development and that you simply must gather and export them.
461 </note>
462 <literallayout class='monospaced'>
463 # bulk export of ALL modifications without separation or division
464 # of the changes
465
466 $ git add .
467 $ git commit -s -a -m &lt;msg&gt;
468 or
469 $ git commit -s -a # and interact with $EDITOR
470 </literallayout>
471 </para>
472
473 <para>
474 The previous operations capture all the local changes in the project source
475 tree in a single Git commit.
476 And, that commit is also stored in the project's source tree.
477 </para>
478
479 <para>
480 Once the changes are exported, you can restore them manually using a template
481 or through integration with the <filename>default_kernel</filename>.
482 </para>
483
484 </section>
485
486 <section id='incremental-planned-sharing'>
487 <title>Incremental/Planned Sharing</title>
488
489 <para>
490 This section describes how to save modifications when you are making incremental
491 commits or practicing planned sharing.
492 The examples in this section assume that you have incrementally committed
493 changes to the tree during development and now need to export them.
494 The sections that follow
495 describe how you can export your changes internally through either patches or by
496 using Git commands.
497 </para>
498
499 <para>
500 During development, the following commands are of interest.
501 For full Git documentation, refer to the Git documentation at
502 <ulink url='http://github.com'></ulink>.
503
504 <literallayout class='monospaced'>
505 # edit a file
506 $ vi &lt;path&gt;/file
507 # stage the change
508 $ git add &lt;path&gt;/file
509 # commit the change
510 $ git commit -s
511 # remove a file
512 $ git rm &lt;path&gt;/file
513 # commit the change
514 $ git commit -s
515
516 ... etc.
517 </literallayout>
518 </para>
519
520 <para>
521 Distributed development with Git is possible when you use a universally
522 agreed-upon unique commit identifier (set by the creator of the commit) that maps to a
523 specific change set with a specific parent.
524 This identifier is created for you when
525 you create a commit, and is re-created when you amend, alter or re-apply
526 a commit.
527 As an individual in isolation, this is of no interest.
528 However, if you
529 intend to share your tree with normal Git <filename>push</filename> and
530 <filename>pull</filename> operations for
531 distributed development, you should consider the ramifications of changing a
532 commit that you have already shared with others.
533 </para>
534
535 <para>
536 Assuming that the changes have not been pushed upstream, or pulled into
537 another repository, you can update both the commit content and commit messages
538 associated with development by using the following commands:
539
540 <literallayout class='monospaced'>
541 $ Git add &lt;path&gt;/file
542 $ Git commit --amend
543 $ Git rebase or Git rebase -i
544 </literallayout>
545 </para>
546
547 <para>
548 Again, assuming that the changes have not been pushed upstream, and that
549 no pending works-in-progress exist (use <filename>git status</filename> to check), then
550 you can revert (undo) commits by using the following commands:
551
552 <literallayout class='monospaced'>
553 # remove the commit, update working tree and remove all
554 # traces of the change
555 $ git reset --hard HEAD^
556 # remove the commit, but leave the files changed and staged for re-commit
557 $ git reset --soft HEAD^
558 # remove the commit, leave file change, but not staged for commit
559 $ git reset --mixed HEAD^
560 </literallayout>
561 </para>
562
563 <para>
564 You can create branches, "cherry-pick" changes, or perform any number of Git
565 operations until the commits are in good order for pushing upstream
566 or for pull requests.
567 After a <filename>push</filename> or <filename>pull</filename> command,
568 commits are normally considered
569 "permanent" and you should not modify them.
570 If the commits need to be changed, you can incrementally do so with new commits.
571 These practices follow standard Git workflow and the <filename>kernel.org</filename> best
572 practices, which is recommended.
573 <note>
574 It is recommended to tag or branch before adding changes to a Yocto Project
575 BSP or before creating a new one.
576 The reason for this recommendation is because the branch or tag provides a
577 reference point to facilitate locating and exporting local changes.
578 </note>
579 </para>
580
581 <section id='export-internally-via-patches'>
582 <title>Exporting Changes Internally by Using Patches</title>
583
584 <para>
585 This section describes how you can extract committed changes from a working directory
586 by exporting them as patches.
587 Once the changes have been extracted, you can use the patches for upstream submission,
588 place them in a Yocto Project template for automatic kernel patching,
589 or apply them in many other common uses.
590 </para>
591
592 <para>
593 This example shows how to create a directory with sequentially numbered patches.
594 Once the directory is created, you can apply it to a repository using the
595 <filename>git am</filename> command to reproduce the original commit and all
596 the related information such as author, date, commit log, and so forth.
597 <note>
598 The new commit identifiers (ID) will be generated upon re-application.
599 This action reflects that the commit is now applied to an underlying commit
600 with a different ID.
601 </note>
602 <literallayout class='monospaced'>
603 # &lt;first-commit&gt; can be a tag if one was created before development
604 # began. It can also be the parent branch if a branch was created
605 # before development began.
606
607 $ git format-patch -o &lt;dir&gt; &lt;first commit&gt;..&lt;last commit&gt;
608 </literallayout>
609 </para>
610
611 <para>
612 In other words:
613 <literallayout class='monospaced'>
614 # Identify commits of interest.
615
616 # If the tree was tagged before development
617 $ git format-patch -o &lt;save dir&gt; &lt;tag&gt;
618
619 # If no tags are available
620 $ git format-patch -o &lt;save dir&gt; HEAD^ # last commit
621 $ git format-patch -o &lt;save dir&gt; HEAD^^ # last 2 commits
622 $ git whatchanged # identify last commit
623 $ git format-patch -o &lt;save dir&gt; &lt;commit id&gt;
624 $ git format-patch -o &lt;save dir&gt; &lt;rev-list&gt;
625 </literallayout>
626 </para>
627 </section>
628
629 <section id='export-internally-via-git'>
630 <title>Exporting Changes Internally by Using Git</title>
631
632 <para>
633 This section describes how you can export changes from a working directory
634 by pushing the changes into a master repository or by making a pull request.
635 Once you have pushed the changes to the master repository, you can then
636 pull those same changes into a new kernel build at a later time.
637 </para>
638
639 <para>
640 Use this command form to push the changes:
641 <literallayout class='monospaced'>
642 $ git push ssh://&lt;master_server&gt;/&lt;path_to_repo&gt;
643 &lt;local_branch&gt;:&lt;remote_branch&gt;
644 </literallayout>
645 </para>
646
647 <para>
648 For example, the following command pushes the changes from your local branch
649 <filename>yocto/standard/common-pc/base</filename> to the remote branch with the same name
650 in the master repository <filename>//git.mycompany.com/pub/git/kernel-3.4</filename>.
651 <literallayout class='monospaced'>
652 $ git push ssh://git.mycompany.com/pub/git/kernel-3.4 \
653 yocto/standard/common-pc/base:yocto/standard/common-pc/base
654 </literallayout>
655 </para>
656
657 <para>
658 A pull request entails using the <filename>git request-pull</filename> command to compose
659 an email to the
660 maintainer requesting that a branch be pulled into the master repository, see
661 <ulink url='http://github.com/guides/pull-requests'></ulink> for an example.
662 <note>
663 Other commands such as <filename>git stash</filename> or branching can also be used to save
664 changes, but are not covered in this document.
665 </note>
666 </para>
667 </section>
668 </section>
669
670 <section id='export-for-external-upstream-submission'>
671 <title>Exporting Changes for External (Upstream) Submission</title>
672
673 <para>
674 This section describes how to export changes for external upstream submission.
675 If the patch series is large or the maintainer prefers to pull
676 changes, you can submit these changes by using a pull request.
677 However, it is common to send patches as an email series.
678 This method allows easy review and integration of the changes.
679 <note>
680 Before sending patches for review be sure you understand the
681 community standards for submitting and documenting changes and follow their best practices.
682 For example, kernel patches should follow standards such as:
683 <itemizedlist>
684 <listitem><para>
685 <ulink url='http://linux.yyz.us/patch-format.html'></ulink></para></listitem>
686 <listitem><para>Documentation/SubmittingPatches (in any linux
687 kernel source tree)</para></listitem>
688 </itemizedlist>
689 </note>
690 </para>
691
692 <para>
693 The messages used to commit changes are a large part of these standards.
694 Consequently, be sure that the headers for each commit have the required information.
695 For information on how to follow the Yocto Project commit message standards, see the
696 "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>How to Submit a
697 Change</ulink>" section in the Yocto Project Development Manual.
698 </para>
699
700 <para>
701 If the initial commits were not properly documented or do not meet those standards,
702 you can re-base by using the <filename>git rebase -i</filename> command to
703 manipulate the commits and
704 get them into the required format.
705 Other techniques such as branching and cherry-picking commits are also viable options.
706 </para>
707
708 <para>
709 Once you complete the commits, you can generate the email that sends the patches
710 to the maintainer(s) or lists that review and integrate changes.
711 The command <filename>git send-email</filename> is commonly used to ensure
712 that patches are properly
713 formatted for easy application and avoid mailer-induced patch damage.
714 </para>
715
716 <para>
717 The following is an example of dumping patches for external submission:
718 <literallayout class='monospaced'>
719 # dump the last 4 commits
720 $ git format-patch --thread -n -o ~/rr/ HEAD^^^^
721 $ git send-email --compose --subject '[RFC 0/N] &lt;patch series summary&gt;' \
722 --to foo@yoctoproject.org --to bar@yoctoproject.org \
723 --cc list@yoctoproject.org ~/rr
724 # the editor is invoked for the 0/N patch, and when complete the entire
725 # series is sent via email for review
726 </literallayout>
727 </para>
728 </section>
729
730 <section id='export-for-import-into-other-scm'>
731 <title>Exporting Changes for Import into Another SCM</title>
732
733 <para>
734 When you want to export changes for import into another
735 Source Code Manager (SCM), you can use any of the previously discussed
736 techniques.
737 However, if the patches are manually applied to a secondary tree and then
738 that tree is checked into the SCM, you can lose change information such as
739 commit logs.
740 This process is not recommended.
741 </para>
742
743 <para>
744 Many SCMs can directly import Git commits, or can translate Git patches so that
745 information is not lost.
746 Those facilities are SCM-dependent and you should use them whenever possible.
747 </para>
748 </section>
749 </section>
750
751 <section id='scm-working-with-the-yocto-project-kernel-in-another-scm'>
752 <title>Working with the Yocto Project Kernel in Another SCM</title>
753
754 <para>
755 This section describes kernel development in an SCM other than Git,
756 which is not the same as exporting changes to another SCM described earlier.
757 For this scenario, you use the OpenEmbedded build system to
758 develop the kernel in a different SCM.
759 The following must be true for you to accomplish this:
760 <itemizedlist>
761 <listitem><para>The delivered Yocto Project kernel must be exported into the second
762 SCM.</para></listitem>
763 <listitem><para>Development must be exported from that secondary SCM into a
764 format that can be used by the OpenEmbedded build system.</para></listitem>
765 </itemizedlist>
766 </para>
767
768 <section id='exporting-delivered-kernel-to-scm'>
769 <title>Exporting the Delivered Kernel to the SCM</title>
770
771 <para>
772 Depending on the SCM, it might be possible to export the entire Yocto Project
773 kernel Git repository, branches and all, into a new environment.
774 This method is preferred because it has the most flexibility and potential to maintain
775 the meta data associated with each commit.
776 </para>
777
778 <para>
779 When a direct import mechanism is not available, it is still possible to
780 export a branch (or series of branches) and check them into a new repository.
781 </para>
782
783 <para>
784 The following commands illustrate some of the steps you could use to
785 import the <filename>yocto/standard/common-pc/base</filename>
786 kernel into a secondary SCM:
787 <literallayout class='monospaced'>
788 $ git checkout yocto/standard/common-pc/base
789 $ cd .. ; echo linux/.git &gt; .cvsignore
790 $ cvs import -m "initial import" linux MY_COMPANY start
791 </literallayout>
792 </para>
793
794 <para>
795 You could now relocate the CVS repository and use it in a centralized manner.
796 </para>
797
798 <para>
799 The following commands illustrate how you can condense and merge two BSPs into a
800 second SCM:
801 <literallayout class='monospaced'>
802 $ git checkout yocto/standard/common-pc/base
803 $ git merge yocto/standard/common-pc-64/base
804 # resolve any conflicts and commit them
805 $ cd .. ; echo linux/.git &gt; .cvsignore
806 $ cvs import -m "initial import" linux MY_COMPANY start
807 </literallayout>
808 </para>
809 </section>
810
811 <section id='importing-changes-for-build'>
812 <title>Importing Changes for the Build</title>
813
814 <para>
815 Once development has reached a suitable point in the second development
816 environment, you need to export the changes as patches.
817 To export them, place the changes in a recipe and
818 automatically apply them to the kernel during patching.
819 </para>
820 </section>
821 </section>
822
823 <section id='bsp-creating'>
824 <title>Creating a BSP Based on an Existing Similar BSP</title>
825
826 <para>
827 This section overviews the process of creating a BSP based on an
828 existing similar BSP.
829 The information is introductory in nature and does not provide step-by-step examples.
830 For detailed information on how to create a new BSP, see
831 the "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>" section in the
832 Yocto Project Board Support Package (BSP) Developer's Guide, or see the
833 <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_creating_one_generic_Atom_BSP_from_another'>Transcript:_creating_one_generic_Atom_BSP_from_another</ulink>
834 wiki page.
835 </para>
836
837 <para>
838 The basic steps you need to follow are:
839 <orderedlist>
840 <listitem><para><emphasis>Make sure you have set up a local Source Directory:</emphasis>
841 You must create a local
842 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>
843 by either creating a Git repository (recommended) or
844 extracting a Yocto Project release tarball.</para></listitem>
845 <listitem><para><emphasis>Choose an existing BSP available with the Yocto Project:</emphasis>
846 Try to map your board features as closely to the features of a BSP that is
847 already supported and exists in the Yocto Project.
848 Starting with something as close as possible to your board makes developing
849 your BSP easier.
850 You can find all the BSPs that are supported and ship with the Yocto Project
851 on the Yocto Project's Download page at
852 <ulink url='&YOCTO_HOME_URL;/download'></ulink>.</para></listitem>
853 <listitem><para><emphasis>Be sure you have the Base BSP:</emphasis>
854 You need to either have a local Git repository of the base BSP set up or
855 have downloaded and extracted the files from a release BSP tarball.
856 Either method gives you access to the BSP source files.</para></listitem>
857 <listitem><para><emphasis>Make a copy of the existing BSP, thus isolating your new
858 BSP work:</emphasis>
859 Copying the existing BSP file structure gives you a new area in which to work.</para></listitem>
860 <listitem><para><emphasis>Make configuration and recipe changes to your new BSP:</emphasis>
861 Configuration changes involve the files in the BSP's <filename>conf</filename>
862 directory.
863 Changes include creating a machine-specific configuration file and editing the
864 <filename>layer.conf</filename> file.
865 The configuration changes identify the kernel you will be using.
866 Recipe changes include removing, modifying, or adding new recipe files that
867 instruct the build process on what features to include in the image.</para></listitem>
868 <listitem><para><emphasis>Prepare for the build:</emphasis>
869 Before you actually initiate the build, you need to set up the build environment
870 by sourcing the environment initialization script.
871 After setting up the environment, you need to make some build configuration
872 changes to the <filename>local.conf</filename> and <filename>bblayers.conf</filename>
873 files.</para></listitem>
874 <listitem><para><emphasis>Build the image:</emphasis>
875 The OpenEmbedded build system uses BitBake to create the image.
876 You need to decide on the type of image you are going to build (e.g. minimal, base,
877 core, sato, and so forth) and then start the build using the <filename>bitbake</filename>
878 command.</para></listitem>
879 </orderedlist>
880 </para>
881 </section>
882
883 <section id='tip-dirty-string'>
884 <title>"-dirty" String</title>
885
886 <para>
887 If kernel images are being built with "-dirty" on the end of the version
888 string, this simply means that modifications in the source
889 directory have not been committed.
890 <literallayout class='monospaced'>
891 $ git status
892 </literallayout>
893 </para>
894
895 <para>
896 You can use the above Git command to report modified, removed, or added files.
897 You should commit those changes to the tree regardless of whether they will be saved,
898 exported, or used.
899 Once you commit the changes you need to rebuild the kernel.
900 </para>
901
902 <para>
903 To brute force pickup and commit all such pending changes, enter the following:
904 <literallayout class='monospaced'>
905 $ git add .
906 $ git commit -s -a -m "getting rid of -dirty"
907 </literallayout>
908 </para>
909
910 <para>
911 Next, rebuild the kernel.
912 </para>
913 </section>
914 </section>
915</chapter> 265</chapter>
916<!-- 266<!--
917vim: expandtab tw=80 ts=4 267vim: expandtab tw=80 ts=4
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-14 17:20:15 +0000