summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2013-01-15 16:29:17 -0800
committerRichard Purdie <richard.purdie@linuxfoundation.org>2013-01-27 13:54:09 +0000
commit89dbdec9929b744303f8653b3c19450a2f7d6c4f (patch)
tree624f3e6534df209d03f83c2674704769c4f64c95
parentcdacd8764b26b945253c7a94941806ce34a44da6 (diff)
downloadpoky-89dbdec9929b744303f8653b3c19450a2f7d6c4f.tar.gz
profile-manual: Adding raw text.
Started to put in the usage chapter. Adding text and sectioning off by paragraph and formatting for literal sections. (From yocto-docs rev: c2d328d19023cd6fac6dcaeb8449e2203b5a17b7) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat
-rw-r--r--documentation/profile-manual/profile-manual-usage.xml1950
1 files changed, 862 insertions, 1088 deletions
diff --git a/documentation/profile-manual/profile-manual-usage.xml b/documentation/profile-manual/profile-manual-usage.xml
index 65e17e24a0..71feb418fd 100644
--- a/documentation/profile-manual/profile-manual-usage.xml
+++ b/documentation/profile-manual/profile-manual-usage.xml
@@ -2,1215 +2,989 @@
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4 4
5<chapter id='dev-manual-newbie'> 5<chapter id='profile-manual-usage'>
6 6
7<title>The Yocto Project Open Source Development Environment</title> 7<title>Basic Usage (with examples) for each of the Yocto Tracing Tools</title>
8 8
9<para> 9<para>
10 This chapter helps you understand the Yocto Project as an open source development project. 10 This chapter presents basic usage examples for each of the tracing
11 In general, working in an open source environment is very different from working in a 11 tools.
12 closed, proprietary environment.
13 Additionally, the Yocto Project uses specific tools and constructs as part of its development
14 environment.
15 This chapter specifically addresses open source philosophy, licensing issues, code repositories,
16 the open source distributed version control system Git, and best practices using the Yocto Project.
17</para> 12</para>
18 13
19<section id='open-source-philosophy'> 14<section id='profile-manual-perf'>
20 <title>Open Source Philosophy</title> 15 <title>perf</title>
21 16
22 <para> 17 <para>
23 Open source philosophy is characterized by software development directed by peer production 18 The 'perf' tool is the profiling and tracing tool that comes
24 and collaboration through an active community of developers. 19 bundled with the Linux kernel.
25 Contrast this to the more standard centralized development models used by commercial software
26 companies where a finite set of developers produces a product for sale using a defined set
27 of procedures that ultimately result in an end product whose architecture and source material
28 are closed to the public.
29 </para> 20 </para>
30 21
31 <para> 22 <para>
32 Open source projects conceptually have differing concurrent agendas, approaches, and production. 23 Don't let the fact that it's part of the kernel fool you into thinking
33 These facets of the development process can come from anyone in the public (community) that has a 24 that it's only for tracing and profiling the kernel - you can indeed
34 stake in the software project. 25 use it to trace and profile just the kernel , but you can also use it
35 The open source environment contains new copyright, licensing, domain, and consumer issues 26 to profile specific applications separately (with or without kernel
36 that differ from the more traditional development environment. 27 context), and you can also use it to trace and profile the kernel
37 In an open source environment, the end product, source material, and documentation are 28 and all applications on the system simultaneously to gain a system-wide
38 all available to the public at no cost. 29 view of what's going on.
39 </para> 30 </para>
40 31
41 <para> 32 <para>
42 A benchmark example of an open source project is the Linux Kernel, which was initially conceived 33 In many ways, it aims to be a superset of all the tracing and profiling
43 and created by Finnish computer science student Linus Torvalds in 1991. 34 tools available in Linux today, including all the other tools covered
44 Conversely, a good example of a non-open source project is the 35 in this HOWTO. The past couple of years have seen perf subsume a lot
45 <trademark class='registered'>Windows</trademark> family of operating 36 of the functionality of those other tools, and at the same time those
46 systems developed by <trademark class='registered'>Microsoft</trademark> Corporation. 37 other tools have removed large portions of their previous functionality
38 and replaced it with calls to the equivalent functionality now
39 implemented by the perf subsystem. Extrapolation suggests that at
40 some point those other tools will simply become completely redundant
41 and go away; until then, we'll cover those other tools in these pages
42 and in many cases show how the same things can be accomplished in
43 perf and the other tools when it seems useful to do so.
47 </para> 44 </para>
48 45
49 <para> 46 <para>
50 Wikipedia has a good historical description of the Open Source Philosophy 47 The coverage below details some of the most common ways you'll likely
51 <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>. 48 want to apply the tool; full documentation can be found either within
52 You can also find helpful information on how to participate in the Linux Community 49 the tool itself or in the man pages at
53 <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>. 50 <ulink url='http://linux.die.net/man/1/perf'>perf(1)</ulink>.
54 </para>
55</section>
56
57<section id="usingpoky-changes-collaborate">
58 <title>Using the Yocto Project in a Team Environment</title>
59
60 <para>
61 It might not be immediately clear how you can use the Yocto Project in a team environment,
62 or scale it for a large team of developers.
63 The specifics of any situation determine the best solution.
64 Granted that the Yocto Project offers immense flexibility regarding this, practices do exist
65 that experience has shown work well.
66 </para> 51 </para>
67 52
68 <para> 53 <section id='perf-setup'>
69 The core component of any development effort with the Yocto Project is often an 54 <title>Setup</title>
70 automated build and testing framework along with an image generation process.
71 You can use these core components to check that the metadata can be built,
72 highlight when commits break the build, and provide up-to-date images that
73 allow developers to test the end result and use it as a base platform for further
74 development.
75 Experience shows that buildbot is a good fit for this role.
76 What works well is to configure buildbot to make two types of builds:
77 incremental and full (from scratch).
78 See "<ulink url='http://autobuilder.yoctoproject.org:8010/'>Welcome to the buildbot for the Yocto Project</ulink>"
79 for an example implementation that uses buildbot.
80 </para>
81 55
82 <para> 56 <para>
83 You can tie an incremental build to a commit hook that triggers the build 57 For this section, we'll assume you've already performed the basic
84 each time a commit is made to the metadata. 58 setup outlined in the General Setup section.
85 This practice results in useful acid tests that determine whether a given commit 59 </para>
86 breaks the build in some serious way.
87 Associating a build to a commit can catch a lot of simple errors.
88 Furthermore, the tests are fast so developers can get quick feedback on changes.
89 </para>
90
91 <para>
92 Full builds build and test everything from the ground up.
93 These types of builds usually happen at predetermined times like during the
94 night when the machine load is low.
95 </para>
96
97 <para>
98 Most teams have many pieces of software undergoing active development at any given time.
99 You can derive large benefits by putting these pieces under the control of a source
100 control system that is compatible (i.e. Git or Subversion (SVN)) with the OpenEmbedded
101 build system that the Yocto Project uses.
102 You can then set the autobuilder to pull the latest revisions of the packages
103 and test the latest commits by the builds.
104 This practice quickly highlights issues.
105 The build system easily supports testing configurations that use both a
106 stable known good revision and a floating revision.
107 The build system can also take just the changes from specific source control branches.
108 This capability allows you to track and test specific changes.
109 </para>
110
111 <para>
112 Perhaps the hardest part of setting this up is defining the software project or
113 the metadata policies that surround the different source control systems.
114 Of course circumstances will be different in each case.
115 However, this situation reveals one of the Yocto Project's advantages -
116 the system itself does not
117 force any particular policy on users, unlike a lot of build systems.
118 The system allows the best policies to be chosen for the given circumstances.
119 </para>
120
121 <para>
122 In general, best practices exist that make your work with the Yocto
123 Project easier in a team environment.
124 This list presents some of these practices you might consider following.
125 Of course, you need to understand that you do not have to follow these
126 practices and your setup can be totally controlled and customized by
127 your team:
128 <itemizedlist>
129 <listitem><para>Use <link linkend='git'>Git</link>
130 as the source control system.</para></listitem>
131 <listitem><para>Maintain your metadata in layers that make sense
132 for your situation.
133 See the "<link linkend='understanding-and-creating-layers'>Understanding
134 and Creating Layers</link>" section for more information on
135 layers.</para></listitem>
136 <listitem><para>Separate the project's metadata and code by using
137 separate Git repositories.
138 See the "<link linkend='yocto-project-repositories'>Yocto Project
139 Source Repositories</link>" section for information on these
140 repositories.
141 See the "<link linkend='getting-setup'>Getting Set Up</link>" section
142 for information on how to set up various Yocto Project related
143 Git repositories.</para></listitem>
144 <listitem><para>Set up the directory for the shared state cache
145 (<ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>)
146 where they make sense.
147 For example, set up the sstate cache for developers using the
148 same office and share source directories on the developer's
149 machines.</para></listitem>
150 <listitem><para>Set up an autobuilder and have it populate the
151 sstate cache and source directories.</para></listitem>
152 </itemizedlist>
153 </para>
154</section>
155 60
156<section id='yocto-project-repositories'> 61 <para>
157 <title>Yocto Project Source Repositories</title> 62 In particular, you'll get the most mileage out of perf if you
63 profile an image built with INHIBIT_PACKAGE_STRIP = "1" in your
64 local.conf.
65 </para>
158 66
159 <para> 67 <para>
160 The Yocto Project team maintains complete source repositories for all Yocto Project files 68 perf runs on the target system for the most part. You can archive
161 at <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'></ulink>. 69 profile data and copy it to the host for analysis, but for the
162 This web-based source code browser is organized into categories by function such as 70 rest of this document we assume you've ssh'ed to the host and
163 IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and so forth. 71 will be running the perf commands on the target.
164 From the interface, you can click on any particular item in the "Name" column and 72 </para>
165 see the URL at the bottom of the page that you need to set up a Git repository for 73 </section>
166 that particular item.
167 Having a local Git repository of the Source Directory (poky) allows you to
168 make changes, contribute to the history, and ultimately enhance the Yocto Project's
169 tools, Board Support Packages, and so forth.
170 </para>
171 74
172 <para> 75 <section id='perf-basic-usage'>
173 Conversely, if you are a developer that is not interested in contributing back to the 76 <title>Basic Usage</title>
174 Yocto Project, you have the ability to simply download and extract release tarballs
175 and use them within the Yocto Project environment.
176 All that is required is a particular release of the Yocto Project and
177 your application source code.
178 </para>
179 77
180 <para> 78 <para>
181 For any supported release of Yocto Project, you can go to the Yocto Project website’s 79 The perf tool is pretty much self-documenting. To remind yourself
182 <ulink url='&YOCTO_HOME_URL;/download'>download page</ulink> and get a 80 of the available commands, simply type 'perf', which will show you
183 tarball of the release. 81 basic usage along with the available perf subcommands:
184 You can also go to this site to download any supported BSP tarballs. 82 <literallayout class='monospaced'>
185 Unpacking the tarball gives you a hierarchical Source Directory that lets you develop 83 root@crownbay:~# perf
186 using the Yocto Project. 84
85 usage: perf [--version] [--help] COMMAND [ARGS]
86
87 The most commonly used perf commands are:
88 annotate Read perf.data (created by perf record) and display annotated code
89 archive Create archive with object files with build-ids found in perf.data file
90 bench General framework for benchmark suites
91 buildid-cache Manage build-id cache.
92 buildid-list List the buildids in a perf.data file
93 diff Read two perf.data files and display the differential profile
94 evlist List the event names in a perf.data file
95 inject Filter to augment the events stream with additional information
96 kmem Tool to trace/measure kernel memory(slab) properties
97 kvm Tool to trace/measure kvm guest os
98 list List all symbolic event types
99 lock Analyze lock events
100 probe Define new dynamic tracepoints
101 record Run a command and record its profile into perf.data
102 report Read perf.data (created by perf record) and display the profile
103 sched Tool to trace/measure scheduler properties (latencies)
104 script Read perf.data (created by perf record) and display trace output
105 stat Run a command and gather performance counter statistics
106 test Runs sanity tests.
107 timechart Tool to visualize total system behavior during a workload
108 top System profiling tool.
109
110 See 'perf help COMMAND' for more information on a specific command.
111 </literallayout>
187 </para> 112 </para>
188 113
189 <para> 114 <section id='using-perf-to-do-basic-profiling'>
190 Once you are set up through either tarball extraction or a checkout of Git repositories, 115 <title>Using perf to do Basic Profiling</title>
191 you are ready to develop.
192 </para>
193 116
194 <para> 117 <para>
195 In summary, here is where you can get the project files needed for development: 118 As a simple test case, we'll profile the 'wget' of a fairly large
196 <itemizedlist> 119 file, which is a minimally interesting case because it has both
197 <listitem><para id='source-repositories'><emphasis><ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Source Repositories:</ulink></emphasis> 120 file and network I/O aspects, and at least in the case of standard
198 This area contains IDE Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and Yocto 121 Yocto images, it's implemented as part of busybox, so the methods
199 Metadata Layers. 122 we use to analyze it can be used in a very similar way to the whole
200 You can create local copies of Git repositories for each of these areas.</para> 123 host of supported busybox applets in Yocto.
201 <para> 124 <literallayout class='monospaced'>
202 <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" /> 125 root@crownbay:~# rm linux-2.6.19.2.tar.bz2; \
203 </para></listitem> 126 wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
204 <listitem><para><anchor id='index-downloads' /><emphasis><ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink></emphasis> 127 </literallayout>
205 This area contains index releases such as 128 The quickest and easiest way to get some basic overall data about
206 the <trademark class='trade'>Eclipse</trademark> 129 what's going on for a particular workload it to profile it using
207 Yocto Plug-in, miscellaneous support, poky, pseudo, installers for cross-development toolchains, 130 'perf stat'. 'perf stat' basically profiles using a few default
208 and all released versions of Yocto Project in the form of images or tarballs. 131 counters and displays the summed counts at the end of the run:
209 Downloading and extracting these files does not produce a local copy of the 132 <literallayout class='monospaced'>
210 Git repository but rather a snapshot of a particular release or image.</para> 133 root@crownbay:~# perf stat wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
211 <para> 134 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
212 <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="4in" /> 135 linux-2.6.19.2.tar.b 100% |***************************************************| 41727k 0:00:00 ETA
213 </para></listitem> 136
214 <listitem><para><emphasis><ulink url='&YOCTO_HOME_URL;/download'>Yocto Project Download Page</ulink></emphasis> 137 Performance counter stats for 'wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2':
215 This page on the Yocto Project website allows you to download any Yocto Project 138
216 release or Board Support Package (BSP) in tarball form. 139 4597.223902 task-clock # 0.077 CPUs utilized
217 The tarballs are similar to those found in the 140 23568 context-switches # 0.005 M/sec
218 <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> area.</para> 141 68 CPU-migrations # 0.015 K/sec
219 <para> 142 241 page-faults # 0.052 K/sec
220 <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" /> 143 3045817293 cycles # 0.663 GHz
221 </para></listitem> 144 &lt;not supported&gt; stalled-cycles-frontend
222 </itemizedlist> 145 &lt;not supported&gt; stalled-cycles-backend
223 </para> 146 858909167 instructions # 0.28 insns per cycle
224</section> 147 165441165 branches # 35.987 M/sec
148 19550329 branch-misses # 11.82% of all branches
149
150 59.836627620 seconds time elapsed
151 </literallayout>
152 Many times such a simple-minded test doesn't yield much of
153 interest, but sometimes it does (see Real-world Yocto bug
154 (slow loop-mounted write speed)).
155 </para>
225 156
226<section id='yocto-project-terms'> 157 <para>
227 <title>Yocto Project Terms</title> 158 Also, note that 'perf stat' isn't restricted to a fixed set of
159 counters - basically any event listed in the output of 'perf list'
160 can be tallied by 'perf stat'. For example, suppose we wanted to
161 see a summary of all the events related to kernel memory
162 allocation/freeing along with cache hits and misses:
163 <literallayout class='monospaced'>
164 root@crownbay:~# perf stat -e kmem:* -e cache-references -e cache-misses wget http:// downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
165 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
166 linux-2.6.19.2.tar.b 100% |***************************************************| 41727k 0:00:00 ETA
167
168 Performance counter stats for 'wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2':
169
170 5566 kmem:kmalloc
171 125517 kmem:kmem_cache_alloc
172 0 kmem:kmalloc_node
173 0 kmem:kmem_cache_alloc_node
174 34401 kmem:kfree
175 69920 kmem:kmem_cache_free
176 133 kmem:mm_page_free
177 41 kmem:mm_page_free_batched
178 11502 kmem:mm_page_alloc
179 11375 kmem:mm_page_alloc_zone_locked
180 0 kmem:mm_page_pcpu_drain
181 0 kmem:mm_page_alloc_extfrag
182 66848602 cache-references
183 2917740 cache-misses # 4.365 % of all cache refs
184
185 44.831023415 seconds time elapsed
186 </literallayout>
187 So 'perf stat' gives us a nice easy way to get a quick overview of
188 what might be happening for a set of events, but normally we'd
189 need a little more detail in order to understand what's going on
190 in a way that we can act on in a useful way.
191 </para>
228 192
229 <para> 193 <para>
230 Following is a list of terms and definitions users new to the Yocto Project development 194 To dive down into a next level of detail, we can use 'perf
231 environment might find helpful. 195 record'/'perf report' which will collect profiling data and
232 While some of these terms are universal, the list includes them just in case: 196 present it to use using an interactive text-based UI (or
233 <itemizedlist> 197 simply as text if we specify --stdio to 'perf report').
234 <listitem><para><emphasis>Append Files:</emphasis> Files that append build information to 198 </para>
235 a recipe file.
236 Append files are known as BitBake append files and <filename>.bbappend</filename> files.
237 The OpenEmbedded build system expects every append file to have a corresponding and
238 underlying recipe (<filename>.bb</filename>) file.
239 Furthermore, the append file and the underlying recipe must have the same root filename.
240 The filenames can differ only in the file type suffix used (e.g.
241 <filename>formfactor_0.0.bb</filename> and <filename>formfactor_0.0.bbappend</filename>).
242 </para>
243 <para>Information in append files overrides the information in the similarly-named recipe file.
244 For an example of an append file in use, see the
245 "<link linkend='using-bbappend-files'>Using .bbappend Files</link>" section.
246 </para></listitem>
247 <listitem><para id='bitbake-term'><emphasis>BitBake:</emphasis>
248 The task executor and scheduler used by
249 the OpenEmbedded build system to build images.
250 For more information on BitBake, see the BitBake documentation
251 in the <filename>bitbake/doc/manual</filename> directory of the
252 <link linkend='source-directory'>Source Directory</link>.</para></listitem>
253 <listitem>
254 <para id='build-directory'><emphasis>Build Directory:</emphasis>
255 This term refers to the area used by the OpenEmbedded build system for builds.
256 The area is created when you <filename>source</filename> the setup
257 environment script that is found in the Source Directory
258 (i.e. <filename>&OE_INIT_FILE;</filename>).
259 The <ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink>
260 variable points to the Build Directory.</para>
261
262 <para>You have a lot of flexibility when creating the Build Directory.
263 Following are some examples that show how to create the directory:
264 <itemizedlist>
265 <listitem><para>Create the Build Directory in your current working directory
266 and name it <filename>build</filename>.
267 This is the default behavior.
268 <literallayout class='monospaced'>
269 $ source &OE_INIT_PATH;
270 </literallayout></para></listitem>
271 <listitem><para>Provide a directory path and specifically name the build
272 directory.
273 This next example creates a Build Directory named <filename>YP-&POKYVERSION;</filename>
274 in your home directory within the directory <filename>mybuilds</filename>.
275 If <filename>mybuilds</filename> does not exist, the directory is created for you:
276 <literallayout class='monospaced'>
277 $ source &OE_INIT_PATH; $HOME/mybuilds/YP-&POKYVERSION;
278 </literallayout></para></listitem>
279 <listitem><para>Provide an existing directory to use as the Build Directory.
280 This example uses the existing <filename>mybuilds</filename> directory
281 as the Build Directory.
282 <literallayout class='monospaced'>
283 $ source &OE_INIT_PATH; $HOME/mybuilds/
284 </literallayout></para></listitem>
285 </itemizedlist>
286 </para></listitem>
287 <listitem><para><emphasis>Build System:</emphasis> In the context of the Yocto Project
288 this term refers to the OpenEmbedded build system used by the project.
289 This build system is based on the project known as "Poky."
290 For some historical information about Poky, see the
291 <link linkend='poky'>Poky</link> term further along in this section.
292 </para></listitem>
293 <listitem><para><emphasis>Classes:</emphasis> Files that provide for logic encapsulation
294 and inheritance allowing commonly used patterns to be defined once and easily used
295 in multiple recipes.
296 Class files end with the <filename>.bbclass</filename> filename extension.
297 </para></listitem>
298 <listitem><para><emphasis>Configuration File:</emphasis> Configuration information in various
299 <filename>.conf</filename> files provides global definitions of variables.
300 The <filename>conf/local.conf</filename> configuration file in the
301 <link linkend='build-directory'>Build Directory</link>
302 contains user-defined variables that affect each build.
303 The <filename>meta-yocto/conf/distro/poky.conf</filename> configuration file
304 defines Yocto ‘distro’ configuration
305 variables used only when building with this policy.
306 Machine configuration files, which
307 are located throughout the
308 <link linkend='source-directory'>Source Directory</link>, define
309 variables for specific hardware and are only used when building for that target
310 (e.g. the <filename>machine/beagleboard.conf</filename> configuration file defines
311 variables for the Texas Instruments ARM Cortex-A8 development board).
312 Configuration files end with a <filename>.conf</filename> filename extension.
313 </para></listitem>
314 <listitem><para><emphasis>Cross-Development Toolchain:</emphasis>
315 A collection of software development
316 tools and utilities that allow you to develop software for targeted architectures.
317 This toolchain contains cross-compilers, linkers, and debuggers that are specific to
318 an architecture.
319 You can use the OpenEmbedded build system to build a cross-development toolchain
320 installer that when run installs the toolchain that contains the development tools you
321 need to cross-compile and test your software.
322 The Yocto Project ships with images that contain installers for
323 toolchains for supported architectures as well.
324 Sometimes this toolchain is referred to as the meta-toolchain.</para></listitem>
325 <listitem><para><emphasis>Image:</emphasis> An image is the result produced when
326 BitBake processes a given collection of recipes and related metadata.
327 Images are the binary output that run on specific hardware or QEMU
328 and for specific use cases.
329 For a list of the supported image types that the Yocto Project provides, see the
330 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
331 chapter in the Yocto Project Reference Manual.</para></listitem>
332 <listitem><para id='layer'><emphasis>Layer:</emphasis> A collection of recipes representing the core,
333 a BSP, or an application stack.
334 For a discussion on BSP Layers, see the
335 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
336 section in the Yocto Project Board Support Packages (BSP) Developer's Guide.</para></listitem>
337 <listitem><para id='metadata'><emphasis>Metadata:</emphasis> The files that BitBake parses when
338 building an image.
339 Metadata includes recipes, classes, and configuration files.</para></listitem>
340 <listitem><para id='oe-core'><emphasis>OE-Core:</emphasis> A core set of metadata originating
341 with OpenEmbedded (OE) that is shared between OE and the Yocto Project.
342 This metadata is found in the <filename>meta</filename> directory of the source
343 directory.</para></listitem>
344 <listitem><para><emphasis>Package:</emphasis> In the context of the Yocto Project,
345 this term refers to the packaged output from a baked recipe.
346 A package is generally the compiled binaries produced from the recipe's sources.
347 You ‘bake’ something by running it through BitBake.</para>
348 <para>It is worth noting that the term "package" can, in general, have subtle
349 meanings. For example, the packages refered to in the
350 "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Packages</ulink>" section are
351 compiled binaries that when installed add functionality to your Linux
352 distribution.</para>
353 <para>Another point worth noting is that historically within the Yocto Project,
354 recipes were referred to as packages - thus, the existence of several BitBake
355 variables that are seemingly mis-named,
356 (e.g. <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>,
357 <ulink url='&YOCTO_DOCS_REF_URL;#var-PRINC'><filename>PRINC</filename></ulink>,
358 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and
359 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>).
360 </para></listitem>
361 <listitem><para id='poky'><emphasis>Poky:</emphasis> The term "poky" can mean several things.
362 In its most general sense, it is an open-source project that was initially developed
363 by OpenedHand. With OpenedHand, poky was developed off of the existing OpenEmbedded
364 build system becoming a build system for embedded images.
365 After Intel Corporation acquired OpenedHand, the project poky became the basis for
366 the Yocto Project's build system.
367 Within the Yocto Project source repositories, poky exists as a separate Git repository
368 that can be cloned to yield a local copy on the host system.
369 Thus, "poky" can refer to the local copy of the Source Directory used to develop within
370 the Yocto Project.</para></listitem>
371 <listitem><para><emphasis>Recipe:</emphasis> A set of instructions for building packages.
372 A recipe describes where you get source code and which patches to apply.
373 Recipes describe dependencies for libraries or for other recipes, and they
374 also contain configuration and compilation options.
375 Recipes contain the logical unit of execution, the software/images to build, and
376 use the <filename>.bb</filename> file extension.</para></listitem>
377 <listitem>
378 <para id='source-directory'><emphasis>Source Directory:</emphasis>
379 This term refers to the directory structure created as a result of either downloading
380 and unpacking a Yocto Project release tarball or creating a local copy of
381 the <filename>poky</filename> Git repository
382 <filename>git://git.yoctoproject.org/poky</filename>.
383 Sometimes you might hear the term "poky directory" used to refer to this
384 directory structure.
385 <note>
386 The OpenEmbedded build system does not support file or directory names that
387 contain spaces.
388 Be sure that the Source Directory you use does not contain these types
389 of names.
390 </note></para>
391 <para>The Source Directory contains BitBake, Documentation, metadata and
392 other files that all support the Yocto Project.
393 Consequently, you must have the Source Directory in place on your development
394 system in order to do any development using the Yocto Project.</para>
395
396 <para>For tarball expansion, the name of the top-level directory of the Source Directory
397 is derived from the Yocto Project release tarball.
398 For example, downloading and unpacking <filename>&YOCTO_POKY_TARBALL;</filename>
399 results in a Source Directory whose top-level folder is named
400 <filename>&YOCTO_POKY;</filename>.
401 If you create a local copy of the Git repository, then you can name the repository
402 anything you like.
403 Throughout much of the documentation, <filename>poky</filename> is used as the name of
404 the top-level folder of the local copy of the poky Git repository.
405 So, for example, cloning the <filename>poky</filename> Git repository results in a
406 local Git repository whose top-level folder is also named <filename>poky</filename>.</para>
407
408 <para>It is important to understand the differences between the Source Directory created
409 by unpacking a released tarball as compared to cloning
410 <filename>git://git.yoctoproject.org/poky</filename>.
411 When you unpack a tarball, you have an exact copy of the files based on the time of
412 release - a fixed release point.
413 Any changes you make to your local files in the Source Directory are on top of the release.
414 On the other hand, when you clone the <filename>poky</filename> Git repository, you have an
415 active development repository.
416 In this case, any local changes you make to the Source Directory can be later applied
417 to active development branches of the upstream <filename>poky</filename> Git
418 repository.</para>
419
420 <para>Finally, if you want to track a set of local changes while starting from the same point
421 as a release tarball, you can create a local Git branch that
422 reflects the exact copy of the files at the time of their release.
423 You do this by using Git tags that are part of the repository.</para>
424
425 <para>For more information on concepts related to Git repositories, branches, and tags,
426 see the
427 "<link linkend='repositories-tags-and-branches'>Repositories, Tags, and Branches</link>"
428 section.</para></listitem>
429 <listitem><para><emphasis>Tasks:</emphasis> Arbitrary groups of software Recipes.
430 You simply use Tasks to hold recipes that, when built, usually accomplish a single task.
431 For example, a task could contain the recipes for a company’s proprietary or value-add software.
432 Or, the task could contain the recipes that enable graphics.
433 A task is really just another recipe.
434 Because task files are recipes, they end with the <filename>.bb</filename> filename
435 extension.</para></listitem>
436 <listitem><para><emphasis>Upstream:</emphasis> A reference to source code or repositories
437 that are not local to the development system but located in a master area that is controlled
438 by the maintainer of the source code.
439 For example, in order for a developer to work on a particular piece of code, they need to
440 first get a copy of it from an "upstream" source.</para></listitem>
441 </itemizedlist>
442 </para>
443</section>
444 199
445<section id='licensing'> 200 <para>
446 <title>Licensing</title> 201 As our first attempt at profiling this workload, we'll simply
202 run 'perf record', handing it the workload we want to profile
203 (everything after 'perf record' and any perf options we hand
204 it - here none - will be executedin a new shell). perf collects
205 samples until the process exits and records them in a file named
206 'perf.data' in the current working directory.
207 <literallayout class='monospaced'>
208 root@crownbay:~# perf record wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
447 209
448 <para> 210 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
449 Because open source projects are open to the public, they have different licensing structures in place. 211 linux-2.6.19.2.tar.b 100% |************************************************| 41727k 0:00:00 ETA
450 License evolution for both Open Source and Free Software has an interesting history. 212 [ perf record: Woken up 1 times to write data ]
451 If you are interested in this history, you can find basic information here: 213 [ perf record: Captured and wrote 0.176 MB perf.data (~7700 samples) ]
452 <itemizedlist> 214 </literallayout>
453 <listitem><para><ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink> 215 To see the results in a 'text-based UI' (tui), simply run
454 </para></listitem> 216 'perf report', which will read the perf.data file in the current
455 <listitem><para><ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license 217 working directory and display the results in an interactive UI:
456 history</ulink></para></listitem> 218 <literallayout class='monospaced'>
457 </itemizedlist> 219 root@crownbay:~# perf report
458 </para> 220 </literallayout>
221 </para>
459 222
460 <para> 223 <para>
461 In general, the Yocto Project is broadly licensed under the Massachusetts Institute of Technology 224 <imagedata fileref="figures/perf-wget-flat-stripped.png" width="6in" depth="7in" align="center" scalefit="1" />
462 (MIT) License. 225 </para>
463 MIT licensing permits the reuse of software within proprietary software as long as the
464 license is distributed with that software.
465 MIT is also compatible with the GNU General Public License (GPL).
466 Patches to the Yocto Project follow the upstream licensing scheme.
467 You can find information on the MIT license at
468 <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>.
469 You can find information on the GNU GPL <ulink url='http://www.opensource.org/licenses/LGPL-3.0'>
470 here</ulink>.
471 </para>
472 226
473 <para> 227 <para>
474 When you build an image using the Yocto Project, the build process uses a 228 The above screenshot displays a 'flat' profile, one entry for
475 known list of licenses to ensure compliance. 229 each 'bucket' corresponding to the functions that were profiled
476 You can find this list in the Yocto Project files directory at 230 during the profiling run, ordered from the most popular to the
477 <filename>meta/files/common-licenses</filename>. 231 least (perf has options to sort in various orders and keys as
478 Once the build completes, the list of all licenses found and used during that build are 232 well as display entries only above a certain threshold and so
479 kept in the 233 on - see the perf documentation for details). Note that this
480 <link linkend='build-directory'>Build Directory</link> at 234 includes both userspace functions (entries containing a [.]) and
481 <filename>tmp/deploy/images/licenses</filename>. 235 kernel functions accounted to the process (entries containing
482 </para> 236 a [k]). (perf has command-line modifiers that can be used to
237 restrict the profiling to kernel or userspace, among others).
238 </para>
483 239
484 <para> 240 <para>
485 If a module requires a license that is not in the base list, the build process 241 Notice also that the above report shows an entry for 'busybox',
486 generates a warning during the build. 242 which is the executable that implements 'wget' in Yocto, but that
487 These tools make it easier for a developer to be certain of the licenses with which 243 instead of a useful function name in that entry, it displays
488 their shipped products must comply. 244 an not-so-friendly hex value instead. The steps below will show
489 However, even with these tools it is still up to the developer to resolve potential licensing issues. 245 how to fix that problem.
490 </para> 246 </para>
491 247
492 <para> 248 <para>
493 The base list of licenses used by the build process is a combination of the Software Package 249 Before we do that, however, let's try running a different profile,
494 Data Exchange (SPDX) list and the Open Source Initiative (OSI) projects. 250 one which shows something a little more interesting. The only
495 <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of the Linux Foundation 251 difference between the new profile and the previous one is that
496 that maintains a specification 252 we'll add the -g option, which will record not just the address
497 for a standard format for communicating the components, licenses, and copyrights 253 of a sampled function, but the entire callchain to the sampled
498 associated with a software package. 254 function as well:
499 <ulink url='http://opensource.org'>OSI</ulink> is a corporation dedicated to the Open Source 255 <literallayout class='monospaced'>
500 Definition and the effort for reviewing and approving licenses that are OSD-conformant. 256 root@crownbay:~# perf record -g wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
501 </para> 257 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
258 linux-2.6.19.2.tar.b 100% |************************************************| 41727k 0:00:00 ETA
259 [ perf record: Woken up 3 times to write data ]
260 [ perf record: Captured and wrote 0.652 MB perf.data (~28476 samples) ]
502 261
503 <para>
504 You can find a list of the combined SPDX and OSI licenses that the Yocto Project uses
505 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/files/common-licenses'>here</ulink>.
506 This wiki page discusses the license infrastructure used by the Yocto Project.
507 </para>
508 262
509 <para> 263 root@crownbay:~# perf report
510 For information that can help you to maintain compliance with various open source licensing 264 </literallayout>
511 during the lifecycle of a product created using the Yocto Project, see the 265 </para>
512 "<link linkend='maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</link>" section.
513 </para>
514</section>
515 266
516<section id='git'> 267 <para>
517 <title>Git</title> 268 <imagedata fileref="figures/perf-wget-g-copy-to-user-expanded-stripped.png" width="6in" depth="7in" align="center" scalefit="1" />
269 </para>
518 270
519 <para> 271 <para>
520 The Yocto Project uses Git, which is a free, open source distributed version control system. 272 Using the callgraph view, we can actually see not only which
521 Git supports distributed development, non-linear development, and can handle large projects. 273 functions took the most time, but we can also see a summary of
522 It is best that you have some fundamental understanding of how Git tracks projects and 274 how those functions were called and learn something about how the
523 how to work with Git if you are going to use Yocto Project for development. 275 program interacts with the kernel in the process.
524 This section provides a quick overview of how Git works and provides you with a summary 276 </para>
525 of some essential Git commands.
526 </para>
527 277
528 <para> 278 <para>
529 For more information on Git, see 279 Notice that each entry in the above screenshot now contains a '+'
530 <ulink url='http://git-scm.com/documentation'></ulink>. 280 on the left-hand side. This means that we can expand the entry and
531 If you need to download Git, go to <ulink url='http://git-scm.com/download'></ulink>. 281 drill down into the callchains that feed into that entry.
532 </para> 282 Pressing 'enter' on any one of them will expand the callchain
283 (you can also press 'E' to expand them all at the same time or 'C'
284 to collapse them all).
285 </para>
533 286
534 <section id='repositories-tags-and-branches'> 287 <para>
535 <title>Repositories, Tags, and Branches</title> 288 In the screenshot above, we've toggled the __copy_to_user_ll()
289 entry and several subnodes all the way down. This lets us see
290 which callchains contributed to the profiled __copy_to_user_ll()
291 function which contributed 1.77% to the total profile.
292 </para>
536 293
537 <para> 294 <para>
538 As mentioned earlier in section 295 As a bit of background explanation for these callchains, think
539 "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>", 296 about what happens at a high level when you run wget to get a file
540 the Yocto Project maintains source repositories at 297 out on the network. Basically what happens is that the data comes
541 <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>. 298 into the kernel via the network connection (socket) and is passed
542 If you look at this web-interface of the repositories, each item is a separate 299 to the userspace program 'wget' (which is actually a part of
543 Git repository. 300 busybox, but that's not important for now), which takes the buffers
301 the kernel passes to it and writes it to a disk file to save it.
544 </para> 302 </para>
545 303
546 <para> 304 <para>
547 Git repositories use branching techniques that track content change (not files) 305 The part of this process that we're looking at in the above call
548 within a project (e.g. a new feature or updated documentation). 306 stacks is the part where the kernel passes the data it's read from
549 Creating a tree-like structure based on project divergence allows for excellent historical 307 the socket down to wget i.e. a copy-to-user.
550 information over the life of a project.
551 This methodology also allows for an environment in which you can do lots of
552 local experimentation on a project as you develop changes or new features.
553 </para> 308 </para>
554 309
555 <para> 310 <para>
556 A Git repository represents all development efforts for a given project. 311 Notice also that here there's also a case where the a hex value
557 For example, the Git repository <filename>poky</filename> contains all changes 312 is displayed in the callstack, here in the expanded
558 and developments for Poky over the course of its entire life. 313 sys_clock_gettime() function. Later we'll see it resolve to a
559 That means that all changes that make up all releases are captured. 314 userspace function call in busybox.
560 The repository maintains a complete history of changes.
561 </para> 315 </para>
562 316
563 <para> 317 <para>
564 You can create a local copy of any repository by "cloning" it with the Git 318 <imagedata fileref="figures/perf-wget-g-copy-from-user-expanded-stripped.png" width="6in" depth="7in" align="center" scalefit="1" />
565 <filename>clone</filename> command.
566 When you clone a Git repository, you end up with an identical copy of the
567 repository on your development system.
568 Once you have a local copy of a repository, you can take steps to develop locally.
569 For examples on how to clone Git repositories, see the section
570 "<link linkend='getting-setup'>Getting Set Up</link>" earlier in this manual.
571 </para> 319 </para>
572 320
573 <para> 321 <para>
574 It is important to understand that Git tracks content change and not files. 322 The above screenshot shows the other half of the journey for the
575 Git uses "branches" to organize different development efforts. 323 data - from the wget program's userspace buffers to disk. To get
576 For example, the <filename>poky</filename> repository has 324 the buffers to disk, the wget program issues a write(2), which
577 <filename>bernard</filename>, 325 does a copy-from-user to the kernel, which then takes care via
578 <filename>edison</filename>, <filename>denzil</filename>, <filename>danny</filename> 326 some circuitous path (probably also present somewhere in the
579 and <filename>master</filename> branches among others. 327 profile data), to get it safely to disk.
580 You can see all the branches by going to
581 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
582 clicking on the
583 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename>
584 link beneath the "Branch" heading.
585 </para> 328 </para>
586 329
587 <para> 330 <para>
588 Each of these branches represents a specific area of development. 331 Now that we've seen the basic layout of the profile data and the
589 The <filename>master</filename> branch represents the current or most recent 332 basics of how to extract useful information out of it, let's get
590 development. 333 back to the task at hand and see if we can get some basic idea
591 All other branches represent off-shoots of the <filename>master</filename> 334 about where the time is spent in the program we're profiling,
592 branch. 335 wget. Remember that wget is actually implemented as an applet
336 in busybox, so while the process name is 'wget', the executable
337 we're actually interested in is busybox. So let's expand the
338 first entry containing busybox:
593 </para> 339 </para>
594 340
595 <para> 341 <para>
596 When you create a local copy of a Git repository, the copy has the same set 342 <imagedata fileref="figures/perf-wget-busybox-expanded-stripped.png" width="6in" depth="7in" align="center" scalefit="1" />
597 of branches as the original.
598 This means you can use Git to create a local working area (also called a branch)
599 that tracks a specific development branch from the source Git repository.
600 in other words, you can define your local Git environment to work on any development
601 branch in the repository.
602 To help illustrate, here is a set of commands that creates a local copy of the
603 <filename>poky</filename> Git repository and then creates and checks out a local
604 Git branch that tracks the Yocto Project &DISTRO; Release (&DISTRO_NAME;) development:
605 <literallayout class='monospaced'>
606 $ cd ~
607 $ git clone git://git.yoctoproject.org/poky
608 $ cd poky
609 $ git checkout -b &DISTRO_NAME; origin/&DISTRO_NAME;
610 </literallayout>
611 In this example, the name of the top-level directory of your local Yocto Project
612 Files Git repository is <filename>poky</filename>,
613 and the name of the local working area (or local branch) you have created and checked
614 out is <filename>&DISTRO_NAME;</filename>.
615 The files in your repository now reflect the same files that are in the
616 <filename>&DISTRO_NAME;</filename> development branch of the Yocto Project's
617 <filename>poky</filename> repository.
618 It is important to understand that when you create and checkout a
619 local working branch based on a branch name,
620 your local environment matches the "tip" of that development branch
621 at the time you created your local branch, which could be
622 different than the files at the time of a similarly named release.
623 In other words, creating and checking out a local branch based on the
624 <filename>&DISTRO_NAME;</filename> branch name is not the same as
625 cloning and checking out the <filename>master</filename> branch.
626 Keep reading to see how you create a local snapshot of a Yocto Project Release.
627 </para> 343 </para>
628 344
629 <para> 345 <para>
630 Git uses "tags" to mark specific changes in a repository. 346 Again, before we expanded we saw that the function was labeled
631 Typically, a tag is used to mark a special point such as the final change 347 with a hex value instead of a symbol as with most of the kernel
632 before a project is released. 348 entries. Expanding the busybox entry doesn't make it any better.
633 You can see the tags used with the <filename>poky</filename> Git repository
634 by going to <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
635 clicking on the
636 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename>
637 link beneath the "Tag" heading.
638 </para> 349 </para>
639 350
640 <para> 351 <para>
641 Some key tags are <filename>bernard-5.0</filename>, <filename>denzil-7.0</filename>, 352 The problem is that perf can't find the symbol information for the
642 and <filename>&DISTRO_NAME;-&POKYVERSION;</filename>. 353 busybox binary, which is actually stripped out by the Yocto build
643 These tags represent Yocto Project releases. 354 system.
644 </para> 355 </para>
645 356
646 <para> 357 <para>
647 When you create a local copy of the Git repository, you also have access to all the 358 One way around that is to put the following in your local.conf
648 tags. 359 when you build the image:
649 Similar to branches, you can create and checkout a local working Git branch based
650 on a tag name.
651 When you do this, you get a snapshot of the Git repository that reflects
652 the state of the files when the change was made associated with that tag.
653 The most common use is to checkout a working branch that matches a specific
654 Yocto Project release.
655 Here is an example:
656 <literallayout class='monospaced'> 360 <literallayout class='monospaced'>
657 $ cd ~ 361 INHIBIT_PACKAGE_STRIP = "1"
658 $ git clone git://git.yoctoproject.org/poky
659 $ cd poky
660 $ git checkout -b my-&DISTRO_NAME;-&POKYVERSION; &DISTRO_NAME;-&POKYVERSION;
661 </literallayout> 362 </literallayout>
662 In this example, the name of the top-level directory of your local Yocto Project 363 However, we already have an image with the binaries stripped,
663 Files Git repository is <filename>poky</filename>. 364 so what can we do to get perf to resolve the symbols? Basically
664 And, the name of the local branch you have created and checked out is 365 we need to install the debuginfo for the busybox package.
665 <filename>my-&DISTRO_NAME;-&POKYVERSION;</filename>.
666 The files in your repository now exactly match the Yocto Project &DISTRO;
667 Release tag (<filename>&DISTRO_NAME;-&POKYVERSION;</filename>).
668 It is important to understand that when you create and checkout a local
669 working branch based on a tag, your environment matches a specific point
670 in time and not a development branch.
671 </para> 366 </para>
672 </section>
673 367
674 <section id='basic-commands'> 368 <para>
675 <title>Basic Commands</title> 369 To generate the debug info for the packages in the image, we can
370 to add dbg-pkgs to EXTRA_IMAGE_FEATURES in local.conf. For example:
371 <literallayout class='monospaced'>
372 EXTRA_IMAGE_FEATURES = "debug-tweaks tools-profile dbg-pkgs"
373 </literallayout>
374 Additionally, in order to generate the type of debuginfo that
375 perf understands, we also need to add the following to local.conf:
376 <literallayout class='monospaced'>
377 PACKAGE_DEBUG_SPLIT_STYLE = 'debug-file-directory'
378 </literallayout>
379 Once we've done that, we can install the debuginfo for busybox.
380 The debug packages once built can be found in
381 build/tmp/deploy/rpm/* on the host system. Find the
382 busybox-dbg-...rpm file and copy it to the target. For example:
383 <literallayout class='monospaced'>
384 [trz@empanada core2]$ scp /home/trz/yocto/crownbay-tracing-dbg/build/tmp/deploy/rpm/core2/busybox-dbg-1.20.2-r2.core2.rpm root@192.168.1.31:
385 root@192.168.1.31's password:
386 busybox-dbg-1.20.2-r2.core2.rpm 100% 1826KB 1.8MB/s 00:01
387 </literallayout>
388 Now install the debug rpm on the target:
389 <literallayout class='monospaced'>
390 root@crownbay:~# rpm -i busybox-dbg-1.20.2-r2.core2.rpm
391 </literallayout>
392 Now that the debuginfo is installed, we see that the busybox
393 entries now display their functions symbolically:
394 </para>
676 395
677 <para> 396 <para>
678 Git has an extensive set of commands that lets you manage changes and perform 397 <imagedata fileref="figures/perf-wget-busybox-debuginfo.png" width="6in" depth="7in" align="center" scalefit="1" />
679 collaboration over the life of a project.
680 Conveniently though, you can manage with a small set of basic operations and workflows
681 once you understand the basic philosophy behind Git.
682 You do not have to be an expert in Git to be functional.
683 A good place to look for instruction on a minimal set of Git commands is
684 <ulink url='http://git-scm.com/documentation'>here</ulink>.
685 If you need to download Git, you can do so
686 <ulink url='http://git-scm.com/download'>here</ulink>.
687 </para> 398 </para>
688 399
689 <para> 400 <para>
690 If you don’t know much about Git, we suggest you educate 401 If we expand one of the entries and press 'enter' on a leaf node,
691 yourself by visiting the links previously mentioned. 402 we're presented with a menu of actions we can take to get more
403 information related to that entry:
692 </para> 404 </para>
693 405
694 <para> 406 <para>
695 The following list briefly describes some basic Git operations as a way to get started. 407 <imagedata fileref="figures/perf-wget-busybox-dso-zoom-menu.png" width="6in" depth="7in" align="center" scalefit="1" />
696 As with any set of commands, this list (in most cases) simply shows the base command and
697 omits the many arguments they support.
698 See the Git documentation for complete descriptions and strategies on how to use these commands:
699 <itemizedlist>
700 <listitem><para><emphasis><filename>git init</filename>:</emphasis> Initializes an empty Git repository.
701 You cannot use Git commands unless you have a <filename>.git</filename> repository.</para></listitem>
702 <listitem><para><emphasis><filename>git clone</filename>:</emphasis> Creates a clone of a repository.
703 During collaboration, this command allows you to create a local repository that is on
704 equal footing with a fellow developer’s repository.</para></listitem>
705 <listitem><para><emphasis><filename>git add</filename>:</emphasis> Adds updated file contents
706 to the index that
707 Git uses to track changes.
708 You must add all files that have changed before you can commit them.</para></listitem>
709 <listitem><para><emphasis><filename>git commit</filename>:</emphasis> Creates a “commit” that documents
710 the changes you made.
711 Commits are used for historical purposes, for determining if a maintainer of a project
712 will allow the change, and for ultimately pushing the change from your local Git repository
713 into the project’s upstream (or master) repository.</para></listitem>
714 <listitem><para><emphasis><filename>git status</filename>:</emphasis> Reports any modified files that
715 possibly need to be added and committed.</para></listitem>
716 <listitem><para><emphasis><filename>git checkout &lt;branch-name&gt;</filename>:</emphasis> Changes
717 your working branch.
718 This command is analogous to “cd”.</para></listitem>
719 <listitem><para><emphasis><filename>git checkout –b &lt;working-branch&gt;</filename>:</emphasis> Creates
720 a working branch on your local machine where you can isolate work.
721 It is a good idea to use local branches when adding specific features or changes.
722 This way if you don’t like what you have done you can easily get rid of the work.</para></listitem>
723 <listitem><para><emphasis><filename>git branch</filename>:</emphasis> Reports
724 existing local branches and
725 tells you the branch in which you are currently working.</para></listitem>
726 <listitem><para><emphasis><filename>git branch -D &lt;branch-name&gt;</filename>:</emphasis>
727 Deletes an existing local branch.
728 You need to be in a local branch other than the one you are deleting
729 in order to delete <filename>&lt;branch-name&gt;</filename>.</para></listitem>
730 <listitem><para><emphasis><filename>git pull</filename>:</emphasis> Retrieves information
731 from an upstream Git
732 repository and places it in your local Git repository.
733 You use this command to make sure you are synchronized with the repository
734 from which you are basing changes (.e.g. the master branch).</para></listitem>
735 <listitem><para><emphasis><filename>git push</filename>:</emphasis> Sends all your local changes you
736 have committed to an upstream Git repository (e.g. a contribution repository).
737 The maintainer of the project draws from these repositories when adding your changes to the
738 project’s master repository.</para></listitem>
739 <listitem><para><emphasis><filename>git merge</filename>:</emphasis> Combines or adds changes from one
740 local branch of your repository with another branch.
741 When you create a local Git repository, the default branch is named “master”.
742 A typical workflow is to create a temporary branch for isolated work, make and commit your
743 changes, switch to your local master branch, merge the changes from the temporary branch into the
744 local master branch, and then delete the temporary branch.</para></listitem>
745 <listitem><para><emphasis><filename>git cherry-pick</filename>:</emphasis> Choose and apply specific
746 commits from one branch into another branch.
747 There are times when you might not be able to merge all the changes in one branch with
748 another but need to pick out certain ones.</para></listitem>
749 <listitem><para><emphasis><filename>gitk</filename>:</emphasis> Provides a GUI view of the branches
750 and changes in your local Git repository.
751 This command is a good way to graphically see where things have diverged in your
752 local repository.</para></listitem>
753 <listitem><para><emphasis><filename>git log</filename>:</emphasis> Reports a history of your changes to the
754 repository.</para></listitem>
755 <listitem><para><emphasis><filename>git diff</filename>:</emphasis> Displays line-by-line differences
756 between your local working files and the same files in the upstream Git repository that your
757 branch currently tracks.</para></listitem>
758 </itemizedlist>
759 </para> 408 </para>
760 </section>
761</section>
762 409
763<section id='workflows'> 410 <para>
764 <title>Workflows</title> 411 One of these actions allows us to show a view that displays a
412 busybox-centric view of the profiled functions (in this case we've
413 also expanded all the nodes using the 'E' key):
414 </para>
765 415
766 <para> 416 <para>
767 This section provides some overview on workflows using Git. 417 <imagedata fileref="figures/perf-wget-busybox-dso-zoom.png" width="6in" depth="7in" align="center" scalefit="1" />
768 In particular, the information covers basic practices that describe roles and actions in a 418 </para>
769 collaborative development environment.
770 Again, if you are familiar with this type of development environment, you might want to just
771 skip this section.
772 </para>
773 419
774 <para> 420 <para>
775 The Yocto Project files are maintained using Git in a "master" branch whose Git history 421 Finally, we can see that now that the busybox debuginfo is
776 tracks every change and whose structure provides branches for all diverging functionality. 422 installed, the previously unresolved symbol in the
777 Although there is no need to use Git, many open source projects do so. 423 sys_clock_gettime() entry mentioned previously is now resolved,
778 For the Yocto Project, a key individual called the "maintainer" is responsible for the "master" 424 and shows that the sys_clock_gettime system call that was the
779 branch of the Git repository. 425 source of 6.75% of the copy-to-user overhead was initiated by
780 The "master" branch is the “upstream” repository where the final builds of the project occur. 426 the handle_input() busybox function:
781 The maintainer is responsible for allowing changes in from other developers and for 427 </para>
782 organizing the underlying branch structure to reflect release strategies and so forth.
783 <note>You can see who is the maintainer for Yocto Project files by examining the
784 <filename>maintainers.inc</filename> file in the Yocto Project
785 <filename>meta-yocto/conf/distro/include</filename> directory.</note>
786 </para>
787 428
788 <para> 429 <para>
789 The project also has contribution repositories known as “contrib” areas. 430 <imagedata fileref="figures/perf-wget-g-copy-to-user-expanded-debuginfo.png" width="6in" depth="7in" align="center" scalefit="1" />
790 These areas temporarily hold changes to the project that have been submitted or committed 431 </para>
791 by the Yocto Project development team and by community members that contribute to the project.
792 The maintainer determines if the changes are qualified to be moved from the "contrib" areas
793 into the "master" branch of the Git repository.
794 </para>
795 432
796 <para> 433 <para>
797 Developers (including contributing community members) create and maintain cloned repositories 434 At the lowest level of detail, we can dive down to the assembly
798 of the upstream "master" branch. 435 level and see which instructions caused the most overhead in a
799 These repositories are local to their development platforms and are used to develop changes. 436 function. Pressing 'enter' on the 'udhcpc_main' function, we're
800 When a developer is satisfied with a particular feature or change, they “push” the changes 437 again presented with a menu:
801 to the appropriate "contrib" repository. 438 </para>
802 </para>
803 439
804 <para> 440 <para>
805 Developers are responsible for keeping their local repository up-to-date with "master". 441 <imagedata fileref="figures/perf-wget-busybox-annotate-menu.png" width="6in" depth="7in" align="center" scalefit="1" />
806 They are also responsible for straightening out any conflicts that might arise within files 442 </para>
807 that are being worked on simultaneously by more than one person.
808 All this work is done locally on the developer’s machine before anything is pushed to a
809 "contrib" area and examined at the maintainer’s level.
810 </para>
811 443
812 <para> 444 <para>
813 A somewhat formal method exists by which developers commit changes and push them into the 445 Selecting 'Annotate udhcpc_main', we get a detailed listing of
814 "contrib" area and subsequently request that the maintainer include them into "master" 446 percentages by instruction for the udhcpc_main function. From the
815 This process is called “submitting a patch” or “submitting a change.” 447 display, we can see that over 50% of the time spent in this
816 For information on submitting patches and changes, see the 448 function is taken up by a couple tests and the move of a
817 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" section. 449 constant (1) to a register:
818 </para> 450 </para>
819 451
820 <para> 452 <para>
821 To summarize the environment: we have a single point of entry for changes into the project’s 453 <imagedata fileref="figures/perf-wget-busybox-annotate-udhcpc.png" width="6in" depth="7in" align="center" scalefit="1" />
822 "master" branch of the Git repository, which is controlled by the project’s maintainer. 454 </para>
823 And, we have a set of developers who independently develop, test, and submit changes
824 to "contrib" areas for the maintainer to examine.
825 The maintainer then chooses which changes are going to become a permanent part of the project.
826 </para>
827 455
828 <para> 456 <para>
829 <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" /> 457 As a segue into tracing, let's try another profile using a
830 </para> 458 different counter, something other than the default 'cycles'.
459 </para>
831 460
832 <para> 461 <para>
833 While each development environment is unique, there are some best practices or methods 462 The tracing and profiling infrastructure in Linux has become
834 that help development run smoothly. 463 unified in a way that allows us to use the same tool with a
835 The following list describes some of these practices. 464 completely different set of counters, not just the standard
836 For more information about Git workflows, see the workflow topics in the 465 hardware counters that traditionally tools have had to restrict
837 <ulink url='http://book.git-scm.com'>Git Community Book</ulink>. 466 themselves to (of course the traditional tools can also make use
838 <itemizedlist> 467 of the expanded possibilities now available to them, and in some
839 <listitem><para><emphasis>Make Small Changes:</emphasis> It is best to keep the changes you commit 468 cases have, as mentioned previously).
840 small as compared to bundling many disparate changes into a single commit. 469 </para>
841 This practice not only keeps things manageable but also allows the maintainer
842 to more easily include or refuse changes.</para>
843 <para>It is also good practice to leave the repository in a state that allows you to
844 still successfully build your project. In other words, do not commit half of a feature,
845 then add the other half in a separate, later commit.
846 Each commit should take you from one buildable project state to another
847 buildable state.</para></listitem>
848 <listitem><para><emphasis>Use Branches Liberally:</emphasis> It is very easy to create, use, and
849 delete local branches in your working Git repository.
850 You can name these branches anything you like.
851 It is helpful to give them names associated with the particular feature or change
852 on which you are working.
853 Once you are done with a feature or change, simply discard the branch.</para></listitem>
854 <listitem><para><emphasis>Merge Changes:</emphasis> The <filename>git merge</filename>
855 command allows you to take the
856 changes from one branch and fold them into another branch.
857 This process is especially helpful when more than a single developer might be working
858 on different parts of the same feature.
859 Merging changes also automatically identifies any collisions or “conflicts”
860 that might happen as a result of the same lines of code being altered by two different
861 developers.</para></listitem>
862 <listitem><para><emphasis>Manage Branches:</emphasis> Because branches are easy to use, you should
863 use a system where branches indicate varying levels of code readiness.
864 For example, you can have a “work” branch to develop in, a “test” branch where the code or
865 change is tested, a “stage” branch where changes are ready to be committed, and so forth.
866 As your project develops, you can merge code across the branches to reflect ever-increasing
867 stable states of the development.</para></listitem>
868 <listitem><para><emphasis>Use Push and Pull:</emphasis> The push-pull workflow is based on the
869 concept of developers “pushing” local commits to a remote repository, which is
870 usually a contribution repository.
871 This workflow is also based on developers “pulling” known states of the project down into their
872 local development repositories.
873 The workflow easily allows you to pull changes submitted by other developers from the
874 upstream repository into your work area ensuring that you have the most recent software
875 on which to develop.
876 The Yocto Project has two scripts named <filename>create-pull-request</filename> and
877 <filename>send-pull-request</filename> that ship with the release to facilitate this
878 workflow.
879 You can find these scripts in the local Yocto Project files Git repository in
880 the <filename>scripts</filename> directory.</para>
881 <para>You can find more information on these scripts in the
882 "<link linkend='pushing-a-change-upstream'>Using
883 Scripts to Push a Change Upstream and Request a Pull</link>" section.
884 </para></listitem>
885 <listitem><para><emphasis>Patch Workflow:</emphasis> This workflow allows you to notify the
886 maintainer through an email that you have a change (or patch) you would like considered
887 for the "master" branch of the Git repository.
888 To send this type of change you format the patch and then send the email using the Git commands
889 <filename>git format-patch</filename> and <filename>git send-email</filename>.
890 You can find information on how to submit changes
891 later in this chapter.</para></listitem>
892 </itemizedlist>
893 </para>
894</section>
895 470
896<section id='tracking-bugs'> 471 <para>
897 <title>Tracking Bugs</title> 472 We can get a list of the available events that can be used to
473 profile a workload via 'perf list':
474 <literallayout class='monospaced'>
475 root@crownbay:~# perf list
476
477 List of pre-defined events (to be used in -e):
478 cpu-cycles OR cycles [Hardware event]
479 stalled-cycles-frontend OR idle-cycles-frontend [Hardware event]
480 stalled-cycles-backend OR idle-cycles-backend [Hardware event]
481 instructions [Hardware event]
482 cache-references [Hardware event]
483 cache-misses [Hardware event]
484 branch-instructions OR branches [Hardware event]
485 branch-misses [Hardware event]
486 bus-cycles [Hardware event]
487 ref-cycles [Hardware event]
488
489 cpu-clock [Software event]
490 task-clock [Software event]
491 page-faults OR faults [Software event]
492 minor-faults [Software event]
493 major-faults [Software event]
494 context-switches OR cs [Software event]
495 cpu-migrations OR migrations [Software event]
496 alignment-faults [Software event]
497 emulation-faults [Software event]
498
499 L1-dcache-loads [Hardware cache event]
500 L1-dcache-load-misses [Hardware cache event]
501 L1-dcache-prefetch-misses [Hardware cache event]
502 L1-icache-loads [Hardware cache event]
503 L1-icache-load-misses [Hardware cache event]
504 .
505 .
506 .
507 rNNN [Raw hardware event descriptor]
508 cpu/t1=v1[,t2=v2,t3 ...]/modifier [Raw hardware event descriptor]
509 (see 'perf list --help' on how to encode it)
510
511 mem:&lt;addr&gt;[:access] [Hardware breakpoint]
512
513 sunrpc:rpc_call_status [Tracepoint event]
514 sunrpc:rpc_bind_status [Tracepoint event]
515 sunrpc:rpc_connect_status [Tracepoint event]
516 sunrpc:rpc_task_begin [Tracepoint event]
517 skb:kfree_skb [Tracepoint event]
518 skb:consume_skb [Tracepoint event]
519 skb:skb_copy_datagram_iovec [Tracepoint event]
520 net:net_dev_xmit [Tracepoint event]
521 net:net_dev_queue [Tracepoint event]
522 net:netif_receive_skb [Tracepoint event]
523 net:netif_rx [Tracepoint event]
524 napi:napi_poll [Tracepoint event]
525 sock:sock_rcvqueue_full [Tracepoint event]
526 sock:sock_exceed_buf_limit [Tracepoint event]
527 udp:udp_fail_queue_rcv_skb [Tracepoint event]
528 hda:hda_send_cmd [Tracepoint event]
529 hda:hda_get_response [Tracepoint event]
530 hda:hda_bus_reset [Tracepoint event]
531 scsi:scsi_dispatch_cmd_start [Tracepoint event]
532 scsi:scsi_dispatch_cmd_error [Tracepoint event]
533 scsi:scsi_eh_wakeup [Tracepoint event]
534 drm:drm_vblank_event [Tracepoint event]
535 drm:drm_vblank_event_queued [Tracepoint event]
536 drm:drm_vblank_event_delivered [Tracepoint event]
537 random:mix_pool_bytes [Tracepoint event]
538 random:mix_pool_bytes_nolock [Tracepoint event]
539 random:credit_entropy_bits [Tracepoint event]
540 gpio:gpio_direction [Tracepoint event]
541 gpio:gpio_value [Tracepoint event]
542 block:block_rq_abort [Tracepoint event]
543 block:block_rq_requeue [Tracepoint event]
544 block:block_rq_issue [Tracepoint event]
545 block:block_bio_bounce [Tracepoint event]
546 block:block_bio_complete [Tracepoint event]
547 block:block_bio_backmerge [Tracepoint event]
548 .
549 .
550 writeback:writeback_wake_thread [Tracepoint event]
551 writeback:writeback_wake_forker_thread [Tracepoint event]
552 writeback:writeback_bdi_register [Tracepoint event]
553 .
554 .
555 writeback:writeback_single_inode_requeue [Tracepoint event]
556 writeback:writeback_single_inode [Tracepoint event]
557 kmem:kmalloc [Tracepoint event]
558 kmem:kmem_cache_alloc [Tracepoint event]
559 kmem:mm_page_alloc [Tracepoint event]
560 kmem:mm_page_alloc_zone_locked [Tracepoint event]
561 kmem:mm_page_pcpu_drain [Tracepoint event]
562 kmem:mm_page_alloc_extfrag [Tracepoint event]
563 vmscan:mm_vmscan_kswapd_sleep [Tracepoint event]
564 vmscan:mm_vmscan_kswapd_wake [Tracepoint event]
565 vmscan:mm_vmscan_wakeup_kswapd [Tracepoint event]
566 vmscan:mm_vmscan_direct_reclaim_begin [Tracepoint event]
567 .
568 .
569 module:module_get [Tracepoint event]
570 module:module_put [Tracepoint event]
571 module:module_request [Tracepoint event]
572 sched:sched_kthread_stop [Tracepoint event]
573 sched:sched_wakeup [Tracepoint event]
574 sched:sched_wakeup_new [Tracepoint event]
575 sched:sched_process_fork [Tracepoint event]
576 sched:sched_process_exec [Tracepoint event]
577 sched:sched_stat_runtime [Tracepoint event]
578 rcu:rcu_utilization [Tracepoint event]
579 workqueue:workqueue_queue_work [Tracepoint event]
580 workqueue:workqueue_execute_end [Tracepoint event]
581 signal:signal_generate [Tracepoint event]
582 signal:signal_deliver [Tracepoint event]
583 timer:timer_init [Tracepoint event]
584 timer:timer_start [Tracepoint event]
585 timer:hrtimer_cancel [Tracepoint event]
586 timer:itimer_state [Tracepoint event]
587 timer:itimer_expire [Tracepoint event]
588 irq:irq_handler_entry [Tracepoint event]
589 irq:irq_handler_exit [Tracepoint event]
590 irq:softirq_entry [Tracepoint event]
591 irq:softirq_exit [Tracepoint event]
592 irq:softirq_raise [Tracepoint event]
593 printk:console [Tracepoint event]
594 task:task_newtask [Tracepoint event]
595 task:task_rename [Tracepoint event]
596 syscalls:sys_enter_socketcall [Tracepoint event]
597 syscalls:sys_exit_socketcall [Tracepoint event]
598 .
599 .
600 .
601 syscalls:sys_enter_unshare [Tracepoint event]
602 syscalls:sys_exit_unshare [Tracepoint event]
603 raw_syscalls:sys_enter [Tracepoint event]
604 raw_syscalls:sys_exit [Tracepoint event]
605 </literallayout>
606 </para>
898 607
899 <para> 608 <note>
900 The Yocto Project uses its own implementation of 609 Tying It Together: These are exactly the same set of events defined
901 <ulink url='http://www.bugzilla.org/about/'>Bugzilla</ulink> to track bugs. 610 by the trace event subsystem and exposed by
902 Implementations of Bugzilla work well for group development because they track bugs and code 611 ftrace/tracecmd/kernelshark as files in
903 changes, can be used to communicate changes and problems with developers, can be used to 612 /sys/kernel/debug/tracing/events, by SystemTap as
904 submit and review patches, and can be used to manage quality assurance. 613 kernel.trace("tracepoint_name") and (partially) accessed by LTTng.
905 The home page for the Yocto Project implementation of Bugzilla is 614 </note>
906 <ulink url='&YOCTO_BUGZILLA_URL;'>&YOCTO_BUGZILLA_URL;</ulink>.
907 </para>
908 615
909 <para> 616 <para>
910 Sometimes it is helpful to submit, investigate, or track a bug against the Yocto Project itself 617 Only a subset of these would be of interest to us when looking at
911 such as when discovering an issue with some component of the build system that acts contrary 618 this workload, so let's choose the most likely subsystems
912 to the documentation or your expectations. 619 (identified by the string before the colon in the Tracepoint events)
913 Following is the general procedure for submitting a new bug using the Yocto Project 620 and do a 'perf stat' run using only those wildcarded subsystems:
914 Bugzilla. 621 <literallayout class='monospaced'>
915 You can find more information on defect management, bug tracking, and feature request 622 root@crownbay:~# perf stat -e skb:* -e net:* -e napi:* -e sched:* -e workqueue:* -e irq:* -e syscalls:* wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
916 processes all accomplished through the Yocto Project Bugzilla on the wiki page 623 Performance counter stats for 'wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2':
917 <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>here</ulink>. 624
918 <orderedlist> 625 23323 skb:kfree_skb
919 <listitem><para>Always use the Yocto Project implementation of Bugzilla to submit 626 0 skb:consume_skb
920 a bug.</para></listitem> 627 49897 skb:skb_copy_datagram_iovec
921 <listitem><para>When submitting a new bug, be sure to choose the appropriate 628 6217 net:net_dev_xmit
922 Classification, Product, and Component for which the issue was found. 629 6217 net:net_dev_queue
923 Defects for Yocto Project fall into one of six classifications: Yocto Project 630 7962 net:netif_receive_skb
924 Components, Infrastructure, Build System &amp; Metadata, Documentation, 631 2 net:netif_rx
925 QA/Testing, and Runtime. 632 8340 napi:napi_poll
926 Each of these Classifications break down into multiple Products and, in some 633 0 sched:sched_kthread_stop
927 cases, multiple Components.</para></listitem> 634 0 sched:sched_kthread_stop_ret
928 <listitem><para>Use the bug form to choose the correct Hardware and Architecture 635 3749 sched:sched_wakeup
929 for which the bug applies.</para></listitem> 636 0 sched:sched_wakeup_new
930 <listitem><para>Indicate the Yocto Project version you were using when the issue 637 0 sched:sched_switch
931 occurred.</para></listitem> 638 29 sched:sched_migrate_task
932 <listitem><para>Be sure to indicate the Severity of the bug. 639 0 sched:sched_process_free
933 Severity communicates how the bug impacted your work.</para></listitem> 640 1 sched:sched_process_exit
934 <listitem><para>Provide a brief summary of the issue. 641 0 sched:sched_wait_task
935 Try to limit your summary to just a line or two and be sure to capture the 642 0 sched:sched_process_wait
936 essence of the issue.</para></listitem> 643 0 sched:sched_process_fork
937 <listitem><para>Provide a detailed description of the issue. 644 1 sched:sched_process_exec
938 You should provide as much detail as you can about the context, behavior, output, 645 0 sched:sched_stat_wait
939 and so forth that surround the issue. 646 2106519415641 sched:sched_stat_sleep
940 You can even attach supporting files for output or log by using the "Add an attachment" 647 0 sched:sched_stat_iowait
941 button.</para></listitem> 648 147453613 sched:sched_stat_blocked
942 <listitem><para>Submit the bug by clicking the "Submit Bug" button.</para></listitem> 649 12903026955 sched:sched_stat_runtime
943 </orderedlist> 650 0 sched:sched_pi_setprio
944 </para> 651 3574 workqueue:workqueue_queue_work
945</section> 652 3574 workqueue:workqueue_activate_work
653 0 workqueue:workqueue_execute_start
654 0 workqueue:workqueue_execute_end
655 16631 irq:irq_handler_entry
656 16631 irq:irq_handler_exit
657 28521 irq:softirq_entry
658 28521 irq:softirq_exit
659 28728 irq:softirq_raise
660 1 syscalls:sys_enter_sendmmsg
661 1 syscalls:sys_exit_sendmmsg
662 0 syscalls:sys_enter_recvmmsg
663 0 syscalls:sys_exit_recvmmsg
664 14 syscalls:sys_enter_socketcall
665 14 syscalls:sys_exit_socketcall
666 .
667 .
668 .
669 16965 syscalls:sys_enter_read
670 16965 syscalls:sys_exit_read
671 12854 syscalls:sys_enter_write
672 12854 syscalls:sys_exit_write
673 .
674 .
675 .
676
677 58.029710972 seconds time elapsed
678 </literallayout>
679 Let's pick one of these tracepoints and tell perf to do a profile
680 using it as the sampling event:
681 <literallayout class='monospaced'>
682 root@crownbay:~# perf record -g -e sched:sched_wakeup wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
683 </literallayout>
684 </para>
946 685
947<section id='how-to-submit-a-change'> 686 <para>
948 <title>How to Submit a Change</title> 687 <imagedata fileref="figures/sched-wakeup-profile.png" width="6in" depth="7in" align="center" scalefit="1" />
688 </para>
949 689
950 <para> 690 <para>
951 Contributions to the Yocto Project and OpenEmbedded are very welcome. 691 The screenshot above shows the results of running a profile using
952 Because the system is extremely configurable and flexible, we recognize that developers 692 sched:sched_switch tracepoint, which shows the relative costs of
953 will want to extend, configure or optimize it for their specific uses. 693 various paths to sched_wakeup (note that sched_wakeup is the
954 You should send patches to the appropriate mailing list so that they 694 name of the tracepoint - it's actually defined just inside
955 can be reviewed and merged by the appropriate maintainer. 695 ttwu_do_wakeup(), which accounts for the function name actually
956 For a list of the Yocto Project and related mailing lists, see the 696 displayed in the profile:
957 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing lists</ulink>" section in 697 <literallayout class='monospaced'>
958 the Yocto Project Reference Manual. 698 /*
959 </para> 699 * Mark the task runnable and perform wakeup-preemption.
700 */
701 static void
702 ttwu_do_wakeup(struct rq *rq, struct task_struct *p, int wake_flags)
703 {
704 trace_sched_wakeup(p, true);
705 .
706 .
707 .
708 }
709 </literallayout>
710 A couple of the more interesting callchains are expanded and
711 displayed above, basically some network receive paths that
712 presumably end up waking up wget (busybox) when network data is
713 ready.
714 </para>
960 715
961 <para> 716 <para>
962 The following is some guidance on which mailing list to use for what type of change: 717 Note that because tracepoints are normally used for tracing,
963 <itemizedlist> 718 the default sampling period for tracepoints is 1 i.e. for
964 <listitem><para>For changes to the core metadata, send your patch to the 719 tracepoints perf will sample on every event occurrence (this
965 <ulink url='&OE_LISTS_URL;/listinfo/openembedded-core'>openembedded-core</ulink> mailing list. 720 can be changed using the -c option). This is in contrast to
966 For example, a change to anything under the <filename>meta</filename> or 721 hardware counters such as for example the default 'cycles'
967 <filename>scripts</filename> directories 722 hardware counter used for normal profiling, where sampling
968 should be sent to this mailing list.</para></listitem> 723 periods are much higher (in the thousands) because profiling should
969 <listitem><para>For changes to BitBake (anything under the <filename>bitbake</filename> 724 have as low an overhead as possible and sampling on every cycle w
970 directory), send your patch to the 725 ould be prohibitively expensive.
971 <ulink url='&OE_LISTS_URL;/listinfo/bitbake-devel'>bitbake-devel</ulink> mailing list.</para></listitem> 726 </para>
972 <listitem><para>For changes to <filename>meta-yocto</filename>, send your patch to the 727 </section>
973 <ulink url='&YOCTO_LISTS_URL;/listinfo/poky'>poky</ulink> mailing list.</para></listitem>
974 <listitem><para>For changes to other layers hosted on
975 <filename>yoctoproject.org</filename> (unless the
976 layer's documentation specifies otherwise), tools, and Yocto Project
977 documentation, use the
978 <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> mailing list.</para></listitem>
979 <listitem><para>For additional recipes that do not fit into the core metadata,
980 you should determine which layer the recipe should go into and submit the
981 change in the manner recommended by the documentation (e.g. README) supplied
982 with the layer. If in doubt, please ask on the
983 <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> or
984 <ulink url='&OE_LISTS_URL;/listinfo/openembedded-devel'>openembedded-devel</ulink>
985 mailing lists.</para></listitem>
986 </itemizedlist>
987 </para>
988 728
989 <para> 729 <section id='using-perf-to-do-basic-tracing'>
990 When you send a patch, be sure to include a "Signed-off-by:" 730 <title>Using perf to do Basic Tracing</title>
991 line in the same style as required by the Linux kernel.
992 Adding this line signifies that you, the submitter, have agreed to the Developer's Certificate of Origin 1.1
993 as follows:
994 <literallayout class='monospaced'>
995 Developer's Certificate of Origin 1.1
996
997 By making a contribution to this project, I certify that:
998
999 (a) The contribution was created in whole or in part by me and I
1000 have the right to submit it under the open source license
1001 indicated in the file; or
1002
1003 (b) The contribution is based upon previous work that, to the best
1004 of my knowledge, is covered under an appropriate open source
1005 license and I have the right under that license to submit that
1006 work with modifications, whether created in whole or in part
1007 by me, under the same open source license (unless I am
1008 permitted to submit under a different license), as indicated
1009 in the file; or
1010
1011 (c) The contribution was provided directly to me by some other
1012 person who certified (a), (b) or (c) and I have not modified
1013 it.
1014
1015 (d) I understand and agree that this project and the contribution
1016 are public and that a record of the contribution (including all
1017 personal information I submit with it, including my sign-off) is
1018 maintained indefinitely and may be redistributed consistent with
1019 this project or the open source license(s) involved.
1020 </literallayout>
1021 </para>
1022 731
1023 <para> 732 <para>
1024 In a collaborative environment, it is necessary to have some sort of standard 733 Profiling is a great tool for solving many problems or for
1025 or method through which you submit changes. 734 getting a high-level view of what's going on with a workload or
1026 Otherwise, things could get quite chaotic. 735 across the system. It is however by definition an approximation,
1027 One general practice to follow is to make small, controlled changes. 736 as suggested by the most prominent word associated with it,
1028 Keeping changes small and isolated aids review, makes merging/rebasing easier 737 'sampling'. On the one hand, it allows a representative picture of
1029 and keeps the change history clean when anyone needs to refer to it in future. 738 what's going on in the system to be cheaply taken, but on the other
1030 </para> 739 hand, that cheapness limits its utility when that data suggests a
740 need to 'dive down' more deeply to discover what's really going
741 on. In such cases, the only way to see what's really going on is
742 to be able to look at (or summarize more intelligently) the
743 individual steps that go into the higher-level behavior exposed
744 by the coarse-grained profiling data.
745 </para>
1031 746
1032 <para> 747 <para>
1033 When you make a commit, you must follow certain standards established by the 748 As a concrete example, we can trace all the events we think might
1034 OpenEmbedded and Yocto Project development teams. 749 be applicable to our workload:
1035 For each commit, you must provide a single-line summary of the change and you 750 <literallayout class='monospaced'>
1036 should almost always provide a more detailed description of what you did (i.e. 751 root@crownbay:~# perf record -g -e skb:* -e net:* -e napi:* -e sched:sched_switch -e sched:sched_wakeup -e irq:*
1037 the body of the commit message). 752 -e syscalls:sys_enter_read -e syscalls:sys_exit_read -e syscalls:sys_enter_write -e syscalls:sys_exit_write
1038 The only exceptions for not providing a detailed description would be if your 753 wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
1039 change is a simple, self-explanatory change that needs no further description 754 </literallayout>
1040 beyond the summary. 755 We can look at the raw trace output using 'perf script' with no
1041 Here are the guidelines for composing a commit message: 756 arguments:
1042 <itemizedlist> 757 <literallayout class='monospaced'>
1043 <listitem><para>Provide a single-line, short summary of the change. 758 root@crownbay:~# perf script
1044 This summary is typically viewable in the "shortlist" of changes. 759
1045 Thus, providing something short and descriptive that gives the reader 760 perf 1262 [000] 11624.857082: sys_exit_read: 0x0
1046 a summary of the change is useful when viewing a list of many commits. 761 perf 1262 [000] 11624.857193: sched_wakeup: comm=migration/0 pid=6 prio=0 success=1 target_cpu=000
1047 This should be prefixed by the recipe name (if changing a recipe), or 762 wget 1262 [001] 11624.858021: softirq_raise: vec=1 [action=TIMER]
1048 else the short form path to the file being changed. 763 wget 1262 [001] 11624.858074: softirq_entry: vec=1 [action=TIMER]
1049 </para></listitem> 764 wget 1262 [001] 11624.858081: softirq_exit: vec=1 [action=TIMER]
1050 <listitem><para>For the body of the commit message, provide detailed information 765 wget 1262 [001] 11624.858166: sys_enter_read: fd: 0x0003, buf: 0xbf82c940, count: 0x0200
1051 that describes what you changed, why you made the change, and the approach 766 wget 1262 [001] 11624.858177: sys_exit_read: 0x200
1052 you used. It may also be helpful if you mention how you tested the change. 767 wget 1262 [001] 11624.858878: kfree_skb: skbaddr=0xeb248d80 protocol=0 location=0xc15a5308
1053 Provide as much detail as you can in the body of the commit message. 768 wget 1262 [001] 11624.858945: kfree_skb: skbaddr=0xeb248000 protocol=0 location=0xc15a5308
1054 </para></listitem> 769 wget 1262 [001] 11624.859020: softirq_raise: vec=1 [action=TIMER]
1055 <listitem><para>If the change addresses a specific bug or issue that is 770 wget 1262 [001] 11624.859076: softirq_entry: vec=1 [action=TIMER]
1056 associated with a bug-tracking ID, include a reference to that ID in 771 wget 1262 [001] 11624.859083: softirq_exit: vec=1 [action=TIMER]
1057 your detailed description. 772 wget 1262 [001] 11624.859167: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
1058 For example, the Yocto Project uses a specific convention for bug 773 wget 1262 [001] 11624.859192: sys_exit_read: 0x1d7
1059 references - any commit that addresses a specific bug should include the 774 wget 1262 [001] 11624.859228: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
1060 bug ID in the description (typically at the beginning) as follows: 775 wget 1262 [001] 11624.859233: sys_exit_read: 0x0
1061 <literallayout class='monospaced'> 776 wget 1262 [001] 11624.859573: sys_enter_read: fd: 0x0003, buf: 0xbf82c580, count: 0x0200
1062 [YOCTO #&lt;bug-id&gt;] 777 wget 1262 [001] 11624.859584: sys_exit_read: 0x200
1063 778 wget 1262 [001] 11624.859864: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
1064 &lt;detailed description of change&gt; 779 wget 1262 [001] 11624.859888: sys_exit_read: 0x400
1065 </literallayout></para></listitem> 780 wget 1262 [001] 11624.859935: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
1066 Where &lt;bug-id&gt; is replaced with the specific bug ID from the 781 wget 1262 [001] 11624.859944: sys_exit_read: 0x400
1067 Yocto Project Bugzilla instance. 782 </literallayout>
1068 </itemizedlist> 783 This gives us a detailed timestamped sequence of events that
1069 </para> 784 occurred within the workload with respect to those events.
785 </para>
1070 786
1071 <para> 787 <para>
1072 You can find more guidance on creating well-formed commit messages at this OpenEmbedded 788 In many ways, profiling can be viewed as a subset of tracing -
1073 wiki page: 789 theoretically, if you have a set of trace events that's sufficient
1074 <ulink url='&OE_HOME_URL;/wiki/Commit_Patch_Message_Guidelines'></ulink>. 790 to capture all the important aspects of a workload, you can derive
1075 </para> 791 any of the results or views that a profiling run can.
792 </para>
1076 793
1077 <para> 794 <para>
1078 Following are general instructions for both pushing changes upstream and for submitting 795 Another aspect of traditional profiling is that while powerful in
1079 changes as patches. 796 many ways, it's limited by the granularity of the underlying data.
1080 </para> 797 Profiling tools offer various ways of sorting and presenting the
798 sample data, which make it much more useful and amenable to user
799 experimentation, but in the end it can't be used in an open-ended
800 way to extract data that just isn't present as a consequence of
801 the fact that conceptually, most of it has been thrown away.
802 </para>
1081 803
1082 <section id='pushing-a-change-upstream'> 804 <para>
1083 <title>Using Scripts to Push a Change Upstream and Request a Pull</title> 805 Full-blown detailed tracing data does however offer the opportunity
806 to manipulate and present the information collected during a
807 tracing run in an infinite variety of ways.
808 </para>
1084 809
1085 <para> 810 <para>
1086 The basic flow for pushing a change to an upstream "contrib" Git repository is as follows: 811 Another way to look at it is that there are only so many ways that
1087 <itemizedlist> 812 the 'primitive' counters can be used on their own to generate
1088 <listitem><para>Make your changes in your local Git repository.</para></listitem> 813 interesting output; to get anything more complicated than simple
1089 <listitem><para>Stage your changes by using the <filename>git add</filename> 814 counts requires some amount of additional logic, which is typically
1090 command on each file you changed.</para></listitem> 815 very specific to the problem at hand. For example, if we wanted to
1091 <listitem><para>Commit the change by using the <filename>git commit</filename> 816 make use of a 'counter' that maps to the value of the time
1092 command and push it to the "contrib" repository. 817 difference between when a process was scheduled to run on a
1093 Be sure to provide a commit message that follows the project’s commit message standards 818 processor and the time it actually ran, we wouldn't expect such
1094 as described earlier.</para></listitem> 819 a counter to exist on its own, but we could derive one called say
1095 <listitem><para>Notify the maintainer that you have pushed a change by making a pull 820 'wakeup_latency' and use it to extract a useful view of that metric
1096 request. 821 from trace data. Likewise, we really can't figure out from standard
1097 The Yocto Project provides two scripts that conveniently let you generate and send 822 profiling tools how much data every process on the system reads and
1098 pull requests to the Yocto Project. 823 writes, along with how many of those reads and writes fail
1099 These scripts are <filename>create-pull-request</filename> and 824 completely. If we have sufficient trace data, however, we could
1100 <filename>send-pull-request</filename>. 825 with the right tools easily extract and present that information,
1101 You can find these scripts in the <filename>scripts</filename> directory 826 but we'd need something other than pre-canned profiling tools to
1102 within the <link linkend='source-directory'>Source Directory</link>.</para> 827 do that.
1103 <para>Using these scripts correctly formats the requests without introducing any
1104 whitespace or HTML formatting.
1105 The maintainer that receives your patches needs to be able to save and apply them
1106 directly from your emails.
1107 Using these scripts is the preferred method for sending patches.</para>
1108 <para>For help on using these scripts, simply provide the
1109 <filename>-h</filename> argument as follows:
1110 <literallayout class='monospaced'>
1111 $ ~/poky/scripts/create-pull-request -h
1112 $ ~/poky/scripts/send-pull-request -h
1113 </literallayout></para></listitem>
1114 </itemizedlist>
1115 </para> 828 </para>
1116 829
1117 <para> 830 <para>
1118 You can find general Git information on how to push a change upstream in the 831 Luckily, there is general-purpose way to handle such needs,
1119 <ulink url='http://book.git-scm.com/3_distributed_workflows.html'>Git Community Book</ulink>. 832 called 'programming languages'. Making programming languages
833 easily available to apply to such problems given the specific
834 format of data is called a 'programming language binding' for
835 that data and language. Perf supports two programming language
836 bindings, one for Python and one for Perl.
1120 </para> 837 </para>
1121 </section>
1122 838
1123 <section id='submitting-a-patch'> 839 <note>
1124 <title>Using Email to Submit a Patch</title> 840 Tying It Together: Language bindings for manipulating and
841 aggregating trace data are of course not a new
842 idea. One of the first projects to do this was IBM's DProbes
843 dpcc compiler, an ANSI C compiler which targeted a low-level
844 assembly language running on an in-kernel interpreter on the
845 target system. This is exactly analagous to what Sun's DTrace
846 did, except that DTrace invented its own language for the purpose.
847 Systemtap, heavily inspired by DTrace, also created its own
848 one-off language, but rather than running the product on an
849 in-kernel interpreter, created an elaborate compiler-based
850 machinery to translate its language into kernel modules written
851 in C.
852 </note>
853
854 <para>
855 Now that we have the trace data in perf.data, we can use
856 'perf script -g' to generate a skeleton script with handlers
857 for the read/write entry/exit events we recorded:
858 <literallayout class='monospaced'>
859 root@crownbay:~# perf script -g python
860 generated Python script: perf-script.py
861 </literallayout>
862 The skeleton script simply creates a python function for each
863 event type in the perf.data file. The body of each function simply
864 prints the event name along with its parameters. For example:
865 <literallayout class='monospaced'>
866 def net__netif_rx(event_name, context, common_cpu,
867 common_secs, common_nsecs, common_pid, common_comm,
868 skbaddr, len, name):
869 print_header(event_name, common_cpu, common_secs, common_nsecs,
870 common_pid, common_comm)
871
872 print "skbaddr=%u, len=%u, name=%s\n" % (skbaddr, len, name),
873 </literallayout>
874 We can run that script directly to print all of the events
875 contained in the perf.data file:
876 <literallayout class='monospaced'>
877 root@crownbay:~# perf script -s perf-script.py
878
879 in trace_begin
880 syscalls__sys_exit_read 0 11624.857082795 1262 perf nr=3, ret=0
881 sched__sched_wakeup 0 11624.857193498 1262 perf comm=migration/0, pid=6, prio=0, success=1, target_cpu=0
882 irq__softirq_raise 1 11624.858021635 1262 wget vec=TIMER
883 irq__softirq_entry 1 11624.858074075 1262 wget vec=TIMER
884 irq__softirq_exit 1 11624.858081389 1262 wget vec=TIMER
885 syscalls__sys_enter_read 1 11624.858166434 1262 wget nr=3, fd=3, buf=3213019456, count=512
886 syscalls__sys_exit_read 1 11624.858177924 1262 wget nr=3, ret=512
887 skb__kfree_skb 1 11624.858878188 1262 wget skbaddr=3945041280, location=3243922184, protocol=0
888 skb__kfree_skb 1 11624.858945608 1262 wget skbaddr=3945037824, location=3243922184, protocol=0
889 irq__softirq_raise 1 11624.859020942 1262 wget vec=TIMER
890 irq__softirq_entry 1 11624.859076935 1262 wget vec=TIMER
891 irq__softirq_exit 1 11624.859083469 1262 wget vec=TIMER
892 syscalls__sys_enter_read 1 11624.859167565 1262 wget nr=3, fd=3, buf=3077701632, count=1024
893 syscalls__sys_exit_read 1 11624.859192533 1262 wget nr=3, ret=471
894 syscalls__sys_enter_read 1 11624.859228072 1262 wget nr=3, fd=3, buf=3077701632, count=1024
895 syscalls__sys_exit_read 1 11624.859233707 1262 wget nr=3, ret=0
896 syscalls__sys_enter_read 1 11624.859573008 1262 wget nr=3, fd=3, buf=3213018496, count=512
897 syscalls__sys_exit_read 1 11624.859584818 1262 wget nr=3, ret=512
898 syscalls__sys_enter_read 1 11624.859864562 1262 wget nr=3, fd=3, buf=3077701632, count=1024
899 syscalls__sys_exit_read 1 11624.859888770 1262 wget nr=3, ret=1024
900 syscalls__sys_enter_read 1 11624.859935140 1262 wget nr=3, fd=3, buf=3077701632, count=1024
901 syscalls__sys_exit_read 1 11624.859944032 1262 wget nr=3, ret=1024
902 </literallayout>
903 That in itself isn't very useful; after all, we can accomplish
904 pretty much the same thing by simply running 'perf script'
905 without arguments in the same directory as the perf.data file.
906 </para>
1125 907
1126 <para> 908 <para>
1127 You can submit patches without using the <filename>create-pull-request</filename> and 909 We can however replace the print statements in the generated
1128 <filename>send-pull-request</filename> scripts described in the previous section. 910 function bodies with whatever we want, and thereby make it
1129 Keep in mind, the preferred method is to use the scripts, however. 911 infinitely more useful.
1130 </para> 912 </para>
1131 913
1132 <para> 914 <para>
1133 Depending on the components changed, you need to submit the email to a specific 915 As a simple example, let's just replace the print statements in
1134 mailing list. 916 the function bodies with a simple function that does nothing but
1135 For some guidance on which mailing list to use, see the list in the 917 increment a per-event count. When the program is run against a
1136 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" section 918 perf.data file, each time a particular event is encountered,
1137 earlier in this manual. 919 a tally is incremented for that event. For example:
1138 For a description of the available mailing lists, see 920 <literallayout class='monospaced'>
1139 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>" 921 def net__netif_rx(event_name, context, common_cpu,
1140 section in the Yocto Project Reference Manual. 922 common_secs, common_nsecs, common_pid, common_comm,
923 skbaddr, len, name):
924 inc_counts(event_name)
925 </literallayout>
926 Each event handler function in the generated code is modified
927 to do this. For convenience, we define a common function called
928 inc_counts() that each handler calls; inc_counts simply tallies
929 a count for each event using the 'counts' hash, which is a
930 specialized has function that does Perl-like autovivification, a
931 capability that's extremely useful for kinds of multi-level
932 aggregation commonly used in processing traces (see perf's
933 documentation on the Python language binding for details):
934 <literallayout class='monospaced'>
935 counts = autodict()
936
937 def inc_counts(event_name):
938 try:
939 counts[event_name] += 1
940 except TypeError:
941 counts[event_name] = 1
942 </literallayout>
943 Finally, at the end of the trace processing run, we want to
944 print the result of all the per-event tallies. For that, we
945 use the special 'trace_end()' function:
946 <literallayout class='monospaced'>
947 def trace_end():
948 for event_name, count in counts.iteritems():
949 print "%-40s %10s\n" % (event_name, count)
950 </literallayout>
951 The end result is a summary of all the events recorded in the
952 trace:
953 <literallayout>
954 skb__skb_copy_datagram_iovec 13148
955 irq__softirq_entry 4796
956 irq__irq_handler_exit 3805
957 irq__softirq_exit 4795
958 syscalls__sys_enter_write 8990
959 net__net_dev_xmit 652
960 skb__kfree_skb 4047
961 sched__sched_wakeup 1155
962 irq__irq_handler_entry 3804
963 irq__softirq_raise 4799
964 net__net_dev_queue 652
965 syscalls__sys_enter_read 17599
966 net__netif_receive_skb 1743
967 syscalls__sys_exit_read 17598
968 net__netif_rx 2
969 napi__napi_poll 1877
970 syscalls__sys_exit_write 8990
971 </literallayout>
972 Note that this is pretty much exactly the same information we get
973 from 'perf stat', which goes a little way to support the idea
974 mentioned previously that given the right kind of trace data,
975 higher-level profiling-type summaries can be derived from it.
1141 </para> 976 </para>
1142 977
1143 <para> 978 <para>
1144 Here is the general procedure on how to submit a patch through email without using the 979 Documentation on using the
1145 scripts: 980 <ulink url='http://linux.die.net/man/1/perf-script-python'>'perf script' python binding</ulink>.
1146 <itemizedlist>
1147 <listitem><para>Make your changes in your local Git repository.</para></listitem>
1148 <listitem><para>Stage your changes by using the <filename>git add</filename>
1149 command on each file you changed.</para></listitem>
1150 <listitem><para>Commit the change by using the
1151 <filename>git commit --signoff</filename> command.
1152 Using the <filename>--signoff</filename> option identifies you as the person
1153 making the change and also satisfies the Developer's Certificate of
1154 Origin (DCO) shown earlier.</para>
1155 <para>When you form a commit you must follow certain standards established by the
1156 Yocto Project development team.
1157 See the earlier section
1158 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1159 for Yocto Project commit message standards.</para></listitem>
1160 <listitem><para>Format the commit into an email message.
1161 To format commits, use the <filename>git format-patch</filename> command.
1162 When you provide the command, you must include a revision list or a number of patches
1163 as part of the command.
1164 For example, these two commands each take the most recent single commit and
1165 format it as an email message in the current directory:
1166 <literallayout class='monospaced'>
1167 $ git format-patch -1
1168 $ git format-patch HEAD~
1169 </literallayout></para>
1170 <para>After the command is run, the current directory contains a
1171 numbered <filename>.patch</filename> file for the commit.</para>
1172 <para>If you provide several commits as part of the command,
1173 the <filename>git format-patch</filename> command produces a numbered
1174 series of files in the current directory – one for each commit.
1175 If you have more than one patch, you should also use the
1176 <filename>--cover</filename> option with the command, which generates a
1177 cover letter as the first "patch" in the series.
1178 You can then edit the cover letter to provide a description for
1179 the series of patches.
1180 For information on the <filename>git format-patch</filename> command,
1181 see <filename>GIT_FORMAT_PATCH(1)</filename> displayed using the
1182 <filename>man git-format-patch</filename> command.</para>
1183 <note>If you are or will be a frequent contributor to the Yocto Project
1184 or to OpenEmbedded, you might consider requesting a contrib area and the
1185 necessary associated rights.</note></listitem>
1186 <listitem><para>Import the files into your mail client by using the
1187 <filename>git send-email</filename> command.
1188 <note>In order to use <filename>git send-email</filename>, you must have the
1189 the proper Git packages installed.
1190 For Ubuntu and Fedora the package is <filename>git-email</filename>.</note></para>
1191 <para>The <filename>git send-email</filename> command sends email by using a local
1192 or remote Mail Transport Agent (MTA) such as
1193 <filename>msmtp</filename>, <filename>sendmail</filename>, or through a direct
1194 <filename>smtp</filename> configuration in your Git <filename>config</filename>
1195 file.
1196 If you are submitting patches through email only, it is very important
1197 that you submit them without any whitespace or HTML formatting that
1198 either you or your mailer introduces.
1199 The maintainer that receives your patches needs to be able to save and
1200 apply them directly from your emails.
1201 A good way to verify that what you are sending will be applicable by the
1202 maintainer is to do a dry run and send them to yourself and then
1203 save and apply them as the maintainer would.</para>
1204 <para>The <filename>git send-email</filename> command is the preferred method
1205 for sending your patches since there is no risk of compromising whitespace
1206 in the body of the message, which can occur when you use your own mail client.
1207 The command also has several options that let you
1208 specify recipients and perform further editing of the email message.
1209 For information on how to use the <filename>git send-email</filename> command,
1210 use the <filename>man git-send-email</filename> command.</para></listitem>
1211 </itemizedlist>
1212 </para> 981 </para>
1213 </section> 982 </section>
983
984
985
986
987
1214</section> 988</section>
1215</chapter> 989</chapter>
1216<!-- 990<!--
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-16 23:30:55 +0000