summaryrefslogtreecommitdiffstats
path: root/documentation/test-manual
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/test-manual')
-rw-r--r--documentation/test-manual/test-manual.html832
-rw-r--r--documentation/test-manual/test-manual.tgzbin73915 -> 0 bytes
2 files changed, 0 insertions, 832 deletions
diff --git a/documentation/test-manual/test-manual.html b/documentation/test-manual/test-manual.html
deleted file mode 100644
index 8896857bb8..0000000000
--- a/documentation/test-manual/test-manual.html
+++ /dev/null
@@ -1,832 +0,0 @@
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Yocto Project Test Environment Manual</title><link rel="stylesheet" type="text/css" href="test-manual-style.css" /><meta name="generator" content="DocBook XSL Stylesheets V1.76.1" /></head><body><div xml:lang="en" class="book" title="Yocto Project Test Environment Manual" id="test-manual" lang="en"><div class="titlepage"><div><div><h1 class="title">
3 Yocto Project Test Environment Manual
4 </h1></div><div><div class="authorgroup">
5 <div class="author"><h3 class="author"><span class="firstname">Scott</span> <span class="surname">Rifenbark</span></h3><div class="affiliation">
6 <span class="orgname">Scotty's Documentation Services, INC<br /></span>
7 </div><code class="email">&lt;<a class="email" href="mailto:srifenbark@gmail.com">srifenbark@gmail.com</a>&gt;</code></div>
8 </div></div><div><p class="copyright">Copyright © 2010-2018 Linux Foundation</p></div><div><div class="legalnotice" title="Legal Notice"><a id="idm45709743826400"></a>
9 <p>
10 Permission is granted to copy, distribute and/or modify this document under
11 the terms of the <a class="ulink" href="http://creativecommons.org/licenses/by-sa/2.0/uk/" target="_top">
12 Creative Commons Attribution-Share Alike 2.0 UK: England &amp; Wales</a> as published by
13 Creative Commons.
14 </p>
15 <div class="note" title="Manual Notes" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Manual Notes</h3>
16 <div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
17 This version of the
18 <span class="emphasis"><em>Yocto Project Test Environment Manual</em></span>
19 is for the 2.6 release of the
20 Yocto Project.
21 To be sure you have the latest version of the manual
22 for this release, go to the
23 <a class="ulink" href="http://www.yoctoproject.org/documentation" target="_top">Yocto Project documentation page</a>
24 and select the manual from that site.
25 Manuals from the site are more up-to-date than manuals
26 derived from the Yocto Project released TAR files.
27 </p></li><li class="listitem"><p>
28 If you located this manual through a web search, the
29 version of the manual might not be the one you want
30 (e.g. the search might have returned a manual much
31 older than the Yocto Project version with which you
32 are working).
33 You can see all Yocto Project major releases by
34 visiting the
35 <a class="ulink" href="https://wiki.yoctoproject.org/wiki/Releases" target="_top">Releases</a>
36 page.
37 If you need a version of this manual for a different
38 Yocto Project release, visit the
39 <a class="ulink" href="http://www.yoctoproject.org/documentation" target="_top">Yocto Project documentation page</a>
40 and select the manual set by using the
41 "ACTIVE RELEASES DOCUMENTATION" or "DOCUMENTS ARCHIVE"
42 pull-down menus.
43 </p></li><li class="listitem"><p>
44 To report any inaccuracies or problems with this
45 manual, send an email to the Yocto Project
46 discussion group at
47 <code class="filename">yocto@yoctoproject.com</code> or log into
48 the freenode <code class="filename">#yocto</code> channel.
49 </p></li></ul></div>
50 </div>
51 </div></div><div><div class="revhistory"><table border="1" width="100%" summary="Revision history"><tr><th align="left" valign="top" colspan="2"><strong>Revision History</strong></th></tr>
52 <tr><td align="left">Revision 2.7</td><td align="left">TBD</td></tr><tr><td align="left" colspan="2">Released with the Yocto Project 2.7 Release.</td></tr>
53 </table></div></div></div><hr /></div><div class="toc"><p><strong>Table of Contents</strong></p><dl><dt><span class="chapter"><a href="#test-manual-intro">1. The Yocto Project Test Environment Manual</a></span></dt><dd><dl><dt><span class="section"><a href="#test-welcome">1.1. Welcome</a></span></dt><dt><span class="section"><a href="#test-yocto-project-autobuilder-overview">1.2. Yocto Project Autobuilder Overview</a></span></dt><dt><span class="section"><a href="#test-project-tests">1.3. Yocto Project Tests - Type Overview</a></span></dt><dt><span class="section"><a href="#test-test-mapping">1.4. How Tests Map to Areas of Code</a></span></dt><dt><span class="section"><a href="#test-examples">1.5. Test Examples</a></span></dt><dd><dl><dt><span class="section"><a href="#bitbake-selftest-example">1.5.1. <code class="filename">bitbake-selftest</code></a></span></dt><dt><span class="section"><a href="#oe-selftest-example">1.5.2. <code class="filename">oe-selftest</code></a></span></dt><dt><span class="section"><a href="#testimage-example">1.5.3. <code class="filename">testimage</code></a></span></dt><dt><span class="section"><a href="#testsdk_ext-example">1.5.4. <code class="filename">testsdk_ext</code></a></span></dt><dt><span class="section"><a href="#testsdk-example">1.5.5. <code class="filename">testsdk</code></a></span></dt><dt><span class="section"><a href="#oe-build-perf-test-example">1.5.6. <code class="filename">oe-build-perf-test</code></a></span></dt></dl></dd><dt><span class="section"><a href="#some-id">1.6. New Section on the Periodic Builds</a></span></dt><dt><span class="section"><a href="#test-configuring-and-triggering-autobuilder-helper-build-scripts">1.7. Configuring and Triggering Autobuilder Helper Build Scripts</a></span></dt><dt><span class="section"><a href="#test-deploying-yocto-autobuilder">1.8. Deploying Yocto Autobuilder</a></span></dt><dd><dl><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-controller">1.8.1. Upstream Autobuilder Deployment on the Controller</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-worker">1.8.2. Upstream Autobuilder Deployment on the Worker</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-no-upstream-users">1.8.3. Upstream Autobuilder Deployment No Upstream Users</a></span></dt></dl></dd><dt><span class="section"><a href="#test-headless-sanity-tests">1.9. Setting Up Headless Sanity Tests</a></span></dt><dt><span class="section"><a href="#test-adding-additional-build-workers">1.10. Adding Additional Build Workers</a></span></dt><dt><span class="section"><a href="#test-setting-up-build-history">1.11. Setting Up Build History</a></span></dt><dt><span class="section"><a href="#test-some-more-notes">1.12. Some More Notes</a></span></dt><dt><span class="section"><a href="#test-yocto-project-helper">1.13. Yocto Project Autobuilder Helper Scripts</a></span></dt></dl></dd></dl></div>
54
55
56 <div class="chapter" title="Chapter 1. The Yocto Project Test Environment Manual" id="test-manual-intro"><div class="titlepage"><div><div><h2 class="title">Chapter 1. The Yocto Project Test Environment Manual<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-manual-intro">¶</a></span></h2></div></div></div><div class="toc"><p><strong>Table of Contents</strong></p><dl><dt><span class="section"><a href="#test-welcome">1.1. Welcome</a></span></dt><dt><span class="section"><a href="#test-yocto-project-autobuilder-overview">1.2. Yocto Project Autobuilder Overview</a></span></dt><dt><span class="section"><a href="#test-project-tests">1.3. Yocto Project Tests - Type Overview</a></span></dt><dt><span class="section"><a href="#test-test-mapping">1.4. How Tests Map to Areas of Code</a></span></dt><dt><span class="section"><a href="#test-examples">1.5. Test Examples</a></span></dt><dd><dl><dt><span class="section"><a href="#bitbake-selftest-example">1.5.1. <code class="filename">bitbake-selftest</code></a></span></dt><dt><span class="section"><a href="#oe-selftest-example">1.5.2. <code class="filename">oe-selftest</code></a></span></dt><dt><span class="section"><a href="#testimage-example">1.5.3. <code class="filename">testimage</code></a></span></dt><dt><span class="section"><a href="#testsdk_ext-example">1.5.4. <code class="filename">testsdk_ext</code></a></span></dt><dt><span class="section"><a href="#testsdk-example">1.5.5. <code class="filename">testsdk</code></a></span></dt><dt><span class="section"><a href="#oe-build-perf-test-example">1.5.6. <code class="filename">oe-build-perf-test</code></a></span></dt></dl></dd><dt><span class="section"><a href="#some-id">1.6. New Section on the Periodic Builds</a></span></dt><dt><span class="section"><a href="#test-configuring-and-triggering-autobuilder-helper-build-scripts">1.7. Configuring and Triggering Autobuilder Helper Build Scripts</a></span></dt><dt><span class="section"><a href="#test-deploying-yocto-autobuilder">1.8. Deploying Yocto Autobuilder</a></span></dt><dd><dl><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-controller">1.8.1. Upstream Autobuilder Deployment on the Controller</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-worker">1.8.2. Upstream Autobuilder Deployment on the Worker</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-no-upstream-users">1.8.3. Upstream Autobuilder Deployment No Upstream Users</a></span></dt></dl></dd><dt><span class="section"><a href="#test-headless-sanity-tests">1.9. Setting Up Headless Sanity Tests</a></span></dt><dt><span class="section"><a href="#test-adding-additional-build-workers">1.10. Adding Additional Build Workers</a></span></dt><dt><span class="section"><a href="#test-setting-up-build-history">1.11. Setting Up Build History</a></span></dt><dt><span class="section"><a href="#test-some-more-notes">1.12. Some More Notes</a></span></dt><dt><span class="section"><a href="#test-yocto-project-helper">1.13. Yocto Project Autobuilder Helper Scripts</a></span></dt></dl></div><div class="section" title="1.1. Welcome"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-welcome">1.1. Welcome<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-welcome">¶</a></span></h2></div></div></div><p>
57 Welcome to the Yocto Project Test Environment Manual!
58 This manual is a work in progress.
59 The manual contains information about the testing environment
60 used by the Yocto Project to make sure each major and minor
61 release works as planned.
62 Other organizations can leverage off the process and testing
63 environment used by the Yocto Project to create their own
64 automated, production test environment.
65 </p><p>
66 Currently, the Yocto Project Test Environment Manual has no
67 projected release date.
68 This manual is a work-in-progress and is being initially loaded
69 with information from the README files and notes from key
70 engineers:
71 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
72 <span class="emphasis"><em><code class="filename">yocto-autobuilder</code>:</em></span>
73 This
74 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/README" target="_top"><code class="filename">README.md</code></a>
75 is not maintained.
76 However, some information from this README file still
77 applies but could need some modification.
78 In particular, information about setting up headless
79 sanity tests and build history.
80 The sections on these will be changing.
81 </p><div class="note" title="IMPORTANT" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">IMPORTANT</h3>
82 The <code class="filename">yocto-autobuilder</code> 
83 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/" target="_top">repository</a>
84 is obsolete and is no longer maintained.
85 The new "Autobuilder" is maintained in the
86 <code class="filename">yocto-autobuilder2</code> 
87 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/" target="_top">repository</a>.
88 </div><p>
89 </p></li><li class="listitem"><p>
90 <span class="emphasis"><em><code class="filename">yocto-autobuilder2</code>:</em></span>
91 This
92 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md" target="_top"><code class="filename">README.md</code></a>
93 is the main README for Yocto Project Autobuilder.
94 The <code class="filename">yocto-autobuilder2</code> repository
95 represents the Yocto Project's testing codebase and
96 exists to configure and use Buildbot to do testing.
97 </p></li><li class="listitem"><p>
98 <span class="emphasis"><em><code class="filename">yocto-autobuilder-helper</code>:</em></span>
99 This
100 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/README" target="_top"><code class="filename">README</code></a>
101 is a valid Autobuilder Git repository that contains
102 Yocto Project Autobuilder Helper Scripts.
103 The <code class="filename">yocto-autobuilder-helper</code>
104 repository contains the "glue" logic that connects any
105 Continuous Improvement (CI) system to run builds,
106 support getting the correct code revisions, configure
107 builds and layers, run builds, and collect results.
108 The code is independent of any CI system, which means
109 the code can work Buildbot, Jenkins, or others.
110 </p></li></ul></div><p>
111 </p></div><div class="section" title="1.2. Yocto Project Autobuilder Overview"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-yocto-project-autobuilder-overview">1.2. Yocto Project Autobuilder Overview<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-yocto-project-autobuilder-overview">¶</a></span></h2></div></div></div><p>
112 The Yocto Project Autobuilder collectively refers to the software,
113 tools, scripts, and procedures used by the Yocto Project to test
114 released software across supported hardware in an automated and
115 regular fashion.
116 Basically, during the development of a Yocto Project release, the
117 Autobuilder tests if things work.
118 The Autobuilder builds all test targets and runs all the tests.
119 </p><p>
120 The Yocto Project uses the unpatched
121 <a class="ulink" href="https://docs.buildbot.net/0.9.15.post1/" target="_top">Buildbot Nine</a>
122 to drive its integration and testing.
123 Buildbot Nine has a plug-in interface that the Yocto Project
124 customizes using code from the
125 <code class="filename">yocto-autobuilder2</code> repository.
126 The resulting customized UI plug-in allows you to
127 visualize builds in a way suited to the project.
128 </p><p>
129 A "helper" layer provides configuration and job management
130 through scripts found in the
131 <code class="filename">yocto-autobuilder-helper</code> repository.
132 The helper layer contains the bulk of the build configuration
133 information and is release-specific, which makes it highly
134 customizable on a per-project basis.
135 The layer is CI system-agnostic and contains a number of helper
136 scripts that can generate build configurations from simple JSON
137 files.
138 </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
139 It is possible to use the outer layers from another
140 Continuous Integration (CI) system such as
141 <a class="ulink" href="https://en.wikipedia.org/wiki/Jenkins_(software)" target="_top">Jenkins</a>
142 instead of Buildbot.
143 </div><p>
144 </p><p>
145 The following figure shows the Yocto Project Autobuilder stack
146 with a topology that includes a controller and a cluster of
147 workers:
148 </p><table border="0" summary="manufactured viewport for HTML img" cellspacing="0" cellpadding="0" width="720"><tr style="height: 540px"><td align="center"><img src="figures/ab-test-stack.png" align="middle" width="720" /></td></tr></table><p>
149 </p></div><div class="section" title="1.3. Yocto Project Tests - Type Overview"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-project-tests">1.3. Yocto Project Tests - Type Overview<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-project-tests">¶</a></span></h2></div></div></div><p>
150 Two kinds of tests exist within the Yocto Project:
151 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
152 <span class="emphasis"><em>Build Testing:</em></span>
153 Tests whether specific configurations build by varying
154 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-MACHINE" target="_top"><code class="filename">MACHINE</code></a>,
155 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-DISTRO" target="_top"><code class="filename">DISTRO</code></a>,
156 and the specific target images being built (or world).
157 </p></li><li class="listitem"><p>
158 <span class="emphasis"><em>Build Performance Testing:</em></span>
159 Tests whether or not commonly used steps during builds work
160 efficiently and avoid regressions.
161 </p></li></ul></div><p>
162 Beyond these types of testing, the Autobuilder tests different
163 pieces by using the following types of tests:
164 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
165 <span class="emphasis"><em>Build Testing:</em></span>
166 Trigger builds of all the different test configurations
167 on the Autobuilder.
168 Builds usually cover each target for different
169 architectures, machines, and distributions.
170 </p></li><li class="listitem"><p>
171 <span class="emphasis"><em>Build Performance Testing:</em></span>
172 Tests to time commonly used usage scenarios are run through
173 <code class="filename">oe-build-perf-test</code>.
174 </p></li><li class="listitem"><p>
175 <span class="emphasis"><em>eSDK Testing:</em></span>
176 Image tests initiated through the following command:
177 </p><pre class="literallayout">
178 $ bitbake <em class="replaceable"><code>image</code></em> -c testsdkext
179 </pre><p>
180 The tests utilize the <code class="filename">testsdkext</code>
181 class and the <code class="filename">do_testsdkext</code> task.
182 </p></li><li class="listitem"><p>
183 <span class="emphasis"><em>Feature Testing:</em></span>
184 Various scenario-based tests are run through the
185 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#testing-and-quality-assurance" target="_top">OpenEmbedded Self-Test</a>
186 (oe-selftest).
187 </p></li><li class="listitem"><p>
188 <span class="emphasis"><em>Image Testing:</em></span>
189 Image tests initiated through the following command:
190 </p><pre class="literallayout">
191 $ bitbake <em class="replaceable"><code>image</code></em> -c testimage
192 </pre><p>
193 The tests utilize the
194 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-classes-testimage*" target="_top"><code class="filename">testimage*</code></a>
195 classes and the
196 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-tasks-testimage" target="_top"><code class="filename">do_testimage</code></a>
197 task.
198 </p></li><li class="listitem"><p>
199 <span class="emphasis"><em>Package Testing:</em></span>
200 A Package Test (ptest) runs tests against packages built
201 by the OpenEmbedded build system on the target machine.
202 See the
203 "<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/dev-manual/dev-manual.html#testing-packages-with-ptest" target="_top">Testing Packages With ptest</a>"
204 section in the Yocto Project Development Tasks Manual and
205 the
206 "<a class="ulink" href="https://wiki.yoctoproject.org/wiki/Ptest" target="_top">Ptest</a>"
207 Wiki page for more information on Ptest.
208 </p></li><li class="listitem"><p>
209 <span class="emphasis"><em>Sanity Checks During the Build Process:</em></span>
210 Tests initiated through the
211 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-classes-insane" target="_top"><code class="filename">insane</code></a>
212 class.
213 </p></li><li class="listitem"><p>
214 <span class="emphasis"><em>SDK Testing:</em></span>
215 Image tests initiated through the following command:
216 </p><pre class="literallayout">
217 $ bitbake <em class="replaceable"><code>image</code></em> -c testsdk
218 </pre><p>
219 The tests utilize the
220 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-classes-testsdk" target="_top"><code class="filename">testsdk</code></a>
221 class and the <code class="filename">do_testsdk</code> task.
222 </p></li><li class="listitem"><p>
223 <span class="emphasis"><em>Unit Testing:</em></span>
224 Unit tests on various components of the system run
225 through <code class="filename">oe-selftest</code> and
226 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#testing-and-quality-assurance" target="_top"><code class="filename">bitbake-selftest</code></a>.
227 </p></li></ul></div><p>
228 </p></div><div class="section" title="1.4. How Tests Map to Areas of Code"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-test-mapping">1.4. How Tests Map to Areas of Code<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-test-mapping">¶</a></span></h2></div></div></div><p>
229 Tests map into the codebase as follows:
230 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
231 <span class="emphasis"><em>bitbake-selftest:</em></span>
232 </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
233 These tests are self-contained and test BitBake
234 as well as its APIs, which include the fetchers.
235 The tests are located in
236 <code class="filename">bitbake/lib/*/tests</code>.
237 </p></li><li class="listitem"><p>
238 From within the BitBake repository, run the
239 following:
240 </p><pre class="literallayout">
241 $ bitbake-selftest
242 </pre><p>
243 </p></li><li class="listitem"><p>
244 The tests are based on
245 <a class="ulink" href="https://docs.python.org/3/library/unittest.html" target="_top">Python unittest</a>.
246 </p></li></ul></div><p>
247 </p></li><li class="listitem"><p>
248 <span class="emphasis"><em>oe-selftest:</em></span>
249 </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
250 These tests use OE to test the workflows, which
251 include testing specific features, behaviors
252 of tasks, and API unit tests.
253 The tests take advantage of parallelism through
254 the "-j" option to run in multiple threads.
255 </p></li><li class="listitem"><p>
256 The tests are based on Python unittest.
257 </p></li><li class="listitem"><p>
258 The code for the tests resides in
259 <code class="filename">meta/lib/oeqa/selftest</code>.
260 </p></li><li class="listitem"><p>
261 To run all the test, enter the following command:
262 </p><pre class="literallayout">
263 $ oe-selftest -a
264 </pre><p>
265 </p></li><li class="listitem"><p>
266 To run a specific test, use the following command
267 form where <em class="replaceable"><code>testname</code></em> is
268 the name of the specific test:
269 </p><pre class="literallayout">
270 $ oe-selftest -r <em class="replaceable"><code>testname</code></em>
271 </pre><p>
272 </p></li></ul></div><p>
273 </p></li><li class="listitem"><p>
274 <span class="emphasis"><em>testimage:</em></span>
275 </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
276 These tests build an image, boot it, and run tests
277 against the image's content.
278 </p></li><li class="listitem"><p>
279 The code for these tests resides in
280 <code class="filename">meta/lib/oeqa/runtime</code>.
281 </p></li><li class="listitem"><p>
282 You need to set the
283 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-IMAGE_CLASSES" target="_top"><code class="filename">IMAGE_CLASSES</code></a>
284 variable as follows:
285 </p><pre class="literallayout">
286 IMAGE_CLASSES += "testimage"
287 </pre><p>
288 </p></li><li class="listitem"><p>
289 Run the tests using the following command form:
290 </p><pre class="literallayout">
291 $ bitbake <em class="replaceable"><code>image</code></em> -c testimage
292 </pre><p>
293 </p></li></ul></div><p>
294 </p></li><li class="listitem"><p>
295 <span class="emphasis"><em>testsdk:</em></span>
296 </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
297 These tests build an SDK, install it, and then
298 run tests against that SDK.
299 </p></li><li class="listitem"><p>
300 The code for these tests resides in
301 <code class="filename">meta/lib/oeqa/sdk</code>.
302 </p></li><li class="listitem"><p>
303 Run the test using the following command form:
304 </p><pre class="literallayout">
305 $ bitbake <em class="replaceable"><code>image</code></em> -c testsdk
306 </pre><p>
307 </p></li></ul></div><p>
308 </p></li><li class="listitem"><p>
309 <span class="emphasis"><em>testsdk_ext:</em></span>
310 </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
311 These tests build an extended SDK (eSDK), install
312 that eSDK, and run tests against the eSDK.
313 </p></li><li class="listitem"><p>
314 The code for these tests resides in
315 <code class="filename">meta/lib/oeqa/esdk</code>.
316 </p></li><li class="listitem"><p>
317 To run the tests, use the following command form:
318 </p><pre class="literallayout">
319 $ bitbake <em class="replaceable"><code>image</code></em> -c testsdkext
320 </pre><p>
321 </p></li></ul></div><p>
322 </p></li><li class="listitem"><p>
323 <span class="emphasis"><em>oe-build-perf-test:</em></span>
324 </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
325 These tests run through commonly used usage
326 scenarios and measure the performance times.
327 </p></li><li class="listitem"><p>
328 The code for these tests resides in
329 NEED A DIRECTORY HERE.
330 </p></li><li class="listitem"><p>
331 NEED SOME INFORMATION ON HOW TO ENABLE THIS
332 TEST OR INCLUDE IT HERE.
333 </p><pre class="literallayout">
334 <em class="replaceable"><code>some setting</code></em>
335 </pre><p>
336 </p></li><li class="listitem"><p>
337 Run the tests using the following command form:
338 </p><pre class="literallayout">
339 $ <em class="replaceable"><code>some command</code></em>
340 </pre><p>
341 </p></li></ul></div><p>
342 </p></li></ul></div><p>
343 </p></div><div class="section" title="1.5. Test Examples"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-examples">1.5. Test Examples<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-examples">¶</a></span></h2></div></div></div><p>
344 This section provides example tests for each of the tests
345 listed in the
346 <a class="link" href="#test-test-mapping" title="1.4. How Tests Map to Areas of Code">How Tests Map to Areas of Code</a>"
347 section.
348 </p><div class="section" title="1.5.1. bitbake-selftest"><div class="titlepage"><div><div><h3 class="title" id="bitbake-selftest-example">1.5.1. <code class="filename">bitbake-selftest</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#bitbake-selftest-example">¶</a></span></h3></div></div></div><p>
349 Content here.
350 </p></div><div class="section" title="1.5.2. oe-selftest"><div class="titlepage"><div><div><h3 class="title" id="oe-selftest-example">1.5.2. <code class="filename">oe-selftest</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#oe-selftest-example">¶</a></span></h3></div></div></div><p>
351 NEED CONTENT HERE.
352 </p></div><div class="section" title="1.5.3. testimage"><div class="titlepage"><div><div><h3 class="title" id="testimage-example">1.5.3. <code class="filename">testimage</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#testimage-example">¶</a></span></h3></div></div></div><p>
353 NEED CONTENT HERE.
354 </p></div><div class="section" title="1.5.4. testsdk_ext"><div class="titlepage"><div><div><h3 class="title" id="testsdk_ext-example">1.5.4. <code class="filename">testsdk_ext</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#testsdk_ext-example">¶</a></span></h3></div></div></div><p>
355 NEED CONTENT HERE.
356 </p></div><div class="section" title="1.5.5. testsdk"><div class="titlepage"><div><div><h3 class="title" id="testsdk-example">1.5.5. <code class="filename">testsdk</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#testsdk-example">¶</a></span></h3></div></div></div><p>
357 NEED CONTENT HERE.
358 </p></div><div class="section" title="1.5.6. oe-build-perf-test"><div class="titlepage"><div><div><h3 class="title" id="oe-build-perf-test-example">1.5.6. <code class="filename">oe-build-perf-test</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#oe-build-perf-test-example">¶</a></span></h3></div></div></div><p>
359 NEED CONTENT HERE.
360 </p></div></div><div class="section" title="1.6. New Section on the Periodic Builds"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="some-id">1.6. New Section on the Periodic Builds<span class="permalink"><a alt="Permalink" title="Permalink" href="#some-id">¶</a></span></h2></div></div></div><p>
361 The following is going to be the replacement content for the
362 section on "Nightly Builds".
363 Not sure what we are going to call these builds.
364 We need a name to replace "Nightly Builds".
365 </p><p>
366 Here is the content from Richards email:
367 </p><pre class="literallayout">
368 In 1.6, we actually dropped the "nightly" bit pretty much everywhere.
369 They are now named MACHINE or MACHINE-DISTRO, e.g. qemuarm or qemuarm-
370 lsb (which tests poky-lsb with qemuarm). We now parallelise not just
371 architecture but by machine so machine and real hardware are now
372 separate. The flow is therefore to build the images+sdks, then test the
373 images+sdks, trying to do as much as possible in parallel.
374
375 We have two types of build trigger, "quick" and "full". quick runs all
376 the things which commonly fail and one random oe-selftest. "full" runs
377 all our targets, runs oe-selftest on all distros and includes ptest and
378 build performance tests. Its slower but more complete and would be used
379 for release builds.
380 </pre><p>
381 </p></div><div class="section" title="1.7. Configuring and Triggering Autobuilder Helper Build Scripts"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-configuring-and-triggering-autobuilder-helper-build-scripts">1.7. Configuring and Triggering Autobuilder Helper Build Scripts<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-configuring-and-triggering-autobuilder-helper-build-scripts">¶</a></span></h2></div></div></div><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
382 This section is created from the information in the
383 <code class="filename">yocto-autobuilder2</code> 
384 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md" target="_top"><code class="filename">README.md</code></a>
385 file.
386 I am making an assumption that we do not want to refer to the
387 Autobuilder stuff as "Autobuilder2".
388 My guess is that since this is the first documentation of any
389 automated test environment and process in the Yocto Project
390 user documentation, we will treat it as the start of things.
391 </div><p>
392 Automatic testing is based on the workers executing builds using
393 Buildbot Nine configured for specific build jobs triggered in an
394 automatic and regular fashion.
395 Worker Configuration and triggering is accomplished through
396 the
397 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/" target="_top">Yocto Project Autobuilder layer</a>
398 and a set of
399 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree" target="_top">helper scripts</a>.
400 </p><p>
401 The configuration and helper scripts have as little code and
402 as few custom Buildbot extensions as possible.
403 The configuration collects required input from the user to
404 furnish the helper scripts with the input needed for workers
405 to accomplish their builds.
406 The input consists of minimal user-customizable parameters
407 used to trigger the helper build scripts.
408 </p><p>
409 Each builder maps to a named configuration in the helper
410 scripts.
411 The configuration is created with the steps and properties
412 required to invoke the helper scripts for a worker's builds.
413 </p><p>
414 Each worker has a custom scheduler created for it and contains
415 parameters configured for the scheduler that can supply the custom
416 versions of the required values for the helper script parameters.
417 </p><p>
418 Following is the code layout for the Autobuilder:
419 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
420 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/builders.py" target="_top"><code class="filename">builders.py</code></a>:</em></span>
421 Configures the builders with the minimal buildsteps
422 to invoke the Yocto Project Autobuilder helper scripts.
423 </p></li><li class="listitem"><p>
424 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/lib/wiki.py" target="_top"><code class="filename">lib/wiki.py</code></a>:</em></span>
425 Implements functionality related to
426 <a class="ulink" href="https://www.mediawiki.org/wiki/MediaWiki" target="_top">MediaWiki</a>.
427 The <code class="filename">wikilog</code> plug-in uses this
428 functionality.
429 Effectively, this functionality provides helper functions
430 for the plug-in.
431 </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
432 Much of this code can be replaced by porting the
433 plug-in so that it is implemented as a
434 <code class="filename">buildbot.util.service.HTTPClient</code>.
435 </div><p>
436 </p></li><li class="listitem"><p>
437 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/reporters/wikilog.py" target="_top"><code class="filename">reporters/wikilog.py</code></a>:</em></span>
438 A custom plug-in that is a Buildbot service that listens for
439 build failures and then writes information about the
440 failure to the configured wiki page.
441 </p></li><li class="listitem"><p>
442 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/steps/writelayerinfo.py" target="_top"><code class="filename">steps/writelayerinfo.py</code></a>:</em></span>
443 Implements a simple, custom buildset that iterates the
444 <code class="filename">repo_</code>, <code class="filename">branch_</code>,
445 and <code class="filename">commit_</code> properties, which are set
446 by the schedulers, and then writes a JSON file with the
447 user's values.
448 </p></li><li class="listitem"><p>
449 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/config.py" target="_top"><code class="filename">config.py</code></a>:</em></span>
450 Contains all values that might need changing to redeploy
451 the Autobuilder code elsewhere.
452 </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
453 The redeployment goal has not been currently met.
454 </div><p>
455 </p></li><li class="listitem"><p>
456 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/master.cfg" target="_top"><code class="filename">master.cfg</code></a>:</em></span>
457 Performs most configuration by making calls into other
458 scripts.
459 Configuration specific for a worker cluster (i.e. a
460 Controller URL) resides here.
461 </p></li><li class="listitem"><p>
462 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/schedulers.py" target="_top"><code class="filename">schedulers.py</code></a>:</em></span>
463 Sets up the force schedulers with controls for modifying
464 inputs for each worker.
465 </p></li><li class="listitem"><p>
466 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/services.py" target="_top"><code class="filename">services.py</code></a>:</em></span>
467 Configures IRC, mail, and Wikilog reporters.
468 </p></li><li class="listitem"><p>
469 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/workers.py" target="_top"><code class="filename">workers.py</code></a>:</em></span>
470 Configures the worker objects.
471 </p></li><li class="listitem"><p>
472 <span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/www.py" target="_top"><code class="filename">www.py</code></a>:</em></span>
473 Sets up the Web User Interface.
474 </p></li></ul></div><p>
475 </p><p>
476 The goal is to keep custom code minimized throughout the
477 Autobuilder.
478 The few customizations implemented support the Yocto Project
479 Autobuilder Helper Script workflows and help replicate the
480 workflows established with the Yocto Autobuilder layer.
481 In particular, the following files accomplish this customization:
482 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
483 <code class="filename">writelayerinfo.py</code>
484 </p></li><li class="listitem"><p>
485 <code class="filename">wikilog.py</code>
486 </p></li><li class="listitem"><p>
487 <code class="filename">wiki.py</code>
488 </p></li></ul></div><p>
489 </p></div><div class="section" title="1.8. Deploying Yocto Autobuilder"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-deploying-yocto-autobuilder">1.8. Deploying Yocto Autobuilder<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-deploying-yocto-autobuilder">¶</a></span></h2></div></div></div><p>
490 Steps to deploy the Yocto Project Autobuilder assume each target
491 system has a copy of Buildbot installed.
492 Additionally, various pieces of functionality require that a copy
493 of the Autobuilder Helper Scripts (i.e.
494 <code class="filename">yocto-autobuilder-helper</code>) is available
495 in the home directory at
496 <code class="filename">~/yocto-autobuilder-helper</code> of the user
497 running Buildbot.
498 </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
499 If you are using a reverse proxy, be aware that modern
500 Buildbot uses a web socket for various communications between
501 the master and the web's User Interface.
502 Refer to the
503 <a class="ulink" href="http://docs.buildbot.net/latest/manual/cfg-www.html#reverse-proxy-configuration" target="_top">Buildbot documentation</a>
504 for information on how to correctly configure a reverse proxy.
505 </div><p>
506 </p><p>
507 The following sections provide steps for Yocto Autobuilder
508 deployment.
509 </p><div class="section" title="1.8.1. Upstream Autobuilder Deployment on the Controller"><div class="titlepage"><div><div><h3 class="title" id="test-upstream-autobuilder-deployment-on-the-controller">1.8.1. Upstream Autobuilder Deployment on the Controller<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-upstream-autobuilder-deployment-on-the-controller">¶</a></span></h3></div></div></div><p>
510 Follow these steps to deploy Yocto Autobuilder on an
511 upstream controller:
512 </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
513 <span class="emphasis"><em>Create the Master Yocto Controller</em></span>:
514 </p><pre class="literallayout">
515 $ buildbot create-master <em class="replaceable"><code>yocto-controller</code></em>
516 </pre><p>
517 </p></li><li class="listitem"><p>
518 <span class="emphasis"><em>Change Your Working Directory to the Master Yocto Controller</em></span>:
519 </p><pre class="literallayout">
520 $ cd <em class="replaceable"><code>yocto-controller</code></em>
521 </pre><p>
522 </p></li><li class="listitem"><p>
523 <span class="emphasis"><em>Create a Local Git Repository of the Yocto Project Autobuilder</em></span>:
524 </p><pre class="literallayout">
525 $ git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb
526 </pre><p>
527 In the previous command, the local repository is
528 created in a <code class="filename">yoctoabb</code>
529 directory inside the directory of the Master
530 Yocto Controller directory.
531 </p></li><li class="listitem"><p>
532 <span class="emphasis"><em>Change Your Working Directory Back to the Master Yocto Controller</em></span>:
533 </p><pre class="literallayout">
534 $ cd ..
535 </pre><p>
536 </p></li><li class="listitem"><p>
537 <span class="emphasis"><em>Create a Relative Symbolic Link to <code class="filename">master.cfg</code></em></span>:
538 </p><pre class="literallayout">
539 $ ln -rs <em class="replaceable"><code>yocto-controller</code></em>/yoctoabb/master.cfg <em class="replaceable"><code>yocto-controller</code></em>/master.cfg
540 </pre><p>
541 The previous command sets up a relative symbolic
542 link to the <code class="filename">master.cfg</code> using
543 a link of the same name.
544 </p></li><li class="listitem"><p>
545 <span class="emphasis"><em>Update the Buildbot URL in <code class="filename">master.cfg</code></em></span>:
546 Use your <code class="filename">$EDITOR</code> to edit the
547 Buildbot URL in the <code class="filename">master.cfg</code>
548 file.
549 Find the following line and replace the URL with
550 the URL for your Buildbot:
551 </p><pre class="literallayout">
552 c['buildbotURL'] = "https://autobuilder.yoctoproject.org/main/"
553 </pre><p>
554 </p></li><li class="listitem"><p>
555 <span class="emphasis"><em>Enable services in <code class="filename">services.py</code></em></span>:
556 Use your <code class="filename">$EDITOR</code> to edit the
557 <code class="filename">services.py</code> file.
558 Set appropriate configuration values to enable
559 desired services.
560 </p></li><li class="listitem"><p>
561 <span class="emphasis"><em>Enable Automatic Authorization (Autorisation) in <code class="filename">www.py</code></em></span>:
562 Use your <code class="filename">$EDITOR</code> to edit the
563 <code class="filename">www.py</code> file.
564 Configure autorisation if desired.
565 </p></li><li class="listitem"><p>
566 <span class="emphasis"><em>Modify Configuration Options in <code class="filename">config.py</code></em></span>:
567 Use your <code class="filename">$EDITOR</code> to edit the
568 <code class="filename">config.py</code> file.
569 Modify configuration options such as worker
570 configurations.
571 </p></li><li class="listitem"><p>
572 <span class="emphasis"><em>Start Buildbot</em></span>:
573 </p><pre class="literallayout">
574 $ buildbot start <em class="replaceable"><code>yocto-controller</code></em>
575 </pre><p>
576 </p></li><li class="listitem"><p>
577 <span class="emphasis"><em>Create a Local Git Repository of the Yocto Autobuilder Helper Scripts:</em></span>:
578 </p><pre class="literallayout">
579 Move up a directory so that you are above the
580 <em class="replaceable"><code>yocto-controller</code></em>
581 location and clone the directory:
582 $ cd ..
583 $ git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper
584 </pre><p>
585 </p></li></ol></div><p>
586 </p></div><div class="section" title="1.8.2. Upstream Autobuilder Deployment on the Worker"><div class="titlepage"><div><div><h3 class="title" id="test-upstream-autobuilder-deployment-on-the-worker">1.8.2. Upstream Autobuilder Deployment on the Worker<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-upstream-autobuilder-deployment-on-the-worker">¶</a></span></h3></div></div></div><p>
587 Follow these steps to deploy Yocto Autobuilder on an
588 upstream worker:
589 </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
590 <span class="emphasis"><em>Create the Worker</em></span>:
591 </p><pre class="literallayout">
592 $ buildbot-worker create-worker <em class="replaceable"><code>yocto-worker</code></em> <em class="replaceable"><code>localhost</code></em> <em class="replaceable"><code>example-worker</code></em> <em class="replaceable"><code>pass</code></em>
593 </pre><p>
594 </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
595 You do not have to hard-code the third
596 parameter (i.e.
597 <em class="replaceable"><code>example-worker</code></em>).
598 For example, you can pass
599 <code class="filename">`hostname`</code> to use the
600 host's configured name.
601 </div><p>
602 </p></li><li class="listitem"><p>
603 <span class="emphasis"><em>Start the Worker</em></span>:
604 </p><pre class="literallayout">
605 $ buildbot-worker start <em class="replaceable"><code>yocto-worker</code></em>
606 </pre><p>
607 </p></li></ol></div><p>
608 </p></div><div class="section" title="1.8.3. Upstream Autobuilder Deployment No Upstream Users"><div class="titlepage"><div><div><h3 class="title" id="test-upstream-autobuilder-deployment-no-upstream-users">1.8.3. Upstream Autobuilder Deployment No Upstream Users<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-upstream-autobuilder-deployment-no-upstream-users">¶</a></span></h3></div></div></div><p>
609 This case has yet to be defined.
610 It requires a custom <code class="filename">config.json</code> file
611 for <code class="filename">yocto-autobuilder-helper</code>.
612 </p></div></div><div class="section" title="1.9. Setting Up Headless Sanity Tests"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-headless-sanity-tests">1.9. Setting Up Headless Sanity Tests<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-headless-sanity-tests">¶</a></span></h2></div></div></div><p>
613 If you plan on using the Yocto Project Autobuilder to run
614 headless sanity testing, you need to do the following:
615 </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
616 Install
617 <a class="link" href="#test-tight-vnc">TightVNC</a>
618 client and server.
619 </p></li><li class="listitem"><p>
620 Create a bank of tap network devices (tap devs)
621 by running the
622 <code class="filename">runqemu-gen-tapdevs</code> script
623 found in the
624 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#source-directory" target="_top">Source Directory</a>
625 at
626 <a class="ulink" href="https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts" target="_top">https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts</a>.
627 </p><p>You must disable interface control on these
628 new tap devices.
629 </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
630 Some services include NetworkManager,
631 connman, or wicd.
632 </div><p>
633 </p></li><li class="listitem"><p>
634 Add "xterm*vt100*geometry: 80x50+10+10" to
635 <code class="filename">.Xdefaults</code>
636 </p></li><li class="listitem"><p>
637 Set up and start the TightVNC session as the
638 Autobuilder user.
639 </p></li><li class="listitem"><p>
640 Manually connect to the VNC session at least once
641 prior to running a QEMU sanity test.
642 </p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
643 Something is getting set during the initial
644 connection that has not been figured out yet.
645 Manually connecting seems to set up the session
646 correctly.
647 </div><p>
648 </p></li></ol></div><p>
649 </p></div><div class="section" title="1.10. Adding Additional Build Workers"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-adding-additional-build-workers">1.10. Adding Additional Build Workers<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-adding-additional-build-workers">¶</a></span></h2></div></div></div><p>
650 The production Yocto Autobuilder uses a cluster of build
651 workers.
652 The cluster shares the same
653 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-SSTATE_DIR" target="_top"><code class="filename">SSTATE_DIR</code></a>
654 and
655 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-DL_DIR" target="_top"><code class="filename">DL_DIR</code></a>
656 through an NFS4 mounted Network Attached Storage (NAS).
657 The main nightly trigger pre-populates the
658 <code class="filename">DL_DIR</code>, which allows the workers to not
659 have to deal with a lot of downloading.
660 In theory, you could also run your build workers with
661 <a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-NO_NETWORK" target="_top"><code class="filename">NO_NETWORK</code></a>
662 to enforce a single point for populating
663 <code class="filename">DL_DIR</code>.
664 </p><p>
665 Running multiple build workers is fairly simple, but does require
666 some setup:
667 </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
668 Ensure the settings in
669 <code class="filename">autobuilder.conf</code> are valid
670 for each worker.
671 Certain variables are set within this file that
672 work with the local configurations on each
673 worker.
674 </p></li><li class="listitem"><p>
675 Within
676 <code class="filename">yocto-controller/controller.cfg</code>,
677 add your worker to the
678 <code class="filename">c['workers']</code> list inside
679 the <code class="filename">BUILDWORKERS</code> section.
680 </p></li><li class="listitem"><p>
681 For each worker change the
682 <code class="filename">WORKER SETTINGS</code> section
683 of
684 <code class="filename">yocto-worker/buildbot.tac</code>
685 to match the settings in
686 <code class="filename">controller.cfg</code>.
687 </p></li><li class="listitem"><p>
688 Workers must reside in the same path as the
689 Build Controller, even if they are on
690 completely different machines.
691 </p></li></ol></div><p>
692 </p></div><div class="section" title="1.11. Setting Up Build History"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-setting-up-build-history">1.11. Setting Up Build History<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-setting-up-build-history">¶</a></span></h2></div></div></div><p>
693 Build History is used to track changes to packages and
694 images.
695 By default, the Autobuilder does not collect build history.
696 The production Autobuilder does have this functionality
697 enabled.
698 </p><p>
699 Setting up build history requires the following
700 steps:
701 </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
702 Create an empty Git repository.
703 Make a single commit to it and then create and
704 push branches for each of the nightly core
705 architectures (i.e.. mips, ppc, x86...).
706 </p></li><li class="listitem"><p>
707 Find a central location to create a clone for the
708 repository created in the previous step.
709 This works best if you have a setup similar to
710 the production Autobuilder (i.e. NAS with many
711 workers).
712 </p></li><li class="listitem"><p>
713 Run the following:
714 </p><pre class="literallayout">
715 # This is an example of how to set up a local build history checkout. Paths
716 # obviously are situationally dependent.
717 $ mkdir /nas/buildhistory
718 $ cd /nas/buildhistory
719 $ git clone ssh://git@git.myproject.org/buildhistory
720 $ git clone ssh://git@git.myproject.org/buildhistory nightly-arm
721 $ git clone ssh://git@git.myproject.org/buildhistory nightly-x86
722 $ git clone ssh://git@git.myproject.org/buildhistory nightly-x86-64
723 $ git clone ssh://git@git.myproject.org/buildhistory nightly-ppc
724 $ git clone ssh://git@git.myproject.org/buildhistory nightly-mips
725 $ for x in `ls|grep nightly` do cd $x; git checkout $x; cd /nas/buildhistory; done
726 </pre><p>
727 </p></li><li class="listitem"><p>
728 Within the <code class="filename">autobuilder.conf</code>
729 of each worker, change the following:
730 </p><pre class="literallayout">
731 BUILD_HISTORY_COLLECT = True
732 BUILD_HISTORY_DIR = "/nas/buildhistory"
733 BUILD_HISTORY_REPO = "ssh://git@git.myproject.org/buildhistory"
734 </pre><p>
735 </p></li></ol></div><p>
736 </p></div><div class="section" title="1.12. Some More Notes"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-some-more-notes">1.12. Some More Notes<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-some-more-notes">¶</a></span></h2></div></div></div><p>
737 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
738 <span class="emphasis"><em>Yocto Autobuilder</em></span>:
739 The Git repository is at
740 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/" target="_top">http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/</a>.
741 </p><p>Essentially an extension to the vanilla buildbot.
742 This extension mainly addresses configuration file handling
743 and Yocto-specific build steps.</p><p>For better maintainability, the Autobuilder (see
744 <code class="filename">Autobuilder.py</code> located at
745 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/lib/python2.7/site-packages/autobuilder" target="_top">http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/lib/python2.7/site-packages/autobuilder</a>),
746 handles configuration from multiple files.</p><p>Additional build steps such as
747 <code class="filename">CheckOutLayers.py</code> or
748 <code class="filename">CreateBBLayersConf</code> are Yocto-specific
749 and simplify the worker's configuration.
750 </p></li><li class="listitem"><p><a id="test-tight-vnc"></a>
751 <span class="emphasis"><em>TightVNC</em></span>:
752 Virtual Network Computing (VNC) is a client/server software
753 package that allows remote network access to graphical
754 desktops.
755 With VNC, you can access your machine from everywhere
756 provided that your machine is connected to the Internet.
757 VNC is free (released under the GNU General Public License)
758 and it is available on most platforms.</p><p>TightVNC is an enhanced version of VNC, which
759 includes new features, improvements, optimizations, and
760 bug fixes over the original VNC version.
761 See the list of features at
762 <a class="ulink" href="http://www.tightvnc.com/intro.php" target="_top">http://www.tightvnc.com/intro.php</a>.
763 </p><p>You need TightVNC in order to run headless sanity
764 tests.
765 See the bullet on
766 <a class="link" href="#test-headless-sanity-tests" title="1.9. Setting Up Headless Sanity Tests">headless sanity tests</a>
767 for more information.
768 </p></li><li class="listitem"><p>
769 <span class="emphasis"><em>Files Used for Yocto-Autobuilder Configuration:</em></span>
770 </p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
771 <span class="emphasis"><em><code class="filename">config/autobuilder.conf</code></em></span>:
772 Used to set Autobuilder-wide parameters, such as
773 where various build artifacts are published
774 (e.g. <code class="filename">DL_DIR</code> and
775 <code class="filename">SSTATE_DIR</code>).
776 Another example is if build artifacts should be
777 published, which is necessary for production
778 Autobuilders but not desktop builders.
779 </p></li><li class="listitem"><p>
780 <span class="emphasis"><em><code class="filename">buildset-config/yoctoAB.conf</code></em></span>:
781 The main Yocto Project Autobuilder configuration
782 file.
783 Documentation for this file and its associated
784 format is in the
785 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/README-NEW-AUTOBUILDER" target="_top"><code class="filename">README-NEW-AUTOBUILDER</code></a>
786 file.
787 </p></li></ul></div><p>
788 </p></li></ul></div><p>
789 </p></div><div class="section" title="1.13. Yocto Project Autobuilder Helper Scripts"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-yocto-project-helper">1.13. Yocto Project Autobuilder Helper Scripts<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-yocto-project-helper">¶</a></span></h2></div></div></div><div class="note" title="WRITER NOTE" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">WRITER NOTE</h3>
790 Deferring this topic per Richard's suggestion.
791 It is placed here temporarily.
792 </div><p>
793 The helper scripts work in conjunction with the Yocto Project
794 Autobuilder.
795 These scripts do the actual build configuration and execution
796 for tests on a per release basis.
797 </p><p>
798 You can use <code class="filename">pre-commit-hook.sh</code> to verify
799 the JSON file before committing it.
800 Create a symbolic link as follows:
801 </p><pre class="literallayout">
802 $ ln -s ../../scripts/pre-commit-hook.sh .git/hooks/pre-commit
803 </pre><p>
804 </p><p>
805 Most users will have to customize the helper script repository
806 to meet their needs.
807 The repository is located at
808 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper" target="_top">http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper</a>.
809 The scripts themselves should be more generically reusable.
810 The <code class="filename">config.json</code> is less reusable as it
811 represents the Yocto Project Autobuilder test matrix.
812 </p><p>
813 Two customization options are possible: 1) variable substitution,
814 and 2) overlaying configuration files.
815 The standard <code class="filename">config.json</code> minimally attempts
816 to allow substitution of the paths.
817 The helper script repository includes a
818 <a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/local-example.json" target="_top"><code class="filename">local-example.json</code></a>
819 to show how you could override these from a separate configuration
820 file.
821 Pass the following into the environment of the autobuilder:
822 </p><pre class="literallayout">
823 ABHELPER_JSON="config.json local-example.json"
824 </pre><p>
825 As another example, you could also pass the following into the
826 environment:
827 </p><pre class="literallayout">
828 ABHELPER_JSON="config.json /some/location/local.json"
829 </pre><p>
830 </p></div></div>
831
832</div></body></html> \ No newline at end of file
diff --git a/documentation/test-manual/test-manual.tgz b/documentation/test-manual/test-manual.tgz
deleted file mode 100644
index 58d2eced77..0000000000
--- a/documentation/test-manual/test-manual.tgz
+++ /dev/null
Binary files differ
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-16 16:37:15 +0000