summaryrefslogtreecommitdiffstats
path: root/documentation/test-manual
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/test-manual')
-rw-r--r--documentation/test-manual/test-manual-customization.xsl29
-rw-r--r--documentation/test-manual/test-manual-intro.xml624
-rw-r--r--documentation/test-manual/test-manual-style.css991
-rw-r--r--documentation/test-manual/test-manual-test-process.xml110
-rw-r--r--documentation/test-manual/test-manual-understand-autobuilder.xml314
-rwxr-xr-xdocumentation/test-manual/test-manual.xml104
6 files changed, 0 insertions, 2172 deletions
diff --git a/documentation/test-manual/test-manual-customization.xsl b/documentation/test-manual/test-manual-customization.xsl
deleted file mode 100644
index 17bf57c2d1..0000000000
--- a/documentation/test-manual/test-manual-customization.xsl
+++ /dev/null
@@ -1,29 +0,0 @@
1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
5
6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
7
8<!--
9
10 <xsl:import href="../template/1.76.1/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
11
12 <xsl:import href="http://docbook.sourceforge.net/release/xsl/1.76.1/xhtml/docbook.xsl" />
13
14-->
15
16 <xsl:include href="../template/permalinks.xsl"/>
17 <xsl:include href="../template/section.title.xsl"/>
18 <xsl:include href="../template/component.title.xsl"/>
19 <xsl:include href="../template/division.title.xsl"/>
20 <xsl:include href="../template/formal.object.heading.xsl"/>
21
22 <xsl:param name="html.stylesheet" select="'test-manual-style.css'" />
23 <xsl:param name="chapter.autolabel" select="1" />
24 <xsl:param name="appendix.autolabel" select="A" />
25 <xsl:param name="section.autolabel" select="1" />
26 <xsl:param name="section.label.includes.component.label" select="1" />
27 <xsl:param name="generate.id.attributes" select="1" />
28
29</xsl:stylesheet>
diff --git a/documentation/test-manual/test-manual-intro.xml b/documentation/test-manual/test-manual-intro.xml
deleted file mode 100644
index 0cdbee4d8e..0000000000
--- a/documentation/test-manual/test-manual-intro.xml
+++ /dev/null
@@ -1,624 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='test-manual-intro'>
7
8<title>The Yocto Project Test Environment Manual</title>
9 <section id='test-welcome'>
10 <title>Welcome</title>
11
12 <para> Welcome to the Yocto Project Test Environment Manual! This manual is a work in
13 progress. The manual contains information about the testing environment used by the
14 Yocto Project to make sure each major and minor release works as intended. All the
15 project's testing infrastructure and processes are publicly visible and available so
16 that the community can see what testing is being performed, how it's being done and the
17 current status of the tests and the project at any given time. It is intended that Other
18 organizations can leverage off the process and testing environment used by the Yocto
19 Project to create their own automated, production test environment, building upon the
20 foundations from the project core. </para>
21
22 <para> Currently, the Yocto Project Test Environment Manual has no projected release date.
23 This manual is a work-in-progress and is being initially loaded with information from
24 the <ulink url="">README</ulink> files and notes from key engineers: <itemizedlist>
25 <listitem>
26 <para>
27 <emphasis><filename>yocto-autobuilder2</filename>:</emphasis> This <ulink
28 url="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md"
29 ><filename>README.md</filename></ulink> is the main README which
30 detials how to set up the Yocto Project Autobuilder. The
31 <filename>yocto-autobuilder2</filename> repository represents the Yocto
32 Project's console UI plugin to Buildbot and the configuration necessary to
33 configure Buildbot to perform the testing the project requires. </para>
34 </listitem>
35 <listitem>
36 <para>
37 <emphasis><filename>yocto-autobuilder-helper</filename>:</emphasis> This
38 <ulink
39 url="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/README"
40 ><filename>README</filename></ulink> and repository contains Yocto
41 Project Autobuilder Helper scripts and configuration. The
42 <filename>yocto-autobuilder-helper</filename> repository contains the
43 "glue" logic that defines which tests to run and how to run them. As a
44 result, it can be used by any Continuous Improvement (CI) system to run
45 builds, support getting the correct code revisions, configure builds and
46 layers, run builds, and collect results. The code is independent of any CI
47 system, which means the code can work Buildbot, Jenkins, or others. This
48 repository has a branch per release of the project defining the tests to run
49 on a per release basis.</para>
50 </listitem>
51 </itemizedlist>
52 </para>
53 </section>
54
55 <section id='test-yocto-project-autobuilder-overview'>
56 <title>Yocto Project Autobuilder Overview</title>
57
58 <para>The Yocto Project Autobuilder collectively refers to the software, tools, scripts, and
59 procedures used by the Yocto Project to test released software across supported hardware
60 in an automated and regular fashion. Basically, during the development of a Yocto
61 Project release, the Autobuilder tests if things work. The Autobuilder builds all test
62 targets and runs all the tests. </para>
63
64 <para>The Yocto Project uses now uses standard upstream <ulink
65 url="https://docs.buildbot.net/0.9.15.post1/">Buildbot</ulink> (version 9) to drive
66 its integration and testing. Buildbot Nine has a plug-in interface that the Yocto
67 Project customizes using code from the <filename>yocto-autobuilder2</filename>
68 repository, adding its own console UI plugin. The resulting UI plug-in allows you to
69 visualize builds in a way suited to the project's needs.</para>
70
71 <para>A <filename>helper</filename> layer provides configuration and job management through
72 scripts found in the <filename>yocto-autobuilder-helper</filename> repository. The
73 <filename>helper</filename> layer contains the bulk of the build configuration
74 information and is release-specific, which makes it highly customizable on a per-project
75 basis. The layer is CI system-agnostic and contains a number of Helper scripts that can
76 generate build configurations from simple JSON files. <note>
77 <para>The project uses Buildbot for historical reasons but also because many of the
78 project developers have knowledge of python. It is possible to use the outer
79 layers from another Continuous Integration (CI) system such as <ulink
80 url="https://en.wikipedia.org/wiki/Jenkins_(software)">Jenkins</ulink>
81 instead of Buildbot. </para>
82 </note>
83 </para>
84
85 <para> The following figure shows the Yocto Project Autobuilder stack with a topology that
86 includes a controller and a cluster of workers: <imagedata
87 fileref="figures/ab-test-cluster.png" width="4.6in" depth="4.35in" align="center"
88 scalefit="1"/>
89 </para>
90 </section>
91
92 <section id='test-project-tests'>
93 <title>Yocto Project Tests - Types of Testing Overview</title>
94
95 <para>The Autobuilder tests different elements of the project by using thefollowing types of
96 tests: <itemizedlist>
97 <listitem>
98 <para>
99 <emphasis>Build Testing:</emphasis> Tests whether specific configurations
100 build by varying <ulink url="&YOCTO_DOCS_REF_URL;#var-MACHINE"
101 ><filename>MACHINE</filename></ulink>, <ulink
102 url="&YOCTO_DOCS_REF_URL;#var-DISTRO"
103 ><filename>DISTRO</filename></ulink>, other configuration options, and
104 the specific target images being built (or world). Used to trigger builds of
105 all the different test configurations on the Autobuilder. Builds usually
106 cover many different targets for different architectures, machines, and
107 distributions, as well as different configurations, such as different init
108 systems. The Autobuilder tests literally hundreds of configurations and
109 targets. <itemizedlist>
110 <listitem>
111 <para>
112 <emphasis>Sanity Checks During the Build Process:</emphasis>
113 Tests initiated through the <ulink
114 url="&YOCTO_DOCS_REF_URL;#ref-classes-insane"
115 ><filename>insane</filename></ulink> class. These checks
116 ensure the output of the builds are correct. For example, does
117 the ELF architecture in the generated binaries match the target
118 system? ARM binaries would not work in a MIPS system! </para>
119 </listitem>
120 </itemizedlist></para>
121 </listitem>
122 <listitem>
123 <para>
124 <emphasis>Build Performance Testing:</emphasis> Tests whether or not
125 commonly used steps during builds work efficiently and avoid regressions.
126 Tests to time commonly used usage scenarios are run through
127 <filename>oe-build-perf-test</filename>. These tests are run on isolated
128 machines so that the time measurements of the tests are accurate and no
129 other processes interfere with the timing results. The project currently
130 tests performance on two different distributions, Fedora and Ubuntu, to
131 ensure we have no single point of failure and can ensure the different
132 distros work effectively. </para>
133 </listitem>
134 <listitem>
135 <para>
136 <emphasis>eSDK Testing:</emphasis> Image tests initiated through the
137 following command:
138 <literallayout class="monospaced">
139 $ bitbake <replaceable>image</replaceable> -c testsdkext
140 </literallayout>
141 The tests utilize the <filename>testsdkext</filename> class and the
142 <filename>do_testsdkext</filename> task. </para>
143 </listitem>
144 <listitem>
145 <para>
146 <emphasis>Feature Testing:</emphasis> Various scenario-based tests are run
147 through the <ulink url="&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance"
148 >OpenEmbedded Self-Test</ulink> (oe-selftest). We test oe-selftest on
149 each of the main distrubutions we support. </para>
150 </listitem>
151 <listitem>
152 <para>
153 <emphasis>Image Testing:</emphasis> Image tests initiated through the
154 following command:
155 <literallayout class="monospaced">
156 $ bitbake <replaceable>image</replaceable> -c testimage
157 </literallayout>
158 The tests utilize the <ulink
159 url="&YOCTO_DOCS_REF_URL;#ref-classes-testimage*"
160 ><filename>testimage*</filename></ulink> classes and the <ulink
161 url="&YOCTO_DOCS_REF_URL;#ref-tasks-testimage"
162 ><filename>do_testimage</filename></ulink> task. </para>
163 </listitem>
164 <listitem>
165 <para>
166 <emphasis>Layer Testing:</emphasis> The Autobuilder has the possibility to
167 test whether specific layers work with the test of the system. The layers
168 tested may be selected by members of the project. Some key community layers
169 are also tested periodically.</para>
170 </listitem>
171 <listitem>
172 <para>
173 <emphasis>Package Testing:</emphasis> A Package Test (ptest) runs tests
174 against packages built by the OpenEmbedded build system on the target
175 machine. See the "<ulink
176 url="&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest">Testing Packages
177 With ptest</ulink>" section in the Yocto Project Development Tasks
178 Manual and the "<ulink url="&YOCTO_WIKI_URL;/wiki/Ptest">Ptest</ulink>" Wiki
179 page for more information on Ptest. </para>
180 </listitem>
181 <listitem>
182 <para>
183 <emphasis>SDK Testing:</emphasis> Image tests initiated through the
184 following command:
185 <literallayout class="monospaced">
186 $ bitbake <replaceable>image</replaceable> -c testsdk
187 </literallayout>
188 The tests utilize the <ulink url="&YOCTO_DOCS_REF_URL;#ref-classes-testsdk"
189 ><filename>testsdk</filename></ulink> class and the
190 <filename>do_testsdk</filename> task. </para>
191 </listitem>
192 <listitem>
193 <para>
194 <emphasis>Unit Testing:</emphasis> Unit tests on various components of the
195 system run through <filename>oe-selftest</filename> and <ulink
196 url="&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance"
197 ><filename>bitbake-selftest</filename></ulink>. </para>
198 </listitem>
199 <listitem>
200 <para>
201 <emphasis>Automatic Upgrade Helper:</emphasis> This target tests whether new
202 versions of software are available and whether we can automatically upgrade
203 to those new versions. If so, this target emails the maintainers with a
204 patch to let them know this is possible.</para>
205 </listitem>
206 </itemizedlist>
207 </para>
208 </section>
209
210 <section id='test-test-mapping'>
211 <title>How Tests Map to Areas of Code</title>
212
213 <para>
214 Tests map into the codebase as follows:
215 <itemizedlist>
216 <listitem><para>
217 <emphasis>bitbake-selftest</emphasis>: </para>
218 <para>These tests are self-contained and test BitBake as well as its APIs, which
219 include the fetchers. The tests are located in
220 <filename>bitbake/lib/*/tests</filename>. </para>
221 <para>From within the BitBake repository, run the following:
222 <literallayout class="monospaced">
223 $ bitbake-selftest
224 </literallayout>
225 </para>
226 <para>To skip tests that access the Internet, use the
227 <filename>BB_SKIP_NETTEST</filename> variable when running
228 "bitbake-selftest" as follows:
229 <literallayout class="monospaced">
230 $ BB_SKIP_NETTEST=yes bitbake-selftest
231 </literallayout></para>
232 <para>The default output is quiet and just prints a summary of what was run. To
233 see more information, there is a verbose
234 option:<literallayout class="monospaced">
235 $ bitbake-selftest -v
236 </literallayout></para>
237 <para>Use this option when you wish to skip tests that access the network, which
238 are mostly necessary to test the fetcher modules. To specify individual test
239 modules to run, append the test module name to the "bitbake-selftest"
240 command. For example, to specify the tests for the bb.data.module, run:
241 <literallayout class="monospaced">
242 $ bitbake-selftest bb.test.data.module
243 </literallayout>You
244 can also specify individual tests by defining the full name and module plus
245 the class path of the test, for example:
246 <literallayout class="monospaced">
247 $ bitbake-selftest bb.tests.data.TestOverrides.test_one_override
248 </literallayout></para>
249 <para>The tests are based on <ulink
250 url="https://docs.python.org/3/library/unittest.html">Python
251 unittest</ulink>. </para></listitem>
252 <listitem><para>
253 <emphasis>oe-selftest</emphasis>: <itemizedlist>
254 <listitem>
255 <para>These tests use OE to test the workflows, which include
256 testing specific features, behaviors of tasks, and API unit
257 tests. </para>
258 </listitem>
259 <listitem>
260 <para>The tests can take advantage of parallelism through the "-j"
261 option, which can specify a number of threads to spread the
262 tests across. Note that all tests from a given class of tests
263 will run in the same thread. To parallelize large numbers of
264 tests you can split the class into multiple units.</para>
265 </listitem>
266 <listitem>
267 <para>The tests are based on Python unittest. </para>
268 </listitem>
269 <listitem>
270 <para>The code for the tests resides in
271 <filename>meta/lib/oeqa/selftest/cases/</filename>. </para>
272 </listitem>
273 <listitem>
274 <para>To run all the tests, enter the following command:
275 <literallayout class="monospaced">
276 $ oe-selftest -a
277 </literallayout>
278 </para>
279 </listitem>
280 <listitem>
281 <para>To run a specific test, use the following command form where
282 <replaceable>testname</replaceable> is the name of the
283 specific test:
284 <literallayout class="monospaced">
285 $ oe-selftest -r <replaceable>testname</replaceable>
286 </literallayout>
287 For example, the following command would run the tinfoil getVar
288 API
289 test:<literallayout class="monospaced">
290 $ oe-selftest -r tinfoil.TinfoilTests.test_getvar
291 </literallayout>It
292 is also possible to run a set of tests. For example the
293 following command will run all of the tinfoil
294 tests:<literallayout class="monospaced">
295 $ oe-selftest -r tinfoil
296 </literallayout></para>
297 </listitem>
298 </itemizedlist>
299 </para></listitem>
300 <listitem><para>
301 <emphasis>testimage:</emphasis>
302 <itemizedlist>
303 <listitem><para>
304 These tests build an image, boot it, and run tests
305 against the image's content.
306 </para></listitem>
307 <listitem><para> The code for these tests resides in <filename>meta/lib/oeqa/runtime/cases/</filename>. </para></listitem>
308 <listitem><para>
309 You need to set the
310 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_CLASSES'><filename>IMAGE_CLASSES</filename></ulink>
311 variable as follows:
312 <literallayout class='monospaced'>
313 IMAGE_CLASSES += "testimage"
314 </literallayout>
315 </para></listitem>
316 <listitem><para>
317 Run the tests using the following command form:
318 <literallayout class='monospaced'>
319 $ bitbake <replaceable>image</replaceable> -c testimage
320 </literallayout>
321 </para></listitem>
322 </itemizedlist>
323 </para></listitem>
324 <listitem><para>
325 <emphasis>testsdk:</emphasis>
326 <itemizedlist>
327 <listitem><para>These tests build an SDK, install it, and then run tests against that SDK. </para></listitem>
328 <listitem><para>The code for these tests resides in <filename>meta/lib/oeqa/sdk/cases/</filename>. </para></listitem>
329 <listitem><para>Run the test using the following command form:
330 <literallayout class="monospaced">
331 $ bitbake <replaceable>image</replaceable> -c testsdk
332 </literallayout>
333 </para></listitem>
334 </itemizedlist>
335 </para></listitem>
336 <listitem><para>
337 <emphasis>testsdk_ext:</emphasis>
338 <itemizedlist>
339 <listitem><para>These tests build an extended SDK (eSDK), install that eSDK, and run tests against the eSDK. </para></listitem>
340 <listitem><para>The code for these tests resides in <filename>meta/lib/oeqa/esdk</filename>. </para></listitem>
341 <listitem><para>To run the tests, use the following command form:
342 <literallayout class="monospaced">
343 $ bitbake <replaceable>image</replaceable> -c testsdkext
344 </literallayout>
345 </para></listitem>
346 </itemizedlist>
347 </para></listitem>
348
349
350 <listitem><para>
351 <emphasis>oe-build-perf-test:</emphasis>
352 <itemizedlist>
353 <listitem><para>These tests run through commonly used usage scenarios and measure the performance times. </para></listitem>
354 <listitem><para>The code for these tests resides in <filename>meta/lib/oeqa/buildperf</filename>. </para></listitem>
355 <listitem><para>To run the tests, use the following command form:
356 <literallayout class="monospaced">
357 $ oe-build-perf-test <replaceable>options</replaceable>
358 </literallayout>The
359 command takes a number of options, such as where to place the
360 test results. The Autobuilder Helper Scripts include the
361 <filename>build-perf-test-wrapper</filename> script with
362 examples of how to use the oe-build-perf-test from the command
363 line.</para>
364 <para>Use the <filename>oe-git-archive</filename> command to store
365 test results into a Git repository. </para>
366 <para>Use the <filename>oe-build-perf-report</filename> command to
367 generate text reports and HTML reports with graphs of the
368 performance data. For examples, see <link linkend=""
369 >http://downloads.yoctoproject.org/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.html</link>
370 and <link linkend=""
371 >http://downloads.yoctoproject.org/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.txt</link>.</para></listitem>
372 <listitem>
373 <para>The tests are contained in
374 <filename>lib/oeqa/buildperf/test_basic.py</filename>.</para>
375 </listitem>
376 </itemizedlist>
377 </para></listitem>
378
379
380
381
382 </itemizedlist>
383 </para>
384 </section>
385
386 <section id='test-examples'>
387 <title>Test Examples</title>
388
389 <para>This section provides example tests for each of the tests listed in the <link
390 linkend="test-test-mapping">How Tests Map to Areas of Code</link> section. </para>
391 <para>For oeqa tests, testcases for each area reside in the main test directory at
392 <filename>meta/lib/oeqa/selftest/cases</filename> directory.</para>
393 <para>For oe-selftest. bitbake testcases reside in the <filename>lib/bb/tests/</filename>
394 directory. </para>
395
396 <section id='bitbake-selftest-example'>
397 <title><filename>bitbake-selftest</filename></title>
398
399 <para>A simple test example from <filename>lib/bb/tests/data.py</filename> is:
400 <literallayout class="monospaced">
401 class DataExpansions(unittest.TestCase):
402 def setUp(self):
403 self.d = bb.data.init()
404 self.d["foo"] = "value_of_foo"
405 self.d["bar"] = "value_of_bar"
406 self.d["value_of_foo"] = "value_of_'value_of_foo'"
407
408 def test_one_var(self):
409 val = self.d.expand("${foo}")
410 self.assertEqual(str(val), "value_of_foo")
411 </literallayout>
412 </para>
413 <para>In this example, a <ulink url=""><filename>DataExpansions</filename></ulink> class
414 of tests is created, derived from standard python unittest. The class has a common
415 <filename>setUp</filename> function which is shared by all the tests in the
416 class. A simple test is then added to test that when a variable is expanded, the
417 correct value is found.</para>
418 <para>Bitbake selftests are straightforward python unittest. Refer to the Python
419 unittest documentation for additional information on writing these tests at: <link
420 linkend="">https://docs.python.org/3/library/unittest.html</link>.</para>
421 </section>
422
423 <section id='oe-selftest-example'>
424 <title><filename>oe-selftest</filename></title>
425
426 <para>These tests are more complex due to the setup required behind the scenes for full
427 builds. Rather than directly using Python's unittest, the code wraps most of the
428 standard objects. The tests can be simple, such as testing a command from within the
429 OE build environment using the following
430 example:<literallayout class="monospaced">
431 class BitbakeLayers(OESelftestTestCase):
432 def test_bitbakelayers_showcrossdepends(self):
433 result = runCmd('bitbake-layers show-cross-depends')
434 self.assertTrue('aspell' in result.output, msg = "No dependencies
435 were shown. bitbake-layers show-cross-depends output:
436 %s"% result.output)
437 </literallayout></para>
438 <para>This example, taken from
439 <filename>meta/lib/oeqa/selftest/cases/bblayers.py</filename>, creates a
440 testcase from the <ulink url=""><filename>OESelftestTestCase</filename></ulink>
441 class, derived from <filename>unittest.TestCase</filename>, which runs the
442 <filename>bitbake-layers</filename> command and checks the output to ensure it
443 contains something we know should be here.</para>
444 <para>The <filename>oeqa.utils.commands</filename> module contains Helpers which can
445 assist with common tasks, including:<itemizedlist>
446 <listitem>
447 <para><emphasis>Obtaining the value of a bitbake variable:</emphasis> Use
448 <filename>oeqa.utils.commands.get_bb_var()</filename> or use
449 <filename>oeqa.utils.commands.get_bb_vars()</filename> for more than
450 one variable</para>
451 </listitem>
452 <listitem>
453 <para><emphasis>Running a bitbake invocation for a build:</emphasis> Use
454 <filename>oeqa.utils.commands.bitbake()</filename></para>
455 </listitem>
456 <listitem>
457 <para><emphasis>Running a command:</emphasis> Use
458 <filename>oeqa.utils.commandsrunCmd()</filename></para>
459 </listitem>
460 </itemizedlist></para>
461 <para>There is also a <filename>oeqa.utils.commands.runqemu()</filename> function for
462 launching the <filename>runqemu</filename> command for testing things within a
463 running, virtualized image.</para>
464 <para>You can run these tests in parallel. Parallelism works per test class, so tests
465 within a given test class should always run in the same build, while tests in
466 different classes or modules may be split into different builds. There is no data
467 store available for these tests since the tests launch the
468 <filename>bitbake</filename> command and exist outside of its context. As a
469 result, common bitbake library functions (bb.*) are also unavailable.</para>
470 </section>
471
472 <section id='testimage-example'>
473 <title><filename>testimage</filename></title>
474
475 <para>These tests are run once an image is up and running, either on target hardware or
476 under QEMU. As a result, they are assumed to be running in a target image
477 environment, as opposed to a host build environment. A simple example from
478 <filename>meta/lib/oeqa/runtime/cases/python.py</filename> contains the
479 following:<literallayout class="monospaced">
480 class PythonTest(OERuntimeTestCase):
481 @OETestDepends(['ssh.SSHTest.test_ssh'])
482 @OEHasPackage(['python3-core'])
483 def test_python3(self):
484 cmd = "python3 -c \"import codecs; print(codecs.encode('Uryyb,
485 jbeyq', 'rot13'))\""
486 status, output = self.target.run(cmd)
487 msg = 'Exit status was not 0. Output: %s' % output
488 self.assertEqual(status, 0, msg=msg)
489 </literallayout></para>
490 <para>In this example, the <ulink url=""><filename>OERuntimeTestCase</filename></ulink>
491 class wraps <filename>unittest.TestCase</filename>. Within the test,
492 <filename>self.target</filename> represents the target system, where commands
493 can be run on it using the <filename>run()</filename> method. </para>
494 <para>To ensure certain test or package dependencies are met, you can use the
495 <filename>OETestDepends</filename> and <filename>OEHasPackage</filename>
496 decorators. For example, the test in this example would only make sense if
497 python3-core is installed in the image.</para>
498 </section>
499
500 <section id='testsdk_ext-example'>
501 <title><filename>testsdk_ext</filename></title>
502
503 <para>These tests are run against built extensible SDKs (eSDKs). The tests can assume
504 that the eSDK environment has already been setup. An example from
505 <filename>meta/lib/oeqa/sdk/cases/devtool.py</filename> contains the
506 following:<literallayout class="monospaced">
507 class DevtoolTest(OESDKExtTestCase):
508 @classmethod
509 def setUpClass(cls):
510 myapp_src = os.path.join(cls.tc.esdk_files_dir, "myapp")
511 cls.myapp_dst = os.path.join(cls.tc.sdk_dir, "myapp")
512 shutil.copytree(myapp_src, cls.myapp_dst)
513 subprocess.check_output(['git', 'init', '.'], cwd=cls.myapp_dst)
514 subprocess.check_output(['git', 'add', '.'], cwd=cls.myapp_dst)
515 subprocess.check_output(['git', 'commit', '-m', "'test commit'"], cwd=cls.myapp_dst)
516
517 @classmethod
518 def tearDownClass(cls):
519 shutil.rmtree(cls.myapp_dst)
520 def _test_devtool_build(self, directory):
521 self._run('devtool add myapp %s' % directory)
522 try:
523 self._run('devtool build myapp')
524 finally:
525 self._run('devtool reset myapp')
526 def test_devtool_build_make(self):
527 self._test_devtool_build(self.myapp_dst)
528 </literallayout>In
529 this example, the <filename>devtool</filename> command is tested to see whether a
530 sample application can be built with the <filename>devtool build</filename> command
531 within the eSDK.</para>
532 </section>
533
534 <section id='testsdk-example'>
535 <title><filename>testsdk</filename></title>
536
537 <para>These tests are run against built SDKs. The tests can assume that an SDK has
538 already been extracted and its environment file has been sourced. A simple example
539 from <filename>meta/lib/oeqa/sdk/cases/python2.py</filename> contains the
540 following:<literallayout class="monospaced">
541 class Python3Test(OESDKTestCase):
542 def setUp(self):
543 if not (self.tc.hasHostPackage("nativesdk-python3-core") or
544 self.tc.hasHostPackage("python3-core-native")):
545 raise unittest.SkipTest("No python3 package in the SDK")
546
547 def test_python3(self):
548 cmd = "python3 -c \"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
549 output = self._run(cmd)
550 self.assertEqual(output, "Hello, world\n")
551 </literallayout>In
552 this example, if nativesdk-python3-core has been installed into the SDK, the code
553 runs the python3 interpreter with a basic command to check it is working correctly.
554 The test would only run if python3 is installed in the SDK.</para>
555 </section>
556
557 <section id='oe-build-perf-test-example'>
558 <title><filename>oe-build-perf-test</filename></title>
559
560 <para>The performance tests usually measure how long operations take and the resource
561 utilisation as that happens. An example from
562 <filename>meta/lib/oeqa/buildperf/test_basic.py</filename> contains the
563 following:<literallayout class="monospaced">
564 class Test3(BuildPerfTestCase):
565
566 def test3(self):
567 """Bitbake parsing (bitbake -p)"""
568 # Drop all caches and parse
569 self.rm_cache()
570 oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
571 self.measure_cmd_resources(['bitbake', '-p'], 'parse_1',
572 'bitbake -p (no caches)')
573 # Drop tmp/cache
574 oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
575 self.measure_cmd_resources(['bitbake', '-p'], 'parse_2',
576 'bitbake -p (no tmp/cache)')
577 # Parse with fully cached data
578 self.measure_cmd_resources(['bitbake', '-p'], 'parse_3',
579 'bitbake -p (cached)')
580 </literallayout>This
581 example shows how three specific parsing timings are measured, with and without
582 various caches, to show how BitBake's parsing performance trends over time.</para>
583 </section>
584 </section>
585 <section id='test-writing-considerations'>
586 <title>Considerations When Writing Tests</title>
587 <para>When writing good tests, there are several things to keep in mind. Since things
588 running on the Autobuilder are accessed concurrently by multiple workers, consider the
589 following:</para>
590 <formalpara>
591 <title>Running "cleanall" is not permitted</title>
592 <para>This can delete files from DL_DIR which would potentially break other builds
593 running in parallel. If this is required, DL_DIR must be set to an isolated
594 directory.</para>
595 </formalpara>
596 <formalpara>
597 <title>Running "cleansstate" is not permitted</title>
598 <para>This can delete files from SSTATE_DIR which would potentially break other builds
599 running in parallel. If this is required, SSTATE_DIR must be set to an isolated
600 directory. Alternatively, you can use the "-f" option with the
601 <filename>bitbake</filename> command to "taint" tasks by changing the sstate
602 checksums to ensure sstate cache items will not be reused.</para>
603 </formalpara>
604 <formalpara>
605 <title>Tests should not change the metadata</title>
606 <para>This is particularly true for oe-selftests since these can run in parallel and
607 changing metadata leads to changing checksums, which confuses BitBake while running
608 in parallel. If this is necessary, copy layers to a temporary location and modify
609 them. Some tests need to change metadata, such as the devtool tests. To prevent the
610 metadate from changes, set up temporary copies of that data first.</para>
611 </formalpara>
612 </section>
613
614
615
616
617
618
619
620
621</chapter>
622<!--
623vim: expandtab tw=80 ts=4
624-->
diff --git a/documentation/test-manual/test-manual-style.css b/documentation/test-manual/test-manual-style.css
deleted file mode 100644
index 10ee4c79cf..0000000000
--- a/documentation/test-manual/test-manual-style.css
+++ /dev/null
@@ -1,991 +0,0 @@
1/*
2 Generic XHTML / DocBook XHTML CSS Stylesheet.
3
4 SPDX-License-Identifier: CC-BY-2.0-UK
5
6 Browser wrangling and typographic design by
7 Oyvind Kolas / pippin@gimp.org
8
9 Customised for Poky by
10 Matthew Allum / mallum@o-hand.com
11
12 Thanks to:
13 Liam R. E. Quin
14 William Skaggs
15 Jakub Steiner
16
17 Structure
18 ---------
19
20 The stylesheet is divided into the following sections:
21
22 Positioning
23 Margins, paddings, width, font-size, clearing.
24 Decorations
25 Borders, style
26 Colors
27 Colors
28 Graphics
29 Graphical backgrounds
30 Nasty IE tweaks
31 Workarounds needed to make it work in internet explorer,
32 currently makes the stylesheet non validating, but up until
33 this point it is validating.
34 Mozilla extensions
35 Transparency for footer
36 Rounded corners on boxes
37
38*/
39
40
41 /*************** /
42 / Positioning /
43/ ***************/
44
45body {
46 font-family: Verdana, Sans, sans-serif;
47
48 min-width: 640px;
49 width: 80%;
50 margin: 0em auto;
51 padding: 2em 5em 5em 5em;
52 color: #333;
53}
54
55h1,h2,h3,h4,h5,h6,h7 {
56 font-family: Arial, Sans;
57 color: #00557D;
58 clear: both;
59}
60
61h1 {
62 font-size: 2em;
63 text-align: left;
64 padding: 0em 0em 0em 0em;
65 margin: 2em 0em 0em 0em;
66}
67
68h2.subtitle {
69 margin: 0.10em 0em 3.0em 0em;
70 padding: 0em 0em 0em 0em;
71 font-size: 1.8em;
72 padding-left: 20%;
73 font-weight: normal;
74 font-style: italic;
75}
76
77h2 {
78 margin: 2em 0em 0.66em 0em;
79 padding: 0.5em 0em 0em 0em;
80 font-size: 1.5em;
81 font-weight: bold;
82}
83
84h3.subtitle {
85 margin: 0em 0em 1em 0em;
86 padding: 0em 0em 0em 0em;
87 font-size: 142.14%;
88 text-align: right;
89}
90
91h3 {
92 margin: 1em 0em 0.5em 0em;
93 padding: 1em 0em 0em 0em;
94 font-size: 140%;
95 font-weight: bold;
96}
97
98h4 {
99 margin: 1em 0em 0.5em 0em;
100 padding: 1em 0em 0em 0em;
101 font-size: 120%;
102 font-weight: bold;
103}
104
105h5 {
106 margin: 1em 0em 0.5em 0em;
107 padding: 1em 0em 0em 0em;
108 font-size: 110%;
109 font-weight: bold;
110}
111
112h6 {
113 margin: 1em 0em 0em 0em;
114 padding: 1em 0em 0em 0em;
115 font-size: 110%;
116 font-weight: bold;
117}
118
119.authorgroup {
120 background-color: transparent;
121 background-repeat: no-repeat;
122 padding-top: 256px;
123 background-image: url("figures/test-manual-title.png");
124 background-position: left top;
125 margin-top: -256px;
126 padding-right: 50px;
127 margin-left: 0px;
128 text-align: right;
129 width: 740px;
130}
131
132h3.author {
133 margin: 0em 0me 0em 0em;
134 padding: 0em 0em 0em 0em;
135 font-weight: normal;
136 font-size: 100%;
137 color: #333;
138 clear: both;
139}
140
141.author tt.email {
142 font-size: 66%;
143}
144
145.titlepage hr {
146 width: 0em;
147 clear: both;
148}
149
150.revhistory {
151 padding-top: 2em;
152 clear: both;
153}
154
155.toc,
156.list-of-tables,
157.list-of-examples,
158.list-of-figures {
159 padding: 1.33em 0em 2.5em 0em;
160 color: #00557D;
161}
162
163.toc p,
164.list-of-tables p,
165.list-of-figures p,
166.list-of-examples p {
167 padding: 0em 0em 0em 0em;
168 padding: 0em 0em 0.3em;
169 margin: 1.5em 0em 0em 0em;
170}
171
172.toc p b,
173.list-of-tables p b,
174.list-of-figures p b,
175.list-of-examples p b{
176 font-size: 100.0%;
177 font-weight: bold;
178}
179
180.toc dl,
181.list-of-tables dl,
182.list-of-figures dl,
183.list-of-examples dl {
184 margin: 0em 0em 0.5em 0em;
185 padding: 0em 0em 0em 0em;
186}
187
188.toc dt {
189 margin: 0em 0em 0em 0em;
190 padding: 0em 0em 0em 0em;
191}
192
193.toc dd {
194 margin: 0em 0em 0em 2.6em;
195 padding: 0em 0em 0em 0em;
196}
197
198div.glossary dl,
199div.variablelist dl {
200}
201
202.glossary dl dt,
203.variablelist dl dt,
204.variablelist dl dt span.term {
205 font-weight: normal;
206 width: 20em;
207 text-align: right;
208}
209
210.variablelist dl dt {
211 margin-top: 0.5em;
212}
213
214.glossary dl dd,
215.variablelist dl dd {
216 margin-top: 0em;
217 margin-left: 25.5em;
218}
219
220.glossary dd p,
221.variablelist dd p {
222 margin-top: 0em;
223 margin-bottom: 1em;
224}
225
226
227div.calloutlist table td {
228 padding: 0em 0em 0em 0em;
229 margin: 0em 0em 0em 0em;
230}
231
232div.calloutlist table td p {
233 margin-top: 0em;
234 margin-bottom: 1em;
235}
236
237div p.copyright {
238 text-align: left;
239}
240
241div.legalnotice p.legalnotice-title {
242 margin-bottom: 0em;
243}
244
245p {
246 line-height: 1.5em;
247 margin-top: 0em;
248
249}
250
251dl {
252 padding-top: 0em;
253}
254
255hr {
256 border: solid 1px;
257}
258
259
260.mediaobject,
261.mediaobjectco {
262 text-align: center;
263}
264
265img {
266 border: none;
267}
268
269ul {
270 padding: 0em 0em 0em 1.5em;
271}
272
273ul li {
274 padding: 0em 0em 0em 0em;
275}
276
277ul li p {
278 text-align: left;
279}
280
281table {
282 width :100%;
283}
284
285th {
286 padding: 0.25em;
287 text-align: left;
288 font-weight: normal;
289 vertical-align: top;
290}
291
292td {
293 padding: 0.25em;
294 vertical-align: top;
295}
296
297p a[id] {
298 margin: 0px;
299 padding: 0px;
300 display: inline;
301 background-image: none;
302}
303
304a {
305 text-decoration: underline;
306 color: #444;
307}
308
309pre {
310 overflow: auto;
311}
312
313a:hover {
314 text-decoration: underline;
315 /*font-weight: bold;*/
316}
317
318/* This style defines how the permalink character
319 appears by itself and when hovered over with
320 the mouse. */
321
322[alt='Permalink'] { color: #eee; }
323[alt='Permalink']:hover { color: black; }
324
325
326div.informalfigure,
327div.informalexample,
328div.informaltable,
329div.figure,
330div.table,
331div.example {
332 margin: 1em 0em;
333 padding: 1em;
334 page-break-inside: avoid;
335}
336
337
338div.informalfigure p.title b,
339div.informalexample p.title b,
340div.informaltable p.title b,
341div.figure p.title b,
342div.example p.title b,
343div.table p.title b{
344 padding-top: 0em;
345 margin-top: 0em;
346 font-size: 100%;
347 font-weight: normal;
348}
349
350.mediaobject .caption,
351.mediaobject .caption p {
352 text-align: center;
353 font-size: 80%;
354 padding-top: 0.5em;
355 padding-bottom: 0.5em;
356}
357
358.epigraph {
359 padding-left: 55%;
360 margin-bottom: 1em;
361}
362
363.epigraph p {
364 text-align: left;
365}
366
367.epigraph .quote {
368 font-style: italic;
369}
370.epigraph .attribution {
371 font-style: normal;
372 text-align: right;
373}
374
375span.application {
376 font-style: italic;
377}
378
379.programlisting {
380 font-family: monospace;
381 font-size: 80%;
382 white-space: pre;
383 margin: 1.33em 0em;
384 padding: 1.33em;
385}
386
387.tip,
388.warning,
389.caution,
390.note {
391 margin-top: 1em;
392 margin-bottom: 1em;
393
394}
395
396/* force full width of table within div */
397.tip table,
398.warning table,
399.caution table,
400.note table {
401 border: none;
402 width: 100%;
403}
404
405
406.tip table th,
407.warning table th,
408.caution table th,
409.note table th {
410 padding: 0.8em 0.0em 0.0em 0.0em;
411 margin : 0em 0em 0em 0em;
412}
413
414.tip p,
415.warning p,
416.caution p,
417.note p {
418 margin-top: 0.5em;
419 margin-bottom: 0.5em;
420 padding-right: 1em;
421 text-align: left;
422}
423
424.acronym {
425 text-transform: uppercase;
426}
427
428b.keycap,
429.keycap {
430 padding: 0.09em 0.3em;
431 margin: 0em;
432}
433
434.itemizedlist li {
435 clear: none;
436}
437
438.filename {
439 font-size: medium;
440 font-family: Courier, monospace;
441}
442
443
444div.navheader, div.heading{
445 position: absolute;
446 left: 0em;
447 top: 0em;
448 width: 100%;
449 background-color: #cdf;
450 width: 100%;
451}
452
453div.navfooter, div.footing{
454 position: fixed;
455 left: 0em;
456 bottom: 0em;
457 background-color: #eee;
458 width: 100%;
459}
460
461
462div.navheader td,
463div.navfooter td {
464 font-size: 66%;
465}
466
467div.navheader table th {
468 /*font-family: Georgia, Times, serif;*/
469 /*font-size: x-large;*/
470 font-size: 80%;
471}
472
473div.navheader table {
474 border-left: 0em;
475 border-right: 0em;
476 border-top: 0em;
477 width: 100%;
478}
479
480div.navfooter table {
481 border-left: 0em;
482 border-right: 0em;
483 border-bottom: 0em;
484 width: 100%;
485}
486
487div.navheader table td a,
488div.navfooter table td a {
489 color: #777;
490 text-decoration: none;
491}
492
493/* normal text in the footer */
494div.navfooter table td {
495 color: black;
496}
497
498div.navheader table td a:visited,
499div.navfooter table td a:visited {
500 color: #444;
501}
502
503
504/* links in header and footer */
505div.navheader table td a:hover,
506div.navfooter table td a:hover {
507 text-decoration: underline;
508 background-color: transparent;
509 color: #33a;
510}
511
512div.navheader hr,
513div.navfooter hr {
514 display: none;
515}
516
517
518.qandaset tr.question td p {
519 margin: 0em 0em 1em 0em;
520 padding: 0em 0em 0em 0em;
521}
522
523.qandaset tr.answer td p {
524 margin: 0em 0em 1em 0em;
525 padding: 0em 0em 0em 0em;
526}
527.answer td {
528 padding-bottom: 1.5em;
529}
530
531.emphasis {
532 font-weight: bold;
533}
534
535
536 /************* /
537 / decorations /
538/ *************/
539
540.titlepage {
541}
542
543.part .title {
544}
545
546.subtitle {
547 border: none;
548}
549
550/*
551h1 {
552 border: none;
553}
554
555h2 {
556 border-top: solid 0.2em;
557 border-bottom: solid 0.06em;
558}
559
560h3 {
561 border-top: 0em;
562 border-bottom: solid 0.06em;
563}
564
565h4 {
566 border: 0em;
567 border-bottom: solid 0.06em;
568}
569
570h5 {
571 border: 0em;
572}
573*/
574
575.programlisting {
576 border: solid 1px;
577}
578
579div.figure,
580div.table,
581div.informalfigure,
582div.informaltable,
583div.informalexample,
584div.example {
585 border: 1px solid;
586}
587
588
589
590.tip,
591.warning,
592.caution,
593.note {
594 border: 1px solid;
595}
596
597.tip table th,
598.warning table th,
599.caution table th,
600.note table th {
601 border-bottom: 1px solid;
602}
603
604.question td {
605 border-top: 1px solid black;
606}
607
608.answer {
609}
610
611
612b.keycap,
613.keycap {
614 border: 1px solid;
615}
616
617
618div.navheader, div.heading{
619 border-bottom: 1px solid;
620}
621
622
623div.navfooter, div.footing{
624 border-top: 1px solid;
625}
626
627 /********* /
628 / colors /
629/ *********/
630
631body {
632 color: #333;
633 background: white;
634}
635
636a {
637 background: transparent;
638}
639
640a:hover {
641 background-color: #dedede;
642}
643
644
645h1,
646h2,
647h3,
648h4,
649h5,
650h6,
651h7,
652h8 {
653 background-color: transparent;
654}
655
656hr {
657 border-color: #aaa;
658}
659
660
661.tip, .warning, .caution, .note {
662 border-color: #fff;
663}
664
665
666.tip table th,
667.warning table th,
668.caution table th,
669.note table th {
670 border-bottom-color: #fff;
671}
672
673
674.warning {
675 background-color: #f0f0f2;
676}
677
678.caution {
679 background-color: #f0f0f2;
680}
681
682.tip {
683 background-color: #f0f0f2;
684}
685
686.note {
687 background-color: #f0f0f2;
688}
689
690.glossary dl dt,
691.variablelist dl dt,
692.variablelist dl dt span.term {
693 color: #044;
694}
695
696div.figure,
697div.table,
698div.example,
699div.informalfigure,
700div.informaltable,
701div.informalexample {
702 border-color: #aaa;
703}
704
705pre.programlisting {
706 color: black;
707 background-color: #fff;
708 border-color: #aaa;
709 border-width: 2px;
710}
711
712.guimenu,
713.guilabel,
714.guimenuitem {
715 background-color: #eee;
716}
717
718
719b.keycap,
720.keycap {
721 background-color: #eee;
722 border-color: #999;
723}
724
725
726div.navheader {
727 border-color: black;
728}
729
730
731div.navfooter {
732 border-color: black;
733}
734
735.writernotes {
736 color: red;
737}
738
739
740 /*********** /
741 / graphics /
742/ ***********/
743
744/*
745body {
746 background-image: url("images/body_bg.jpg");
747 background-attachment: fixed;
748}
749
750.navheader,
751.note,
752.tip {
753 background-image: url("images/note_bg.jpg");
754 background-attachment: fixed;
755}
756
757.warning,
758.caution {
759 background-image: url("images/warning_bg.jpg");
760 background-attachment: fixed;
761}
762
763.figure,
764.informalfigure,
765.example,
766.informalexample,
767.table,
768.informaltable {
769 background-image: url("images/figure_bg.jpg");
770 background-attachment: fixed;
771}
772
773*/
774h1,
775h2,
776h3,
777h4,
778h5,
779h6,
780h7{
781}
782
783/*
784Example of how to stick an image as part of the title.
785
786div.article .titlepage .title
787{
788 background-image: url("figures/white-on-black.png");
789 background-position: center;
790 background-repeat: repeat-x;
791}
792*/
793
794div.preface .titlepage .title,
795div.colophon .title,
796div.chapter .titlepage .title,
797div.article .titlepage .title
798{
799}
800
801div.section div.section .titlepage .title,
802div.sect2 .titlepage .title {
803 background: none;
804}
805
806
807h1.title {
808 background-color: transparent;
809 background-image: url("figures/test-title.png");
810 background-repeat: no-repeat;
811 height: 256px;
812 text-indent: -9000px;
813 overflow:hidden;
814}
815
816h2.subtitle {
817 background-color: transparent;
818 text-indent: -9000px;
819 overflow:hidden;
820 width: 0px;
821 display: none;
822}
823
824 /*************************************** /
825 / pippin.gimp.org specific alterations /
826/ ***************************************/
827
828/*
829div.heading, div.navheader {
830 color: #777;
831 font-size: 80%;
832 padding: 0;
833 margin: 0;
834 text-align: left;
835 position: absolute;
836 top: 0px;
837 left: 0px;
838 width: 100%;
839 height: 50px;
840 background: url('/gfx/heading_bg.png') transparent;
841 background-repeat: repeat-x;
842 background-attachment: fixed;
843 border: none;
844}
845
846div.heading a {
847 color: #444;
848}
849
850div.footing, div.navfooter {
851 border: none;
852 color: #ddd;
853 font-size: 80%;
854 text-align:right;
855
856 width: 100%;
857 padding-top: 10px;
858 position: absolute;
859 bottom: 0px;
860 left: 0px;
861
862 background: url('/gfx/footing_bg.png') transparent;
863}
864*/
865
866
867
868 /****************** /
869 / nasty ie tweaks /
870/ ******************/
871
872/*
873div.heading, div.navheader {
874 width:expression(document.body.clientWidth + "px");
875}
876
877div.footing, div.navfooter {
878 width:expression(document.body.clientWidth + "px");
879 margin-left:expression("-5em");
880}
881body {
882 padding:expression("4em 5em 0em 5em");
883}
884*/
885
886 /**************************************** /
887 / mozilla vendor specific css extensions /
888/ ****************************************/
889/*
890div.navfooter, div.footing{
891 -moz-opacity: 0.8em;
892}
893
894div.figure,
895div.table,
896div.informalfigure,
897div.informaltable,
898div.informalexample,
899div.example,
900.tip,
901.warning,
902.caution,
903.note {
904 -moz-border-radius: 0.5em;
905}
906
907b.keycap,
908.keycap {
909 -moz-border-radius: 0.3em;
910}
911*/
912
913table tr td table tr td {
914 display: none;
915}
916
917
918hr {
919 display: none;
920}
921
922table {
923 border: 0em;
924}
925
926 .photo {
927 float: right;
928 margin-left: 1.5em;
929 margin-bottom: 1.5em;
930 margin-top: 0em;
931 max-width: 17em;
932 border: 1px solid gray;
933 padding: 3px;
934 background: white;
935}
936 .seperator {
937 padding-top: 2em;
938 clear: both;
939 }
940
941 #validators {
942 margin-top: 5em;
943 text-align: right;
944 color: #777;
945 }
946 @media print {
947 body {
948 font-size: 8pt;
949 }
950 .noprint {
951 display: none;
952 }
953 }
954
955
956.tip,
957.note {
958 background: #f0f0f2;
959 color: #333;
960 padding: 20px;
961 margin: 20px;
962}
963
964.tip h3,
965.note h3 {
966 padding: 0em;
967 margin: 0em;
968 font-size: 2em;
969 font-weight: bold;
970 color: #333;
971}
972
973.tip a,
974.note a {
975 color: #333;
976 text-decoration: underline;
977}
978
979.footnote {
980 font-size: small;
981 color: #333;
982}
983
984/* Changes the announcement text */
985.tip h3,
986.warning h3,
987.caution h3,
988.note h3 {
989 font-size:large;
990 color: #00557D;
991}
diff --git a/documentation/test-manual/test-manual-test-process.xml b/documentation/test-manual/test-manual-test-process.xml
deleted file mode 100644
index 6e2157c621..0000000000
--- a/documentation/test-manual/test-manual-test-process.xml
+++ /dev/null
@@ -1,110 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='test-manual-test-process'>
7
8<title>Project Testing and Release Process</title>
9 <section id='test-daily-devel'>
10 <title>Day to Day Development</title>
11
12 <para>This section details how the project tests changes, through automation on the
13 Autobuilder or with the assistance of QA teams, through to making releases.</para>
14
15 <para>The project aims to test changes against our test matrix before those changes are
16 merged into the master branch. As such, changes are queued up in batches either in the
17 <filename>master-next</filename> branch in the main trees, or in user trees such as
18 <filename>ross/mut</filename> in <filename>poky-contrib</filename> (Ross Burton
19 helps review and test patches and this is his testing tree).</para>
20 <para>We have two broad categories of test builds, including "full" and "quick". On the
21 Autobuilder, these can be seen as "a-quick" and "a-full", simply for ease of sorting in
22 the UI. Use our Autobuilder console view to see where me manage most test-related items,
23 available at: <link linkend=""
24 >https://autobuilder.yoctoproject.org/typhoon/#/console</link>.</para>
25 <para>Builds are triggered manually when the test branches are ready. The builds are
26 monitored by the SWAT team. For additional information, see <link linkend=""
27 >https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team</link>. If
28 successful, the changes would usually be merged to the <filename>master</filename>
29 branch. If not successful, someone would respond to the changes on the mailing list
30 explaining that there was a failure in testing. The choice of quick or full would depend
31 on the type of changes and the speed with which the result was required.</para>
32 <para>The Autobuilder does build the <filename>master</filename> branch once daily for
33 several reasons, in particular, to ensure the current <filename>master</filename> branch
34 does build, but also to keep <filename>yocto-testresults</filename> (<link linkend=""
35 >http://git.yoctoproject.org/cgit.cgi/yocto-testresults/</link>), buildhistory
36 (<link linkend="">http://git.yoctoproject.org/cgit.cgi/poky-buildhistory/</link>),
37 and our sstate up to date. On the weekend, there is a master-next build instead to
38 ensure the test results are updated for the less frequently run targets.</para>
39 <para>Performance builds (buildperf-* targets in the console) are triggered separately every
40 six hours and automatically push their results to the buildstats repository at: <link
41 linkend="">http://git.yoctoproject.org/cgit.cgi/yocto-buildstats/</link>. </para>
42 <para>The 'quick' targets have been selected to be the ones which catch the most failures or
43 give the most valuable data. We run 'fast' ptests in this case for example but not the
44 ones which take a long time. The quick target doesn't include *-lsb builds for all
45 architectures, some world builds and doesn't trigger performance tests or ltp testing.
46 The full build includes all these things and is slower but more comprehensive.</para>
47 </section>
48
49 <section id='test-yocto-project-autobuilder-overview'>
50 <title>Release Builds</title>
51
52 <para>The project typically has two major releases a year with a six month cadence in April
53 and October. Between these there would be a number of milestone releases (usually four)
54 with the final one being stablization only along with point releases of our stable
55 branches.</para>
56 <para>The build and release process for these project releases is similar to that in <link
57 linkend="test-daily-devel">Day to Day Development</link>, in that the a-full target
58 of the Autobuilder is used but in addition the form is configured to generate and
59 publish artefacts and the milestone number, version, release candidate number and other
60 information is entered. The box to "generate an email to QA"is also checked.</para>
61 <para>When the build completes, an email is sent out using the send-qa-email script in the
62 <filename>yocto-autobuilder-helper</filename> repository to the list of people
63 configured for that release. Release builds are placed into a directory in <link
64 linkend="">https://autobuilder.yocto.io/pub/releases</link> on the Autobuilder which
65 is included in the email. The process from here is more manual and control is
66 effectively passed to release engineering. The next steps include:<itemizedlist>
67 <listitem>
68 <para>QA teams respond to the email saying which tests they plan to run and when
69 the results will be available.</para>
70 </listitem>
71 <listitem>
72 <para>QA teams run their tests and share their results in the yocto-
73 testresults-contrib repository, along with a summary of their findings.
74 </para>
75 </listitem>
76 <listitem>
77 <para>Release engineering prepare the release as per their process. </para>
78 </listitem>
79 <listitem>
80 <para>Test results from the QA teams are included into the release in separate
81 directories and also uploaded to the yocto-testresults repository alongside
82 the other test results for the given revision.</para>
83 </listitem>
84 <listitem>
85 <para>The QA report in the final release is regenerated using resulttool to
86 include the new test results and the test summaries from the teams (as
87 headers to the generated report).</para>
88 </listitem>
89 <listitem>
90 <para>The release is checked against the release checklist and release readiness
91 criteria.</para>
92 </listitem>
93 <listitem>
94 <para>A final decision on whether to release is made by the YP TSC who have
95 final oversight on release readiness.</para>
96 </listitem>
97 </itemizedlist></para>
98 </section>
99
100
101
102
103
104
105
106
107</chapter>
108<!--
109vim: expandtab tw=80 ts=4
110-->
diff --git a/documentation/test-manual/test-manual-understand-autobuilder.xml b/documentation/test-manual/test-manual-understand-autobuilder.xml
deleted file mode 100644
index 8600367be7..0000000000
--- a/documentation/test-manual/test-manual-understand-autobuilder.xml
+++ /dev/null
@@ -1,314 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='test-manual-understand-autobuilder'>
7
8<title>Understanding the Yocto Project Autobuilder</title>
9 <section>
10 <title>Execution Flow within the Autobuilder</title>
11 <para>The "a-full" and "a-quick" targets are the usual entry points into the Autobuilder and
12 it makes sense to follow the process through the system starting there. This is best
13 visualised from the Autobuilder Console view (<link linkend=""
14 >https://autobuilder.yoctoproject.org/typhoon/#/console</link>). </para>
15 <para>Each item along the top of that view represents some "target build" and these targets
16 are all run in parallel. The 'full' build will trigger the majority of them, the "quick"
17 build will trigger some subset of them. The Autobuilder effectively runs whichever
18 configuration is defined for each of those targets on a seperate buildbot worker. To
19 understand the configuration, you need to look at the entry on
20 <filename>config.json</filename> file within the
21 <filename>yocto-autobuilder-helper</filename> repository. The targets are defined in
22 the ‘overrides' section, a quick example could be qemux86-64 which looks
23 like:<literallayout class="monospaced">
24 "qemux86-64" : {
25 "MACHINE" : "qemux86-64",
26 "TEMPLATE" : "arch-qemu",
27 "step1" : {
28 "extravars" : [
29 "IMAGE_FSTYPES_append = ' wic wic.bmap'"
30 ]
31 }
32 },
33 </literallayout>And
34 to expand that, you need the "arch-qemu" entry from the "templates" section, which looks
35 like:<literallayout class="monospaced">
36 "arch-qemu" : {
37 "BUILDINFO" : true,
38 "BUILDHISTORY" : true,
39 "step1" : {
40 "BBTARGETS" : "core-image-sato core-image-sato-dev core-image-sato-sdk core-image-minimal core-image-minimal-dev core-image-sato:do_populate_sdk",
41 "SANITYTARGETS" : "core-image-minimal:do_testimage core-image-sato:do_testimage core-image-sato-sdk:do_testimage core-image-sato:do_testsdk"
42 },
43 "step2" : {
44 "SDKMACHINE" : "x86_64",
45 "BBTARGETS" : "core-image-sato:do_populate_sdk core-image-minimal:do_populate_sdk_ext core-image-sato:do_populate_sdk_ext",
46 "SANITYTARGETS" : "core-image-sato:do_testsdk core-image-minimal:do_testsdkext core-image-sato:do_testsdkext"
47 },
48 "step3" : {
49 "BUILDHISTORY" : false,
50 "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; DISPLAY=:1 oe-selftest ${HELPERSTMACHTARGS} -j 15"],
51 "ADDLAYER" : ["${BUILDDIR}/../meta-selftest"]
52 }
53 },
54 </literallayout>Combining
55 these two entries you can see that "qemux86-64" is a three step build where the
56 <filename>bitbake BBTARGETS</filename> would be run, then <filename>bitbake
57 SANITYTARGETS</filename> for each step; all for
58 <filename>MACHINE="qemx86-64"</filename> but with differing SDKMACHINE settings. In
59 step 1 an extra variable is added to the <filename>auto.conf</filename> file to enable
60 wic image generation.</para>
61 <para>While not every detail of this is covered here, you can see how the templating
62 mechanism allows quite complex configurations to be built up yet allows duplication and
63 repetition to be kept to a minimum.</para>
64 <para>The different build targets are designed to allow for parallelisation, so different
65 machines are usually built in parallel, operations using the same machine and metadata
66 are built sequentially, with the aim of trying to optimise build efficiency as much as
67 possible.</para>
68 <para>The <filename>config.json</filename> file is processed by the scripts in the Helper
69 repository in the <filename>scripts</filename> directory. The following section details
70 how this works.</para>
71 </section>
72
73 <section id='test-autobuilder-target-exec-overview'>
74 <title>Autobuilder Target Execution Overview</title>
75
76 <para>For each given target in a build, the Autobuilder executes several steps. These are
77 configured in <filename>yocto-autobuilder2/builders.py</filename> and roughly consist
78 of: <orderedlist>
79 <listitem id='test-list-tgt-exec-clobberdir'>
80 <para><emphasis>Run <filename>clobberdir</filename></emphasis></para>
81 <para>This cleans out any previous build. Old builds are left around to allow
82 easier debugging of failed builds. For additional information, see <link
83 linkend="test-clobberdir"><filename>clobberdir</filename></link>.</para>
84 </listitem>
85 <listitem>
86 <para><emphasis>Obtain yocto-autobuilder-helper</emphasis></para>
87 <para>This step clones the <filename>yocto-autobuilder-helper</filename> git
88 repository. This is necessary to prevent the requirement to maintain all the
89 release or project-specific code within Buildbot. The branch chosen matches
90 the release being built so we can support older releases and still make
91 changes in newer ones.</para>
92 </listitem>
93 <listitem>
94 <para><emphasis>Write layerinfo.json</emphasis></para>
95 <para>This transfers data in the Buildbot UI when the build was configured to
96 the Helper.</para>
97 </listitem>
98 <listitem>
99 <para><emphasis>Call scripts/shared-repo-unpack</emphasis></para>
100 <para>This is a call into the Helper scripts to set up a checkout of all the
101 pieces this build might need. It might clone the BitBake repository and the
102 OpenEmbedded-Core repository. It may clone the Poky repository, as well as
103 additional layers. It will use the data from the
104 <filename>layerinfo.json</filename> file to help understand the
105 configuration. It will also use a local cache of repositories to speed up
106 the clone checkouts. For additional information, see <link
107 linkend="test-autobuilder-clone-cache">Autobuilder Clone
108 Cache</link>.</para>
109 <para>This step has two possible modes of operation. If the build is part of a
110 parent build, its possible that all the repositories needed may already be
111 available, ready in a pre-prepared directory. An "a-quick" or "a-full" build
112 would prepare this before starting the other sub-target builds. This is done
113 for two reasons:<itemizedlist>
114 <listitem>
115 <para>the upstream may change during a build, for example, from a
116 forced push and this ensures we have matching content for the
117 whole build</para>
118 </listitem>
119 <listitem>
120 <para>if 15 Workers all tried to pull the same data from the same
121 repos, we can hit resource limits on upstream servers as they
122 can think they are under some kind of network attack</para>
123 </listitem>
124 </itemizedlist>This pre-prepared directory is shared among the Workers over
125 NFS. If the build is an individual build and there is no "shared" directory
126 available, it would clone from the cache and the upstreams as necessary.
127 This is considered the fallback mode.</para>
128 </listitem>
129 <listitem>
130 <para><emphasis>Call scripts/run-config</emphasis></para>
131 <para>This is another call into the Helper scripts where its expected that the
132 main functionality of this target will be executed.</para>
133 </listitem>
134 </orderedlist></para>
135 </section>
136 <section id='test-autobuilder-tech'>
137 <title>Autobuilder Technology</title>
138 <para>The Autobuilder has Yocto Project-specific functionality to allow builds to operate
139 with increased efficiency and speed.</para>
140 <section id='test-clobberdir'>
141 <title>clobberdir</title>
142 <para>When deleting files, the Autobuilder uses <filename>clobberdir</filename>, which
143 is a special script that moves files to a special location, rather than deleting
144 them. Files in this location are deleted by an <filename>rm</filename> command,
145 which is run under <filename>ionice -c 3</filename>. For example, the deletion only
146 happens when there is idle IO capacity on the Worker. The Autobuilder Worker Janitor
147 runs this deletion. See <link linkend="test-autobuilder-worker-janitor">Autobuilder
148 Worker Janitor</link>.</para>
149 </section>
150 <section id='test-autobuilder-clone-cache'>
151 <title>Autobuilder Clone Cache</title>
152 <para>Cloning repositories from scratch each time they are required was slow on the
153 Autobuilder. We therefore have a stash of commonly used repositories pre-cloned on
154 the Workers. Data is fetched from these during clones first, then "topped up" with
155 later revisions from any upstream when necesary. The cache is maintained by the
156 Autobuilder Worker Janitor. See <link linkend="test-autobuilder-worker-janitor"
157 >Autobuilder Worker Janitor</link>.</para>
158 </section>
159 <section id='test-autobuilder-worker-janitor'>
160 <title>Autobuilder Worker Janitor</title>
161 <para>This is a process running on each Worker that performs two basic operations,
162 including background file deletion at IO idle (see <link
163 linkend="test-list-tgt-exec-clobberdir">Target Execution: clobberdir</link>) and
164 maintainenance of a cache of cloned repositories to improve the speed the system can
165 checkout repositories.</para>
166 </section>
167 <section id='test-shared-dl-dir'>
168 <title>Shared DL_DIR</title>
169 <para>The Workers are all connected over NFS which allows DL_DIR to be shared between
170 them. This reduces network accesses from the system and allows the build to be sped
171 up. Usage of the directory within the build system is designed to be able to be
172 shared over NFS.</para>
173 </section>
174 <section id='test-shared-sstate-cache'>
175 <title>Shared SSTATE_DIR</title>
176 <para>The Workers are all connected over NFS which allows the
177 <filename>sstate</filename> directory to be shared between them. This means once
178 a Worker has built an artefact, all the others can benefit from it. Usage of the
179 directory within the directory is designed for sharing over NFS.</para>
180 </section>
181 <section id='test-resulttool'>
182 <title>Resulttool</title>
183 <para>All of the different tests run as part of the build generate output into
184 <filename>testresults.json</filename> files. This allows us to determine which
185 tests ran in a given build and their status. Additional information, such as failure
186 logs or the time taken to run the tests, may also be included.</para>
187 <para>Resulttool is part of OpenEmbedded-Core and is used to manipulate these json
188 results files. It has the ability to merge files together, display reports of the
189 test results and compare different result files.</para>
190 <para>For details, see <link linkend=""
191 >https://wiki.yoctoproject.org/wiki/Resulttool</link>.</para>
192 </section>
193 </section>
194 <section id='test-run-config-tgt-execution'>
195 <title>run-config Target Execution</title>
196 <para>The <filename>scripts/run-config</filename> execution is where most of the work within
197 the Autobuilder happens. It runs through a number of steps; the first are general setup
198 steps that are run once and include:<orderedlist>
199 <listitem>
200 <para>Set up any <filename>buildtools-tarball</filename> if configured.</para>
201 </listitem>
202 <listitem>
203 <para>Call "buildhistory-init" if buildhistory is configured.</para>
204 </listitem>
205 </orderedlist></para>
206 <para>For each step that is configured in <filename>config.json</filename>, it will perform
207 the following:</para>
208 <para>
209 <remark>## WRITER's question: What does "logging in as stepXa" and others refer to
210 below? ##</remark>
211 <orderedlist>
212 <listitem id="test-run-config-add-layers-step">
213 <para dir="ltr">Add any layers that are specified using the
214 <filename>bitbake-layers add-layer</filename> command (logging as
215 stepXa)</para>
216 </listitem>
217 <listitem>
218 <para dir="ltr">Call the <filename>scripts/setup-config</filename> script to
219 generate the necessary <filename>auto.conf</filename> configuration file for
220 the build</para>
221 </listitem>
222 <listitem>
223 <para dir="ltr">Run the <filename>bitbake BBTARGETS</filename> command (logging
224 as stepXb)</para>
225 </listitem>
226 <listitem>
227 <para dir="ltr">Run the <filename>bitbake SANITYTARGETS</filename> command
228 (logging as stepXc)</para>
229 </listitem>
230 <listitem>
231 <para dir="ltr">Run the <filename>EXTRACMDS</filename> command, which are run
232 within the BitBake build environment (logging as stepXd)</para>
233 </listitem>
234 <listitem>
235 <para dir="ltr">Run the <filename>EXTRAPLAINCMDS</filename> command(s), which
236 are run outside the BitBake build environment (logging as stepXd)</para>
237 </listitem>
238 <listitem>
239 <para dir="ltr">Remove any layers added in <link
240 linkend="test-run-config-add-layers-step">step 1</link> using the
241 <filename>bitbake-layers remove-layer</filename> command (logging as
242 stepXa)</para>
243 </listitem>
244 </orderedlist>
245 </para>
246 <para>Once the execution steps above complete, <filename>run-config</filename> executes a
247 set of post-build steps, including:<orderedlist>
248 <listitem>
249 <para dir="ltr">Call <filename>scripts/publish-artifacts</filename> to collect
250 any output which is to be saved from the build.</para>
251 </listitem>
252 <listitem>
253 <para dir="ltr">Call <filename>scripts/collect-results</filename> to collect any
254 test results to be saved from the build.</para>
255 </listitem>
256 <listitem>
257 <para dir="ltr">Call <filename>scripts/upload-error-reports</filename> to send
258 any error reports generated to the remote server.</para>
259 </listitem>
260 <listitem>
261 <para dir="ltr">Cleanup the build directory using <link
262 linkend="test-clobberdir"><filename>clobberdir</filename></link> if the
263 build was successful, else rename it to "build-renamed" for potential future
264 debugging.</para>
265 </listitem>
266 </orderedlist></para>
267 </section>
268 <section id='test-deploying-yp-autobuilder'>
269 <title>Deploying Yocto Autobuilder</title>
270 <para>The most up to date information about how to setup and deploy your own Autbuilder can
271 be found in README.md in the <filename>yocto-autobuilder2</filename> repository.</para>
272 <para>We hope that people can use the <filename>yocto-autobuilder2</filename> code directly
273 but it is inevitable that users will end up needing to heavily customise the
274 <filename>yocto-autobuilder-helper</filename> repository, particularly the
275 <filename>config.json</filename> file as they will want to define their own test
276 matrix.</para>
277 <para>The Autobuilder supports wo customization options: <itemizedlist>
278 <listitem>
279 <para>variable substitution</para>
280 </listitem>
281 <listitem>
282 <para>overlaying configuration files</para>
283 </listitem>
284 </itemizedlist>The standard <filename>config.json</filename> minimally attempts to allow
285 substitution of the paths. The Helper script repository includes a
286 <filename>local-example.json</filename> file to show how you could override these
287 from a separate configuration file. Pass the following into the environment of the
288 Autobuilder:<literallayout class="monospaced">
289 $ ABHELPER_JSON="config.json local-example.json"
290 </literallayout>As
291 another example, you could also pass the following into the
292 environment:<literallayout class="monospaced">
293 $ ABHELPER_JSON="config.json <replaceable>/some/location/</replaceable>local.json"
294 </literallayout>One
295 issue users often run into is validation of the <filename>config.json</filename> files.
296 A tip for minimizing issues from invalid json files is to use a Git
297 <filename>pre-commit-hook.sh</filename> script to verify the JSON file before
298 committing it. Create a symbolic link as
299 follows:<literallayout class="monospaced">
300 $ ln -s ../../scripts/pre-commit-hook.sh .git/hooks/pre-commit
301 </literallayout></para>
302 </section>
303
304
305
306
307
308
309
310
311</chapter>
312<!--
313vim: expandtab tw=80 ts=4
314-->
diff --git a/documentation/test-manual/test-manual.xml b/documentation/test-manual/test-manual.xml
deleted file mode 100755
index d454566c5f..0000000000
--- a/documentation/test-manual/test-manual.xml
+++ /dev/null
@@ -1,104 +0,0 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<book id='test-manual' lang='en'
7 xmlns:xi="http://www.w3.org/2003/XInclude"
8 xmlns="http://docbook.org/ns/docbook"
9 >
10 <bookinfo>
11
12 <mediaobject>
13 <imageobject>
14 <imagedata fileref='figures/test-manual-title.png'
15 format='SVG'
16 align='left' scalefit='1' width='100%'/>
17 </imageobject>
18 </mediaobject>
19
20 <title>
21 Yocto Project Test Environment Manual
22 </title>
23
24 <authorgroup>
25 <author>
26 <affiliation>
27 <orgname>&ORGNAME;</orgname>
28 </affiliation>
29 <email>&ORGEMAIL;</email>
30 </author>
31 </authorgroup>
32
33 <revhistory>
34 <revision>
35 <revnumber>3.1.1</revnumber>
36 <date>TBD</date>
37 <revremark>DRAFT - Work-in-Progress - posted June 16, 2020</revremark>
38 </revision>
39 </revhistory>
40
41 <copyright>
42 <year>&COPYRIGHT_YEAR;</year>
43 <holder>Linux Foundation</holder>
44 </copyright>
45
46 <legalnotice>
47 <para>
48 Permission is granted to copy, distribute and/or modify this document under
49 the terms of the <ulink type="http" url="http://creativecommons.org/licenses/by-sa/2.0/uk/">
50 Creative Commons Attribution-Share Alike 2.0 UK: England &amp; Wales</ulink> as published by
51 Creative Commons.
52 </para>
53 <note><title>Manual Notes</title>
54 <itemizedlist>
55 <listitem><para>
56 This version of the
57 <emphasis>Yocto Project Test Environment Manual</emphasis>
58 is for the &YOCTO_DOC_VERSION; release of the
59 Yocto Project.
60 To be sure you have the latest version of the manual
61 for this release, go to the
62 <ulink url='&YOCTO_HOME_URL;/documentation'>Yocto Project documentation page</ulink>
63 and select the manual from that site.
64 Manuals from the site are more up-to-date than manuals
65 derived from the Yocto Project released TAR files.
66 </para></listitem>
67 <listitem><para>
68 If you located this manual through a web search, the
69 version of the manual might not be the one you want
70 (e.g. the search might have returned a manual much
71 older than the Yocto Project version with which you
72 are working).
73 You can see all Yocto Project major releases by
74 visiting the
75 <ulink url='&YOCTO_WIKI_URL;/wiki/Releases'>Releases</ulink>
76 page.
77 If you need a version of this manual for a different
78 Yocto Project release, visit the
79 <ulink url='&YOCTO_HOME_URL;/documentation'>Yocto Project documentation page</ulink>
80 and select the manual set by using the
81 "ACTIVE RELEASES DOCUMENTATION" or "DOCUMENTS ARCHIVE"
82 pull-down menus.
83 </para></listitem>
84 <listitem><para>
85 To report any inaccuracies or problems with this
86 manual, send an email to the Yocto Project
87 discussion group at
88 <filename>yocto@yoctoproject.com</filename> or log into
89 the freenode <filename>#yocto</filename> channel.
90 </para></listitem>
91 </itemizedlist>
92 </note>
93 </legalnotice>
94
95 </bookinfo>
96
97 <xi:include href="test-manual-intro.xml"/>
98 <xi:include href="test-manual-test-process.xml"/>
99 <xi:include href="test-manual-understand-autobuilder.xml"/>
100
101</book>
102<!--
103vim: expandtab tw=80 ts=4
104-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-12 01:56:56 +0000