summaryrefslogtreecommitdiffstats
path: root/documentation
diff options
context:
space:
mode:
authorMark Morton <Mark.Morton@windriver.com>2020-09-14 16:11:31 -0700
committerRichard Purdie <richard.purdie@linuxfoundation.org>2020-09-17 10:09:35 +0100
commit74f212d2e1db87ccfecaf9385ab280123d5fb168 (patch)
tree79e85fac733b78eb11cb19ca9753ed44dbf8358e /documentation
parentfc876832cb57a1a3e61150778a9cfd7acffbc2b6 (diff)
downloadpoky-74f212d2e1db87ccfecaf9385ab280123d5fb168.tar.gz
sphinx: test-manual code block, link, and format update
(From yocto-docs rev: 928c212ec4ad6e09524fdf8147aa7daf244677b0) Signed-off-by: Mark Morton <mark.morton@windriver.com> Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation')
-rw-r--r--documentation/test-manual/test-manual-intro.rst287
-rw-r--r--documentation/test-manual/test-manual-test-process.rst14
-rw-r--r--documentation/test-manual/test-manual-understand-autobuilder.rst140
3 files changed, 257 insertions, 184 deletions
diff --git a/documentation/test-manual/test-manual-intro.rst b/documentation/test-manual/test-manual-intro.rst
index 2f11e2beb3..b0bc24c9b0 100644
--- a/documentation/test-manual/test-manual-intro.rst
+++ b/documentation/test-manual/test-manual-intro.rst
@@ -23,26 +23,25 @@ project core.
23 23
24Currently, the Yocto Project Test Environment Manual has no projected 24Currently, the Yocto Project Test Environment Manual has no projected
25release date. This manual is a work-in-progress and is being initially 25release date. This manual is a work-in-progress and is being initially
26loaded with information from the `README <>`__ files and notes from key 26loaded with information from the README files and notes from key
27engineers: 27engineers:
28 28
29- *``yocto-autobuilder2``:* This 29- *yocto-autobuilder2:* This
30 ```README.md`http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md 30 :yocto_git:`README.md </cgit.cgi/yocto-autobuilder2/tree/README.md>`
31 is the main README which detials how to set up the Yocto Project 31 is the main README which detials how to set up the Yocto Project
32 Autobuilder. The ``yocto-autobuilder2`` repository represents the 32 Autobuilder. The ``yocto-autobuilder2`` repository represents the
33 Yocto Project's console UI plugin to Buildbot and the configuration 33 Yocto Project's console UI plugin to Buildbot and the configuration
34 necessary to configure Buildbot to perform the testing the project 34 necessary to configure Buildbot to perform the testing the project
35 requires. 35 requires.
36 36
37- *``yocto-autobuilder-helper``:* This 37- *yocto-autobuilder-helper:* This :yocto_git:`README </cgit.cgi/yocto-autobuilder-helper/tree/README/>`
38 ```README`http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/README
39 and repository contains Yocto Project Autobuilder Helper scripts and 38 and repository contains Yocto Project Autobuilder Helper scripts and
40 configuration. The ``yocto-autobuilder-helper`` repository contains 39 configuration. The ``yocto-autobuilder-helper`` repository contains
41 the "glue" logic that defines which tests to run and how to run them. 40 the "glue" logic that defines which tests to run and how to run them.
42 As a result, it can be used by any Continuous Improvement (CI) system 41 As a result, it can be used by any Continuous Improvement (CI) system
43 to run builds, support getting the correct code revisions, configure 42 to run builds, support getting the correct code revisions, configure
44 builds and layers, run builds, and collect results. The code is 43 builds and layers, run builds, and collect results. The code is
45 independent of any CI system, which means the code can work Buildbot, 44 independent of any CI system, which means the code can work `Buildbot <https://docs.buildbot.net/0.9.15.post1/>`__,
46 Jenkins, or others. This repository has a branch per release of the 45 Jenkins, or others. This repository has a branch per release of the
47 project defining the tests to run on a per release basis. 46 project defining the tests to run on a per release basis.
48 47
@@ -124,21 +123,22 @@ thefollowing types of tests:
124 ensure we have no single point of failure and can ensure the 123 ensure we have no single point of failure and can ensure the
125 different distros work effectively. 124 different distros work effectively.
126 125
127- *eSDK Testing:* Image tests initiated through the following command: 126- *eSDK Testing:* Image tests initiated through the following command::
128 $ bitbake image -c testsdkext The tests utilize the ``testsdkext`` 127
129 class and the ``do_testsdkext`` task. 128 $ bitbake image -c testsdkext
129
130 The tests utilize the ``testsdkext`` class and the ``do_testsdkext`` task.
130 131
131- *Feature Testing:* Various scenario-based tests are run through the 132- *Feature Testing:* Various scenario-based tests are run through the
132 `OpenEmbedded 133 :ref:`OpenEmbedded Self test (oe-selftest) <ref-manual/ref-release-process:Testing and Quality Assurance>`. We test oe-selftest on each of the main distrubutions
133 Self-Test <&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance>`__
134 (oe-selftest). We test oe-selftest on each of the main distrubutions
135 we support. 134 we support.
136 135
137- *Image Testing:* Image tests initiated through the following command: 136- *Image Testing:* Image tests initiated through the following command::
138 $ bitbake image -c testimage The tests utilize the 137
139 :ref:`testimage* <ref-classes-testimage*>` 138 $ bitbake image -c testimage
140 classes and the 139
141 :ref:`ref-tasks-testimage` task. 140 The tests utilize the :ref:`testimage* <ref-classes-testimage*>`
141 classes and the :ref:`ref-tasks-testimage` task.
142 142
143- *Layer Testing:* The Autobuilder has the possibility to test whether 143- *Layer Testing:* The Autobuilder has the possibility to test whether
144 specific layers work with the test of the system. The layers tested 144 specific layers work with the test of the system. The layers tested
@@ -147,20 +147,22 @@ thefollowing types of tests:
147 147
148- *Package Testing:* A Package Test (ptest) runs tests against packages 148- *Package Testing:* A Package Test (ptest) runs tests against packages
149 built by the OpenEmbedded build system on the target machine. See the 149 built by the OpenEmbedded build system on the target machine. See the
150 "`Testing Packages With 150 :ref:`Testing Packages With
151 ptest <&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest>`__" section 151 ptest <dev-manual/dev-manual-common-tasks:Testing Packages With ptest>` section
152 in the Yocto Project Development Tasks Manual and the 152 in the Yocto Project Development Tasks Manual and the
153 ":yocto_wiki:`Ptest </wiki/Ptest>`" Wiki page for more 153 ":yocto_wiki:`Ptest </wiki/Ptest>`" Wiki page for more
154 information on Ptest. 154 information on Ptest.
155 155
156- *SDK Testing:* Image tests initiated through the following command: $ 156- *SDK Testing:* Image tests initiated through the following command::
157 bitbake image -c testsdk The tests utilize the 157
158 :ref:`testsdk <ref-classes-testsdk>` class and 158 $ bitbake image -c testsdk
159
160 The tests utilize the :ref:`testsdk <ref-classes-testsdk>` class and
159 the ``do_testsdk`` task. 161 the ``do_testsdk`` task.
160 162
161- *Unit Testing:* Unit tests on various components of the system run 163- *Unit Testing:* Unit tests on various components of the system run
162 through ``oe-selftest`` and 164 through :ref:`bitbake-selftest <ref-manual/ref-release-process:Testing and Quality Assurance>` and
163 ```bitbake-selftest`` <&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance>`__. 165 :ref:`oe-selftest <ref-manual/ref-release-process:Testing and Quality Assurance>`.
164 166
165- *Automatic Upgrade Helper:* This target tests whether new versions of 167- *Automatic Upgrade Helper:* This target tests whether new versions of
166 software are available and whether we can automatically upgrade to 168 software are available and whether we can automatically upgrade to
@@ -174,36 +176,43 @@ How Tests Map to Areas of Code
174 176
175Tests map into the codebase as follows: 177Tests map into the codebase as follows:
176 178
177- *bitbake-selftest*: 179- *bitbake-selftest:*
178 180
179 These tests are self-contained and test BitBake as well as its APIs, 181 These tests are self-contained and test BitBake as well as its APIs,
180 which include the fetchers. The tests are located in 182 which include the fetchers. The tests are located in
181 ``bitbake/lib/*/tests``. 183 ``bitbake/lib/*/tests``.
182 184
183 From within the BitBake repository, run the following: $ 185 From within the BitBake repository, run the following::
184 bitbake-selftest 186
187 $ bitbake-selftest
185 188
186 To skip tests that access the Internet, use the ``BB_SKIP_NETTEST`` 189 To skip tests that access the Internet, use the ``BB_SKIP_NETTEST``
187 variable when running "bitbake-selftest" as follows: $ 190 variable when running "bitbake-selftest" as follows::
188 BB_SKIP_NETTEST=yes bitbake-selftest 191
192 $ BB_SKIP_NETTEST=yes bitbake-selftest
189 193
190 The default output is quiet and just prints a summary of what was 194 The default output is quiet and just prints a summary of what was
191 run. To see more information, there is a verbose option:$ 195 run. To see more information, there is a verbose option::
192 bitbake-selftest -v 196
197 $ bitbake-selftest -v
193 198
194 Use this option when you wish to skip tests that access the network, 199 Use this option when you wish to skip tests that access the network,
195 which are mostly necessary to test the fetcher modules. To specify 200 which are mostly necessary to test the fetcher modules. To specify
196 individual test modules to run, append the test module name to the 201 individual test modules to run, append the test module name to the
197 "bitbake-selftest" command. For example, to specify the tests for the 202 "bitbake-selftest" command. For example, to specify the tests for the
198 bb.data.module, run: $ bitbake-selftest bb.test.data.moduleYou can 203 bb.data.module, run::
199 also specify individual tests by defining the full name and module 204
200 plus the class path of the test, for example: $ bitbake-selftest 205 $ bitbake-selftest bb.test.data.module
201 bb.tests.data.TestOverrides.test_one_override 206
207 You can also specify individual tests by defining the full name and module
208 plus the class path of the test, for example::
209
210 $ bitbake-selftest bb.tests.data.TestOverrides.test_one_override
202 211
203 The tests are based on `Python 212 The tests are based on `Python
204 unittest <https://docs.python.org/3/library/unittest.html>`__. 213 unittest <https://docs.python.org/3/library/unittest.html>`__.
205 214
206- *oe-selftest*: 215- *oe-selftest:*
207 216
208 - These tests use OE to test the workflows, which include testing 217 - These tests use OE to test the workflows, which include testing
209 specific features, behaviors of tasks, and API unit tests. 218 specific features, behaviors of tasks, and API unit tests.
@@ -219,31 +228,40 @@ Tests map into the codebase as follows:
219 - The code for the tests resides in 228 - The code for the tests resides in
220 ``meta/lib/oeqa/selftest/cases/``. 229 ``meta/lib/oeqa/selftest/cases/``.
221 230
222 - To run all the tests, enter the following command: $ oe-selftest 231 - To run all the tests, enter the following command::
223 -a 232
233 $ oe-selftest -a
224 234
225 - To run a specific test, use the following command form where 235 - To run a specific test, use the following command form where
226 testname is the name of the specific test: $ oe-selftest -r 236 testname is the name of the specific test::
227 testname For example, the following command would run the tinfoil 237
228 getVar API test:$ oe-selftest -r 238 $ oe-selftest -r <testname>
229 tinfoil.TinfoilTests.test_getvarIt is also possible to run a set 239
240 For example, the following command would run the tinfoil
241 getVar API test::
242
243 $ oe-selftest -r tinfoil.TinfoilTests.test_getvar
244
245 It is also possible to run a set
230 of tests. For example the following command will run all of the 246 of tests. For example the following command will run all of the
231 tinfoil tests:$ oe-selftest -r tinfoil 247 tinfoil tests::
248
249 $ oe-selftest -r tinfoil
232 250
233- *testimage:* 251- *testimage:*
234 252
235 - These tests build an image, boot it, and run tests against the 253 - These tests build an image, boot it, and run tests against the
236 image's content. 254 image's content.
237 255
238 - The code for these tests resides in 256 - The code for these tests resides in ``meta/lib/oeqa/runtime/cases/``.
239 ``meta/lib/oeqa/runtime/cases/``. 257
258 - You need to set the :term:`IMAGE_CLASSES` variable as follows::
240 259
241 - You need to set the 260 IMAGE_CLASSES += "testimage"
242 :term:`IMAGE_CLASSES`
243 variable as follows: IMAGE_CLASSES += "testimage"
244 261
245 - Run the tests using the following command form: $ bitbake image -c 262 - Run the tests using the following command form::
246 testimage 263
264 $ bitbake image -c testimage
247 265
248- *testsdk:* 266- *testsdk:*
249 267
@@ -252,8 +270,9 @@ Tests map into the codebase as follows:
252 270
253 - The code for these tests resides in ``meta/lib/oeqa/sdk/cases/``. 271 - The code for these tests resides in ``meta/lib/oeqa/sdk/cases/``.
254 272
255 - Run the test using the following command form: $ bitbake image -c 273 - Run the test using the following command form::
256 testsdk 274
275 $ bitbake image -c testsdk
257 276
258- *testsdk_ext:* 277- *testsdk_ext:*
259 278
@@ -262,8 +281,9 @@ Tests map into the codebase as follows:
262 281
263 - The code for these tests resides in ``meta/lib/oeqa/esdk``. 282 - The code for these tests resides in ``meta/lib/oeqa/esdk``.
264 283
265 - To run the tests, use the following command form: $ bitbake image 284 - To run the tests, use the following command form::
266 -c testsdkext 285
286 $ bitbake image -c testsdkext
267 287
268- *oe-build-perf-test:* 288- *oe-build-perf-test:*
269 289
@@ -272,8 +292,11 @@ Tests map into the codebase as follows:
272 292
273 - The code for these tests resides in ``meta/lib/oeqa/buildperf``. 293 - The code for these tests resides in ``meta/lib/oeqa/buildperf``.
274 294
275 - To run the tests, use the following command form: $ 295 - To run the tests, use the following command form::
276 oe-build-perf-test optionsThe command takes a number of options, 296
297 $ oe-build-perf-test <options>
298
299 The command takes a number of options,
277 such as where to place the test results. The Autobuilder Helper 300 such as where to place the test results. The Autobuilder Helper
278 Scripts include the ``build-perf-test-wrapper`` script with 301 Scripts include the ``build-perf-test-wrapper`` script with
279 examples of how to use the oe-build-perf-test from the command 302 examples of how to use the oe-build-perf-test from the command
@@ -285,9 +308,9 @@ Tests map into the codebase as follows:
285 Use the ``oe-build-perf-report`` command to generate text reports 308 Use the ``oe-build-perf-report`` command to generate text reports
286 and HTML reports with graphs of the performance data. For 309 and HTML reports with graphs of the performance data. For
287 examples, see 310 examples, see
288 `http://downloads.yoctoproject.org/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.html <#>`__ 311 :yocto_dl:`/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.html`
289 and 312 and
290 `http://downloads.yoctoproject.org/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.txt <#>`__. 313 :yocto_dl:`/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.txt`.
291 314
292 - The tests are contained in ``lib/oeqa/buildperf/test_basic.py``. 315 - The tests are contained in ``lib/oeqa/buildperf/test_basic.py``.
293 316
@@ -295,7 +318,7 @@ Test Examples
295============= 318=============
296 319
297This section provides example tests for each of the tests listed in the 320This section provides example tests for each of the tests listed in the
298`How Tests Map to Areas of Code <#test-test-mapping>`__ section. 321:ref:`test-manual/test-manual-intro:How Tests Map to Areas of Code` section.
299 322
300For oeqa tests, testcases for each area reside in the main test 323For oeqa tests, testcases for each area reside in the main test
301directory at ``meta/lib/oeqa/selftest/cases`` directory. 324directory at ``meta/lib/oeqa/selftest/cases`` directory.
@@ -308,14 +331,20 @@ directory.
308``bitbake-selftest`` 331``bitbake-selftest``
309-------------------- 332--------------------
310 333
311A simple test example from ``lib/bb/tests/data.py`` is: class 334A simple test example from ``lib/bb/tests/data.py`` is::
312DataExpansions(unittest.TestCase): def setUp(self): self.d = 335
313bb.data.init() self.d["foo"] = "value_of_foo" self.d["bar"] = 336 class DataExpansions(unittest.TestCase):
314"value_of_bar" self.d["value_of_foo"] = "value_of_'value_of_foo'" def 337 def setUp(self):
315test_one_var(self): val = self.d.expand("${foo}") 338 self.d = bb.data.init()
316self.assertEqual(str(val), "value_of_foo") 339 self.d["foo"] = "value_of_foo"
340 self.d["bar"] = "value_of_bar"
341 self.d["value_of_foo"] = "value_of_'value_of_foo'"
317 342
318In this example, a ```DataExpansions`` <>`__ class of tests is created, 343 def test_one_var(self):
344 val = self.d.expand("${foo}")
345 self.assertEqual(str(val), "value_of_foo")
346
347In this example, a ``DataExpansions`` class of tests is created,
319derived from standard python unittest. The class has a common ``setUp`` 348derived from standard python unittest. The class has a common ``setUp``
320function which is shared by all the tests in the class. A simple test is 349function which is shared by all the tests in the class. A simple test is
321then added to test that when a variable is expanded, the correct value 350then added to test that when a variable is expanded, the correct value
@@ -323,7 +352,7 @@ is found.
323 352
324Bitbake selftests are straightforward python unittest. Refer to the 353Bitbake selftests are straightforward python unittest. Refer to the
325Python unittest documentation for additional information on writing 354Python unittest documentation for additional information on writing
326these tests at: `https://docs.python.org/3/library/unittest.html <#>`__. 355these tests at: https://docs.python.org/3/library/unittest.html.
327 356
328.. _oe-selftest-example: 357.. _oe-selftest-example:
329 358
@@ -334,14 +363,15 @@ These tests are more complex due to the setup required behind the scenes
334for full builds. Rather than directly using Python's unittest, the code 363for full builds. Rather than directly using Python's unittest, the code
335wraps most of the standard objects. The tests can be simple, such as 364wraps most of the standard objects. The tests can be simple, such as
336testing a command from within the OE build environment using the 365testing a command from within the OE build environment using the
337following example:class BitbakeLayers(OESelftestTestCase): def 366following example::
338test_bitbakelayers_showcrossdepends(self): result = 367
339runCmd('bitbake-layers show-cross-depends') self.assertTrue('aspell' in 368 class BitbakeLayers(OESelftestTestCase):
340result.output, msg = "No dependencies were shown. bitbake-layers 369 def test_bitbakelayers_showcrossdepends(self):
341show-cross-depends output: %s"% result.output) 370 result = runCmd('bitbake-layers show-cross-depends')
371 self.assertTrue('aspell' in result.output, msg = "No dependencies were shown. bitbake-layers show-cross-depends output: %s"% result.output)
342 372
343This example, taken from ``meta/lib/oeqa/selftest/cases/bblayers.py``, 373This example, taken from ``meta/lib/oeqa/selftest/cases/bblayers.py``,
344creates a testcase from the ```OESelftestTestCase`` <>`__ class, derived 374creates a testcase from the ``OESelftestTestCase`` class, derived
345from ``unittest.TestCase``, which runs the ``bitbake-layers`` command 375from ``unittest.TestCase``, which runs the ``bitbake-layers`` command
346and checks the output to ensure it contains something we know should be 376and checks the output to ensure it contains something we know should be
347here. 377here.
@@ -367,7 +397,7 @@ so tests within a given test class should always run in the same build,
367while tests in different classes or modules may be split into different 397while tests in different classes or modules may be split into different
368builds. There is no data store available for these tests since the tests 398builds. There is no data store available for these tests since the tests
369launch the ``bitbake`` command and exist outside of its context. As a 399launch the ``bitbake`` command and exist outside of its context. As a
370result, common bitbake library functions (bb.*) are also unavailable. 400result, common bitbake library functions (bb.\*) are also unavailable.
371 401
372.. _testimage-example: 402.. _testimage-example:
373 403
@@ -378,14 +408,18 @@ These tests are run once an image is up and running, either on target
378hardware or under QEMU. As a result, they are assumed to be running in a 408hardware or under QEMU. As a result, they are assumed to be running in a
379target image environment, as opposed to a host build environment. A 409target image environment, as opposed to a host build environment. A
380simple example from ``meta/lib/oeqa/runtime/cases/python.py`` contains 410simple example from ``meta/lib/oeqa/runtime/cases/python.py`` contains
381the following:class PythonTest(OERuntimeTestCase): 411the following::
382@OETestDepends(['ssh.SSHTest.test_ssh']) @OEHasPackage(['python3-core']) 412
383def test_python3(self): cmd = "python3 -c \\"import codecs; 413 class PythonTest(OERuntimeTestCase):
384print(codecs.encode('Uryyb, jbeyq', 'rot13'))\"" status, output = 414 @OETestDepends(['ssh.SSHTest.test_ssh'])
385self.target.run(cmd) msg = 'Exit status was not 0. Output: %s' % output 415 @OEHasPackage(['python3-core'])
386self.assertEqual(status, 0, msg=msg) 416 def test_python3(self):
387 417 cmd = "python3 -c \\"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
388In this example, the ```OERuntimeTestCase`` <>`__ class wraps 418 status, output = self.target.run(cmd)
419 msg = 'Exit status was not 0. Output: %s' % output
420 self.assertEqual(status, 0, msg=msg)
421
422In this example, the ``OERuntimeTestCase`` class wraps
389``unittest.TestCase``. Within the test, ``self.target`` represents the 423``unittest.TestCase``. Within the test, ``self.target`` represents the
390target system, where commands can be run on it using the ``run()`` 424target system, where commands can be run on it using the ``run()``
391method. 425method.
@@ -402,19 +436,30 @@ the image.
402 436
403These tests are run against built extensible SDKs (eSDKs). The tests can 437These tests are run against built extensible SDKs (eSDKs). The tests can
404assume that the eSDK environment has already been setup. An example from 438assume that the eSDK environment has already been setup. An example from
405``meta/lib/oeqa/sdk/cases/devtool.py`` contains the following:class 439``meta/lib/oeqa/sdk/cases/devtool.py`` contains the following::
406DevtoolTest(OESDKExtTestCase): @classmethod def setUpClass(cls): 440
407myapp_src = os.path.join(cls.tc.esdk_files_dir, "myapp") cls.myapp_dst = 441 class DevtoolTest(OESDKExtTestCase):
408os.path.join(cls.tc.sdk_dir, "myapp") shutil.copytree(myapp_src, 442 @classmethod def setUpClass(cls):
409cls.myapp_dst) subprocess.check_output(['git', 'init', '.'], 443 myapp_src = os.path.join(cls.tc.esdk_files_dir, "myapp")
410cwd=cls.myapp_dst) subprocess.check_output(['git', 'add', '.'], 444 cls.myapp_dst = os.path.join(cls.tc.sdk_dir, "myapp")
411cwd=cls.myapp_dst) subprocess.check_output(['git', 'commit', '-m', 445 shutil.copytree(myapp_src, cls.myapp_dst)
412"'test commit'"], cwd=cls.myapp_dst) @classmethod def 446 subprocess.check_output(['git', 'init', '.'], cwd=cls.myapp_dst)
413tearDownClass(cls): shutil.rmtree(cls.myapp_dst) def 447 subprocess.check_output(['git', 'add', '.'], cwd=cls.myapp_dst)
414\_test_devtool_build(self, directory): self._run('devtool add myapp %s' 448 subprocess.check_output(['git', 'commit', '-m', "'test commit'"], cwd=cls.myapp_dst)
415% directory) try: self._run('devtool build myapp') finally: 449
416self._run('devtool reset myapp') def test_devtool_build_make(self): 450 @classmethod
417self._test_devtool_build(self.myapp_dst)In this example, the ``devtool`` 451 def tearDownClass(cls):
452 shutil.rmtree(cls.myapp_dst)
453 def _test_devtool_build(self, directory):
454 self._run('devtool add myapp %s' % directory)
455 try:
456 self._run('devtool build myapp')
457 finally:
458 self._run('devtool reset myapp')
459 def test_devtool_build_make(self):
460 self._test_devtool_build(self.myapp_dst)
461
462In this example, the ``devtool``
418command is tested to see whether a sample application can be built with 463command is tested to see whether a sample application can be built with
419the ``devtool build`` command within the eSDK. 464the ``devtool build`` command within the eSDK.
420 465
@@ -426,14 +471,20 @@ the ``devtool build`` command within the eSDK.
426These tests are run against built SDKs. The tests can assume that an SDK 471These tests are run against built SDKs. The tests can assume that an SDK
427has already been extracted and its environment file has been sourced. A 472has already been extracted and its environment file has been sourced. A
428simple example from ``meta/lib/oeqa/sdk/cases/python2.py`` contains the 473simple example from ``meta/lib/oeqa/sdk/cases/python2.py`` contains the
429following:class Python3Test(OESDKTestCase): def setUp(self): if not 474following::
430(self.tc.hasHostPackage("nativesdk-python3-core") or 475
431self.tc.hasHostPackage("python3-core-native")): raise 476 class Python3Test(OESDKTestCase):
432unittest.SkipTest("No python3 package in the SDK") def 477 def setUp(self):
433test_python3(self): cmd = "python3 -c \\"import codecs; 478 if not (self.tc.hasHostPackage("nativesdk-python3-core") or
434print(codecs.encode('Uryyb, jbeyq', 'rot13'))\"" output = self._run(cmd) 479 self.tc.hasHostPackage("python3-core-native")):
435self.assertEqual(output, "Hello, world\n")In this example, if 480 raise unittest.SkipTest("No python3 package in the SDK")
436nativesdk-python3-core has been installed into the SDK, the code runs 481
482 def test_python3(self):
483 cmd = "python3 -c \\"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
484 output = self._run(cmd)
485 self.assertEqual(output, "Hello, world\n")
486
487In this example, if nativesdk-python3-core has been installed into the SDK, the code runs
437the python3 interpreter with a basic command to check it is working 488the python3 interpreter with a basic command to check it is working
438correctly. The test would only run if python3 is installed in the SDK. 489correctly. The test would only run if python3 is installed in the SDK.
439 490
@@ -444,17 +495,25 @@ correctly. The test would only run if python3 is installed in the SDK.
444 495
445The performance tests usually measure how long operations take and the 496The performance tests usually measure how long operations take and the
446resource utilisation as that happens. An example from 497resource utilisation as that happens. An example from
447``meta/lib/oeqa/buildperf/test_basic.py`` contains the following:class 498``meta/lib/oeqa/buildperf/test_basic.py`` contains the following::
448Test3(BuildPerfTestCase): def test3(self): """Bitbake parsing (bitbake 499
449-p)""" # Drop all caches and parse self.rm_cache() 500 class Test3(BuildPerfTestCase):
450oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True) 501 def test3(self):
451self.measure_cmd_resources(['bitbake', '-p'], 'parse_1', 'bitbake -p (no 502 """Bitbake parsing (bitbake -p)"""
452caches)') # Drop tmp/cache 503 # Drop all caches and parse
453oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True) 504 self.rm_cache()
454self.measure_cmd_resources(['bitbake', '-p'], 'parse_2', 'bitbake -p (no 505 oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
455tmp/cache)') # Parse with fully cached data 506 self.measure_cmd_resources(['bitbake', '-p'], 'parse_1',
456self.measure_cmd_resources(['bitbake', '-p'], 'parse_3', 'bitbake -p 507 'bitbake -p (no caches)')
457(cached)')This example shows how three specific parsing timings are 508 # Drop tmp/cache
509 oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
510 self.measure_cmd_resources(['bitbake', '-p'], 'parse_2',
511 'bitbake -p (no tmp/cache)')
512 # Parse with fully cached data
513 self.measure_cmd_resources(['bitbake', '-p'], 'parse_3',
514 'bitbake -p (cached)')
515
516This example shows how three specific parsing timings are
458measured, with and without various caches, to show how BitBake’s parsing 517measured, with and without various caches, to show how BitBake’s parsing
459performance trends over time. 518performance trends over time.
460 519
diff --git a/documentation/test-manual/test-manual-test-process.rst b/documentation/test-manual/test-manual-test-process.rst
index df825f8efa..96e71bf314 100644
--- a/documentation/test-manual/test-manual-test-process.rst
+++ b/documentation/test-manual/test-manual-test-process.rst
@@ -23,11 +23,11 @@ We have two broad categories of test builds, including "full" and
23"quick". On the Autobuilder, these can be seen as "a-quick" and 23"quick". On the Autobuilder, these can be seen as "a-quick" and
24"a-full", simply for ease of sorting in the UI. Use our Autobuilder 24"a-full", simply for ease of sorting in the UI. Use our Autobuilder
25console view to see where me manage most test-related items, available 25console view to see where me manage most test-related items, available
26at: `https://autobuilder.yoctoproject.org/typhoon/#/console <#>`__. 26at: :yocto_ab:`/typhoon/#/console`.
27 27
28Builds are triggered manually when the test branches are ready. The 28Builds are triggered manually when the test branches are ready. The
29builds are monitored by the SWAT team. For additional information, see 29builds are monitored by the SWAT team. For additional information, see
30`https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team <#>`__. 30:yocto_wiki:`/wiki/Yocto_Build_Failure_Swat_Team`.
31If successful, the changes would usually be merged to the ``master`` 31If successful, the changes would usually be merged to the ``master``
32branch. If not successful, someone would respond to the changes on the 32branch. If not successful, someone would respond to the changes on the
33mailing list explaining that there was a failure in testing. The choice 33mailing list explaining that there was a failure in testing. The choice
@@ -37,9 +37,9 @@ which the result was required.
37The Autobuilder does build the ``master`` branch once daily for several 37The Autobuilder does build the ``master`` branch once daily for several
38reasons, in particular, to ensure the current ``master`` branch does 38reasons, in particular, to ensure the current ``master`` branch does
39build, but also to keep ``yocto-testresults`` 39build, but also to keep ``yocto-testresults``
40(`http://git.yoctoproject.org/cgit.cgi/yocto-testresults/ <#>`__), 40(:yocto_git:`/cgit.cgi/yocto-testresults/`),
41buildhistory 41buildhistory
42(`http://git.yoctoproject.org/cgit.cgi/poky-buildhistory/ <#>`__), and 42(:yocto_git:`/cgit.cgi/poky-buildhistory/`), and
43our sstate up to date. On the weekend, there is a master-next build 43our sstate up to date. On the weekend, there is a master-next build
44instead to ensure the test results are updated for the less frequently 44instead to ensure the test results are updated for the less frequently
45run targets. 45run targets.
@@ -47,7 +47,7 @@ run targets.
47Performance builds (buildperf-\* targets in the console) are triggered 47Performance builds (buildperf-\* targets in the console) are triggered
48separately every six hours and automatically push their results to the 48separately every six hours and automatically push their results to the
49buildstats repository at: 49buildstats repository at:
50`http://git.yoctoproject.org/cgit.cgi/yocto-buildstats/ <#>`__. 50:yocto_git:`/cgit.cgi/yocto-buildstats/`.
51 51
52The 'quick' targets have been selected to be the ones which catch the 52The 'quick' targets have been selected to be the ones which catch the
53most failures or give the most valuable data. We run 'fast' ptests in 53most failures or give the most valuable data. We run 'fast' ptests in
@@ -56,8 +56,6 @@ target doesn't include \*-lsb builds for all architectures, some world
56builds and doesn't trigger performance tests or ltp testing. The full 56builds and doesn't trigger performance tests or ltp testing. The full
57build includes all these things and is slower but more comprehensive. 57build includes all these things and is slower but more comprehensive.
58 58
59.. _test-yocto-project-autobuilder-overview:
60
61Release Builds 59Release Builds
62============== 60==============
63 61
@@ -76,7 +74,7 @@ box to "generate an email to QA"is also checked.
76When the build completes, an email is sent out using the send-qa-email 74When the build completes, an email is sent out using the send-qa-email
77script in the ``yocto-autobuilder-helper`` repository to the list of 75script in the ``yocto-autobuilder-helper`` repository to the list of
78people configured for that release. Release builds are placed into a 76people configured for that release. Release builds are placed into a
79directory in `https://autobuilder.yocto.io/pub/releases <#>`__ on the 77directory in https://autobuilder.yocto.io/pub/releases on the
80Autobuilder which is included in the email. The process from here is 78Autobuilder which is included in the email. The process from here is
81more manual and control is effectively passed to release engineering. 79more manual and control is effectively passed to release engineering.
82The next steps include: 80The next steps include:
diff --git a/documentation/test-manual/test-manual-understand-autobuilder.rst b/documentation/test-manual/test-manual-understand-autobuilder.rst
index 8d3c844156..0809190b5b 100644
--- a/documentation/test-manual/test-manual-understand-autobuilder.rst
+++ b/documentation/test-manual/test-manual-understand-autobuilder.rst
@@ -10,7 +10,7 @@ Execution Flow within the Autobuilder
10The “a-full” and “a-quick” targets are the usual entry points into the 10The “a-full” and “a-quick” targets are the usual entry points into the
11Autobuilder and it makes sense to follow the process through the system 11Autobuilder and it makes sense to follow the process through the system
12starting there. This is best visualised from the Autobuilder Console 12starting there. This is best visualised from the Autobuilder Console
13view (`https://autobuilder.yoctoproject.org/typhoon/#/console <#>`__). 13view (:yocto_ab:`/typhoon/#/console`).
14 14
15Each item along the top of that view represents some “target build” and 15Each item along the top of that view represents some “target build” and
16these targets are all run in parallel. The ‘full’ build will trigger the 16these targets are all run in parallel. The ‘full’ build will trigger the
@@ -20,32 +20,48 @@ each of those targets on a seperate buildbot worker. To understand the
20configuration, you need to look at the entry on ``config.json`` file 20configuration, you need to look at the entry on ``config.json`` file
21within the ``yocto-autobuilder-helper`` repository. The targets are 21within the ``yocto-autobuilder-helper`` repository. The targets are
22defined in the ‘overrides’ section, a quick example could be qemux86-64 22defined in the ‘overrides’ section, a quick example could be qemux86-64
23which looks like:"qemux86-64" : { "MACHINE" : "qemux86-64", "TEMPLATE" : 23which looks like::
24"arch-qemu", "step1" : { "extravars" : [ "IMAGE_FSTYPES_append = ' wic 24
25wic.bmap'" ] } },And to expand that, you need the “arch-qemu” entry from 25 "qemux86-64" : {
26the “templates” section, which looks like:"arch-qemu" : { "BUILDINFO" : 26 "MACHINE" : "qemux86-64",
27true, "BUILDHISTORY" : true, "step1" : { "BBTARGETS" : "core-image-sato 27 "TEMPLATE" : "arch-qemu",
28core-image-sato-dev core-image-sato-sdk core-image-minimal 28 "step1" : {
29core-image-minimal-dev core-image-sato:do_populate_sdk", "SANITYTARGETS" 29 "extravars" : [
30: "core-image-minimal:do_testimage core-image-sato:do_testimage 30 "IMAGE_FSTYPES_append = ' wic wic.bmap'"
31core-image-sato-sdk:do_testimage core-image-sato:do_testsdk" }, "step2" 31 ]
32: { "SDKMACHINE" : "x86_64", "BBTARGETS" : 32 }
33"core-image-sato:do_populate_sdk core-image-minimal:do_populate_sdk_ext 33 },
34core-image-sato:do_populate_sdk_ext", "SANITYTARGETS" : 34
35"core-image-sato:do_testsdk core-image-minimal:do_testsdkext 35And to expand that, you need the “arch-qemu” entry from
36core-image-sato:do_testsdkext" }, "step3" : { "BUILDHISTORY" : false, 36the “templates” section, which looks like::
37"EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; DISPLAY=:1 oe-selftest 37
38${HELPERSTMACHTARGS} -j 15"], "ADDLAYER" : 38 "arch-qemu" : {
39["${BUILDDIR}/../meta-selftest"] } },Combining these two entries you can 39 "BUILDINFO" : true,
40see that “qemux86-64” is a three step build where the 40 "BUILDHISTORY" : true,
41``bitbake BBTARGETS`` would be run, then ``bitbake 41 "step1" : {
42 SANITYTARGETS`` for each step; all for 42 "BBTARGETS" : "core-image-sato core-image-sato-dev core-image-sato-sdk core-image-minimal core-image-minimal-dev core-image-sato:do_populate_sdk",
43 "SANITYTARGETS" : "core-image-minimal:do_testimage core-image-sato:do_testimage core-image-sato-sdk:do_testimage core-image-sato:do_testsdk"
44 },
45 "step2" : {
46 "SDKMACHINE" : "x86_64",
47 "BBTARGETS" : "core-image-sato:do_populate_sdk core-image-minimal:do_populate_sdk_ext core-image-sato:do_populate_sdk_ext",
48 "SANITYTARGETS" : "core-image-sato:do_testsdk core-image-minimal:do_testsdkext core-image-sato:do_testsdkext"
49 },
50 "step3" : {
51 "BUILDHISTORY" : false,
52 "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; DISPLAY=:1 oe-selftest ${HELPERSTMACHTARGS} -j 15"],
53 "ADDLAYER" : ["${BUILDDIR}/../meta-selftest"]
54 }
55 },
56
57Combining these two entries you can see that “qemux86-64” is a three step build where the
58``bitbake BBTARGETS`` would be run, then ``bitbake SANITYTARGETS`` for each step; all for
43``MACHINE=”qemx86-64”`` but with differing SDKMACHINE settings. In step 59``MACHINE=”qemx86-64”`` but with differing SDKMACHINE settings. In step
441 an extra variable is added to the ``auto.conf`` file to enable wic 601 an extra variable is added to the ``auto.conf`` file to enable wic
45image generation. 61image generation.
46 62
47While not every detail of this is covered here, you can see how the 63While not every detail of this is covered here, you can see how the
48templating mechanism allows quite complex configurations to be built up 64template mechanism allows quite complex configurations to be built up
49yet allows duplication and repetition to be kept to a minimum. 65yet allows duplication and repetition to be kept to a minimum.
50 66
51The different build targets are designed to allow for parallelisation, 67The different build targets are designed to allow for parallelisation,
@@ -66,13 +82,13 @@ For each given target in a build, the Autobuilder executes several
66steps. These are configured in ``yocto-autobuilder2/builders.py`` and 82steps. These are configured in ``yocto-autobuilder2/builders.py`` and
67roughly consist of: 83roughly consist of:
68 84
691. *Run ``clobberdir``* 85#. *Run clobberdir*.
70 86
71 This cleans out any previous build. Old builds are left around to 87 This cleans out any previous build. Old builds are left around to
72 allow easier debugging of failed builds. For additional information, 88 allow easier debugging of failed builds. For additional information,
73 see ```clobberdir`` <#test-clobberdir>`__. 89 see :ref:`test-manual/test-manual-understand-autobuilder:clobberdir`.
74 90
752. *Obtain yocto-autobuilder-helper* 91#. *Obtain yocto-autobuilder-helper*
76 92
77 This step clones the ``yocto-autobuilder-helper`` git repository. 93 This step clones the ``yocto-autobuilder-helper`` git repository.
78 This is necessary to prevent the requirement to maintain all the 94 This is necessary to prevent the requirement to maintain all the
@@ -80,12 +96,12 @@ roughly consist of:
80 matches the release being built so we can support older releases and 96 matches the release being built so we can support older releases and
81 still make changes in newer ones. 97 still make changes in newer ones.
82 98
833. *Write layerinfo.json* 99#. *Write layerinfo.json*
84 100
85 This transfers data in the Buildbot UI when the build was configured 101 This transfers data in the Buildbot UI when the build was configured
86 to the Helper. 102 to the Helper.
87 103
884. *Call scripts/shared-repo-unpack* 104#. *Call scripts/shared-repo-unpack*
89 105
90 This is a call into the Helper scripts to set up a checkout of all 106 This is a call into the Helper scripts to set up a checkout of all
91 the pieces this build might need. It might clone the BitBake 107 the pieces this build might need. It might clone the BitBake
@@ -94,7 +110,7 @@ roughly consist of:
94 from the ``layerinfo.json`` file to help understand the 110 from the ``layerinfo.json`` file to help understand the
95 configuration. It will also use a local cache of repositories to 111 configuration. It will also use a local cache of repositories to
96 speed up the clone checkouts. For additional information, see 112 speed up the clone checkouts. For additional information, see
97 `Autobuilder Clone Cache <#test-autobuilder-clone-cache>`__. 113 :ref:`test-manual/test-manual-understand-autobuilder:Autobuilder Clone Cache`.
98 114
99 This step has two possible modes of operation. If the build is part 115 This step has two possible modes of operation. If the build is part
100 of a parent build, its possible that all the repositories needed may 116 of a parent build, its possible that all the repositories needed may
@@ -114,7 +130,7 @@ roughly consist of:
114 available, it would clone from the cache and the upstreams as 130 available, it would clone from the cache and the upstreams as
115 necessary. This is considered the fallback mode. 131 necessary. This is considered the fallback mode.
116 132
1175. *Call scripts/run-config* 133#. *Call scripts/run-config*
118 134
119 This is another call into the Helper scripts where its expected that 135 This is another call into the Helper scripts where its expected that
120 the main functionality of this target will be executed. 136 the main functionality of this target will be executed.
@@ -137,8 +153,7 @@ special script that moves files to a special location, rather than
137deleting them. Files in this location are deleted by an ``rm`` command, 153deleting them. Files in this location are deleted by an ``rm`` command,
138which is run under ``ionice -c 3``. For example, the deletion only 154which is run under ``ionice -c 3``. For example, the deletion only
139happens when there is idle IO capacity on the Worker. The Autobuilder 155happens when there is idle IO capacity on the Worker. The Autobuilder
140Worker Janitor runs this deletion. See `Autobuilder Worker 156Worker Janitor runs this deletion. See :ref:`test-manual/test-manual-understand-autobuilder:Autobuilder Worker Janitor`.
141Janitor <#test-autobuilder-worker-janitor>`__.
142 157
143.. _test-autobuilder-clone-cache: 158.. _test-autobuilder-clone-cache:
144 159
@@ -150,8 +165,7 @@ on the Autobuilder. We therefore have a stash of commonly used
150repositories pre-cloned on the Workers. Data is fetched from these 165repositories pre-cloned on the Workers. Data is fetched from these
151during clones first, then "topped up" with later revisions from any 166during clones first, then "topped up" with later revisions from any
152upstream when necesary. The cache is maintained by the Autobuilder 167upstream when necesary. The cache is maintained by the Autobuilder
153Worker Janitor. See `Autobuilder Worker 168Worker Janitor. See :ref:`test-manual/test-manual-understand-autobuilder:Autobuilder Worker Janitor`.
154Janitor <#test-autobuilder-worker-janitor>`__.
155 169
156.. _test-autobuilder-worker-janitor: 170.. _test-autobuilder-worker-janitor:
157 171
@@ -159,8 +173,7 @@ Autobuilder Worker Janitor
159-------------------------- 173--------------------------
160 174
161This is a process running on each Worker that performs two basic 175This is a process running on each Worker that performs two basic
162operations, including background file deletion at IO idle (see `Target 176operations, including background file deletion at IO idle (see :ref:`test-manual/test-manual-understand-autobuilder:Autobuilder Target Execution Overview`: Run clobberdir) and
163Execution: clobberdir <#test-list-tgt-exec-clobberdir>`__) and
164maintainenance of a cache of cloned repositories to improve the speed 177maintainenance of a cache of cloned repositories to improve the speed
165the system can checkout repositories. 178the system can checkout repositories.
166 179
@@ -181,7 +194,7 @@ Shared SSTATE_DIR
181 194
182The Workers are all connected over NFS which allows the ``sstate`` 195The Workers are all connected over NFS which allows the ``sstate``
183directory to be shared between them. This means once a Worker has built 196directory to be shared between them. This means once a Worker has built
184an artefact, all the others can benefit from it. Usage of the directory 197an artifact, all the others can benefit from it. Usage of the directory
185within the directory is designed for sharing over NFS. 198within the directory is designed for sharing over NFS.
186 199
187.. _test-resulttool: 200.. _test-resulttool:
@@ -198,7 +211,7 @@ Resulttool is part of OpenEmbedded-Core and is used to manipulate these
198json results files. It has the ability to merge files together, display 211json results files. It has the ability to merge files together, display
199reports of the test results and compare different result files. 212reports of the test results and compare different result files.
200 213
201For details, see `https://wiki.yoctoproject.org/wiki/Resulttool <#>`__. 214For details, see :yocto_wiki:`/wiki/Resulttool`.
202 215
203.. _test-run-config-tgt-execution: 216.. _test-run-config-tgt-execution:
204 217
@@ -209,50 +222,46 @@ The ``scripts/run-config`` execution is where most of the work within
209the Autobuilder happens. It runs through a number of steps; the first 222the Autobuilder happens. It runs through a number of steps; the first
210are general setup steps that are run once and include: 223are general setup steps that are run once and include:
211 224
2121. Set up any ``buildtools-tarball`` if configured. 225#. Set up any ``buildtools-tarball`` if configured.
213 226
2142. Call "buildhistory-init" if buildhistory is configured. 227#. Call "buildhistory-init" if buildhistory is configured.
215 228
216For each step that is configured in ``config.json``, it will perform the 229For each step that is configured in ``config.json``, it will perform the
217following: 230following:
218 231
219## WRITER's question: What does "logging in as stepXa" and others refer 232#. Add any layers that are specified using the
220to below? ##
221
2221. Add any layers that are specified using the
223 ``bitbake-layers add-layer`` command (logging as stepXa) 233 ``bitbake-layers add-layer`` command (logging as stepXa)
224 234
2252. Call the ``scripts/setup-config`` script to generate the necessary 235#. Call the ``scripts/setup-config`` script to generate the necessary
226 ``auto.conf`` configuration file for the build 236 ``auto.conf`` configuration file for the build
227 237
2283. Run the ``bitbake BBTARGETS`` command (logging as stepXb) 238#. Run the ``bitbake BBTARGETS`` command (logging as stepXb)
229 239
2304. Run the ``bitbake SANITYTARGETS`` command (logging as stepXc) 240#. Run the ``bitbake SANITYTARGETS`` command (logging as stepXc)
231 241
2325. Run the ``EXTRACMDS`` command, which are run within the BitBake build 242#. Run the ``EXTRACMDS`` command, which are run within the BitBake build
233 environment (logging as stepXd) 243 environment (logging as stepXd)
234 244
2356. Run the ``EXTRAPLAINCMDS`` command(s), which are run outside the 245#. Run the ``EXTRAPLAINCMDS`` command(s), which are run outside the
236 BitBake build environment (logging as stepXd) 246 BitBake build environment (logging as stepXd)
237 247
2387. Remove any layers added in `step 248#. Remove any layers added in step
239 1 <#test-run-config-add-layers-step>`__ using the 249 1 using the ``bitbake-layers remove-layer`` command (logging as stepXa)
240 ``bitbake-layers remove-layer`` command (logging as stepXa)
241 250
242Once the execution steps above complete, ``run-config`` executes a set 251Once the execution steps above complete, ``run-config`` executes a set
243of post-build steps, including: 252of post-build steps, including:
244 253
2451. Call ``scripts/publish-artifacts`` to collect any output which is to 254#. Call ``scripts/publish-artifacts`` to collect any output which is to
246 be saved from the build. 255 be saved from the build.
247 256
2482. Call ``scripts/collect-results`` to collect any test results to be 257#. Call ``scripts/collect-results`` to collect any test results to be
249 saved from the build. 258 saved from the build.
250 259
2513. Call ``scripts/upload-error-reports`` to send any error reports 260#. Call ``scripts/upload-error-reports`` to send any error reports
252 generated to the remote server. 261 generated to the remote server.
253 262
2544. Cleanup the build directory using 263#. Cleanup the build directory using
255 ```clobberdir`` <#test-clobberdir>`__ if the build was successful, 264 :ref:`test-manual/test-manual-understand-autobuilder:clobberdir` if the build was successful,
256 else rename it to “build-renamed” for potential future debugging. 265 else rename it to “build-renamed” for potential future debugging.
257 266
258.. _test-deploying-yp-autobuilder: 267.. _test-deploying-yp-autobuilder:
@@ -279,11 +288,18 @@ The standard ``config.json`` minimally attempts to allow substitution of
279the paths. The Helper script repository includes a 288the paths. The Helper script repository includes a
280``local-example.json`` file to show how you could override these from a 289``local-example.json`` file to show how you could override these from a
281separate configuration file. Pass the following into the environment of 290separate configuration file. Pass the following into the environment of
282the Autobuilder:$ ABHELPER_JSON="config.json local-example.json"As 291the Autobuilder::
283another example, you could also pass the following into the 292
284environment:$ ABHELPER_JSON="config.json /some/location/local.json"One 293 $ ABHELPER_JSON="config.json local-example.json"
285issue users often run into is validation of the ``config.json`` files. A 294
295As another example, you could also pass the following into the
296environment::
297
298 $ ABHELPER_JSON="config.json /some/location/local.json"
299
300One issue users often run into is validation of the ``config.json`` files. A
286tip for minimizing issues from invalid json files is to use a Git 301tip for minimizing issues from invalid json files is to use a Git
287``pre-commit-hook.sh`` script to verify the JSON file before committing 302``pre-commit-hook.sh`` script to verify the JSON file before committing
288it. Create a symbolic link as follows:$ ln -s 303it. Create a symbolic link as follows::
289../../scripts/pre-commit-hook.sh .git/hooks/pre-commit 304
305 $ ln -s ../../scripts/pre-commit-hook.sh .git/hooks/pre-commit