1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
|
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Yocto Project Test Environment Manual</title><link rel="stylesheet" type="text/css" href="test-manual-style.css" /><meta name="generator" content="DocBook XSL Stylesheets V1.76.1" /></head><body><div xml:lang="en" class="book" title="Yocto Project Test Environment Manual" id="test-manual" lang="en"><div class="titlepage"><div><div><h1 class="title">
Yocto Project Test Environment Manual
</h1></div><div><div class="authorgroup">
<div class="author"><h3 class="author"><span class="firstname">Scott</span> <span class="surname">Rifenbark</span></h3><div class="affiliation">
<span class="orgname">Scotty's Documentation Services, INC<br /></span>
</div><code class="email"><<a class="email" href="mailto:srifenbark@gmail.com">srifenbark@gmail.com</a>></code></div>
</div></div><div><p class="copyright">Copyright © 2010-2018 Linux Foundation</p></div><div><div class="legalnotice" title="Legal Notice"><a id="idm45709743826400"></a>
<p>
Permission is granted to copy, distribute and/or modify this document under
the terms of the <a class="ulink" href="http://creativecommons.org/licenses/by-sa/2.0/uk/" target="_top">
Creative Commons Attribution-Share Alike 2.0 UK: England & Wales</a> as published by
Creative Commons.
</p>
<div class="note" title="Manual Notes" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Manual Notes</h3>
<div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
This version of the
<span class="emphasis"><em>Yocto Project Test Environment Manual</em></span>
is for the 2.6 release of the
Yocto Project.
To be sure you have the latest version of the manual
for this release, go to the
<a class="ulink" href="http://www.yoctoproject.org/documentation" target="_top">Yocto Project documentation page</a>
and select the manual from that site.
Manuals from the site are more up-to-date than manuals
derived from the Yocto Project released TAR files.
</p></li><li class="listitem"><p>
If you located this manual through a web search, the
version of the manual might not be the one you want
(e.g. the search might have returned a manual much
older than the Yocto Project version with which you
are working).
You can see all Yocto Project major releases by
visiting the
<a class="ulink" href="https://wiki.yoctoproject.org/wiki/Releases" target="_top">Releases</a>
page.
If you need a version of this manual for a different
Yocto Project release, visit the
<a class="ulink" href="http://www.yoctoproject.org/documentation" target="_top">Yocto Project documentation page</a>
and select the manual set by using the
"ACTIVE RELEASES DOCUMENTATION" or "DOCUMENTS ARCHIVE"
pull-down menus.
</p></li><li class="listitem"><p>
To report any inaccuracies or problems with this
manual, send an email to the Yocto Project
discussion group at
<code class="filename">yocto@yoctoproject.com</code> or log into
the freenode <code class="filename">#yocto</code> channel.
</p></li></ul></div>
</div>
</div></div><div><div class="revhistory"><table border="1" width="100%" summary="Revision history"><tr><th align="left" valign="top" colspan="2"><strong>Revision History</strong></th></tr>
<tr><td align="left">Revision 2.7</td><td align="left">TBD</td></tr><tr><td align="left" colspan="2">Released with the Yocto Project 2.7 Release.</td></tr>
</table></div></div></div><hr /></div><div class="toc"><p><strong>Table of Contents</strong></p><dl><dt><span class="chapter"><a href="#test-manual-intro">1. The Yocto Project Test Environment Manual</a></span></dt><dd><dl><dt><span class="section"><a href="#test-welcome">1.1. Welcome</a></span></dt><dt><span class="section"><a href="#test-yocto-project-autobuilder-overview">1.2. Yocto Project Autobuilder Overview</a></span></dt><dt><span class="section"><a href="#test-project-tests">1.3. Yocto Project Tests - Type Overview</a></span></dt><dt><span class="section"><a href="#test-test-mapping">1.4. How Tests Map to Areas of Code</a></span></dt><dt><span class="section"><a href="#test-examples">1.5. Test Examples</a></span></dt><dd><dl><dt><span class="section"><a href="#bitbake-selftest-example">1.5.1. <code class="filename">bitbake-selftest</code></a></span></dt><dt><span class="section"><a href="#oe-selftest-example">1.5.2. <code class="filename">oe-selftest</code></a></span></dt><dt><span class="section"><a href="#testimage-example">1.5.3. <code class="filename">testimage</code></a></span></dt><dt><span class="section"><a href="#testsdk_ext-example">1.5.4. <code class="filename">testsdk_ext</code></a></span></dt><dt><span class="section"><a href="#testsdk-example">1.5.5. <code class="filename">testsdk</code></a></span></dt><dt><span class="section"><a href="#oe-build-perf-test-example">1.5.6. <code class="filename">oe-build-perf-test</code></a></span></dt></dl></dd><dt><span class="section"><a href="#some-id">1.6. New Section on the Periodic Builds</a></span></dt><dt><span class="section"><a href="#test-configuring-and-triggering-autobuilder-helper-build-scripts">1.7. Configuring and Triggering Autobuilder Helper Build Scripts</a></span></dt><dt><span class="section"><a href="#test-deploying-yocto-autobuilder">1.8. Deploying Yocto Autobuilder</a></span></dt><dd><dl><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-controller">1.8.1. Upstream Autobuilder Deployment on the Controller</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-worker">1.8.2. Upstream Autobuilder Deployment on the Worker</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-no-upstream-users">1.8.3. Upstream Autobuilder Deployment No Upstream Users</a></span></dt></dl></dd><dt><span class="section"><a href="#test-headless-sanity-tests">1.9. Setting Up Headless Sanity Tests</a></span></dt><dt><span class="section"><a href="#test-adding-additional-build-workers">1.10. Adding Additional Build Workers</a></span></dt><dt><span class="section"><a href="#test-setting-up-build-history">1.11. Setting Up Build History</a></span></dt><dt><span class="section"><a href="#test-some-more-notes">1.12. Some More Notes</a></span></dt><dt><span class="section"><a href="#test-yocto-project-helper">1.13. Yocto Project Autobuilder Helper Scripts</a></span></dt></dl></dd></dl></div>
<div class="chapter" title="Chapter 1. The Yocto Project Test Environment Manual" id="test-manual-intro"><div class="titlepage"><div><div><h2 class="title">Chapter 1. The Yocto Project Test Environment Manual<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-manual-intro">¶</a></span></h2></div></div></div><div class="toc"><p><strong>Table of Contents</strong></p><dl><dt><span class="section"><a href="#test-welcome">1.1. Welcome</a></span></dt><dt><span class="section"><a href="#test-yocto-project-autobuilder-overview">1.2. Yocto Project Autobuilder Overview</a></span></dt><dt><span class="section"><a href="#test-project-tests">1.3. Yocto Project Tests - Type Overview</a></span></dt><dt><span class="section"><a href="#test-test-mapping">1.4. How Tests Map to Areas of Code</a></span></dt><dt><span class="section"><a href="#test-examples">1.5. Test Examples</a></span></dt><dd><dl><dt><span class="section"><a href="#bitbake-selftest-example">1.5.1. <code class="filename">bitbake-selftest</code></a></span></dt><dt><span class="section"><a href="#oe-selftest-example">1.5.2. <code class="filename">oe-selftest</code></a></span></dt><dt><span class="section"><a href="#testimage-example">1.5.3. <code class="filename">testimage</code></a></span></dt><dt><span class="section"><a href="#testsdk_ext-example">1.5.4. <code class="filename">testsdk_ext</code></a></span></dt><dt><span class="section"><a href="#testsdk-example">1.5.5. <code class="filename">testsdk</code></a></span></dt><dt><span class="section"><a href="#oe-build-perf-test-example">1.5.6. <code class="filename">oe-build-perf-test</code></a></span></dt></dl></dd><dt><span class="section"><a href="#some-id">1.6. New Section on the Periodic Builds</a></span></dt><dt><span class="section"><a href="#test-configuring-and-triggering-autobuilder-helper-build-scripts">1.7. Configuring and Triggering Autobuilder Helper Build Scripts</a></span></dt><dt><span class="section"><a href="#test-deploying-yocto-autobuilder">1.8. Deploying Yocto Autobuilder</a></span></dt><dd><dl><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-controller">1.8.1. Upstream Autobuilder Deployment on the Controller</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-on-the-worker">1.8.2. Upstream Autobuilder Deployment on the Worker</a></span></dt><dt><span class="section"><a href="#test-upstream-autobuilder-deployment-no-upstream-users">1.8.3. Upstream Autobuilder Deployment No Upstream Users</a></span></dt></dl></dd><dt><span class="section"><a href="#test-headless-sanity-tests">1.9. Setting Up Headless Sanity Tests</a></span></dt><dt><span class="section"><a href="#test-adding-additional-build-workers">1.10. Adding Additional Build Workers</a></span></dt><dt><span class="section"><a href="#test-setting-up-build-history">1.11. Setting Up Build History</a></span></dt><dt><span class="section"><a href="#test-some-more-notes">1.12. Some More Notes</a></span></dt><dt><span class="section"><a href="#test-yocto-project-helper">1.13. Yocto Project Autobuilder Helper Scripts</a></span></dt></dl></div><div class="section" title="1.1. Welcome"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-welcome">1.1. Welcome<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-welcome">¶</a></span></h2></div></div></div><p>
Welcome to the Yocto Project Test Environment Manual!
This manual is a work in progress.
The manual contains information about the testing environment
used by the Yocto Project to make sure each major and minor
release works as planned.
Other organizations can leverage off the process and testing
environment used by the Yocto Project to create their own
automated, production test environment.
</p><p>
Currently, the Yocto Project Test Environment Manual has no
projected release date.
This manual is a work-in-progress and is being initially loaded
with information from the README files and notes from key
engineers:
</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
<span class="emphasis"><em><code class="filename">yocto-autobuilder</code>:</em></span>
This
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/README" target="_top"><code class="filename">README.md</code></a>
is not maintained.
However, some information from this README file still
applies but could need some modification.
In particular, information about setting up headless
sanity tests and build history.
The sections on these will be changing.
</p><div class="note" title="IMPORTANT" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">IMPORTANT</h3>
The <code class="filename">yocto-autobuilder</code>
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/" target="_top">repository</a>
is obsolete and is no longer maintained.
The new "Autobuilder" is maintained in the
<code class="filename">yocto-autobuilder2</code>
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/" target="_top">repository</a>.
</div><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em><code class="filename">yocto-autobuilder2</code>:</em></span>
This
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md" target="_top"><code class="filename">README.md</code></a>
is the main README for Yocto Project Autobuilder.
The <code class="filename">yocto-autobuilder2</code> repository
represents the Yocto Project's testing codebase and
exists to configure and use Buildbot to do testing.
</p></li><li class="listitem"><p>
<span class="emphasis"><em><code class="filename">yocto-autobuilder-helper</code>:</em></span>
This
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/README" target="_top"><code class="filename">README</code></a>
is a valid Autobuilder Git repository that contains
Yocto Project Autobuilder Helper Scripts.
The <code class="filename">yocto-autobuilder-helper</code>
repository contains the "glue" logic that connects any
Continuous Improvement (CI) system to run builds,
support getting the correct code revisions, configure
builds and layers, run builds, and collect results.
The code is independent of any CI system, which means
the code can work Buildbot, Jenkins, or others.
</p></li></ul></div><p>
</p></div><div class="section" title="1.2. Yocto Project Autobuilder Overview"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-yocto-project-autobuilder-overview">1.2. Yocto Project Autobuilder Overview<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-yocto-project-autobuilder-overview">¶</a></span></h2></div></div></div><p>
The Yocto Project Autobuilder collectively refers to the software,
tools, scripts, and procedures used by the Yocto Project to test
released software across supported hardware in an automated and
regular fashion.
Basically, during the development of a Yocto Project release, the
Autobuilder tests if things work.
The Autobuilder builds all test targets and runs all the tests.
</p><p>
The Yocto Project uses the unpatched
<a class="ulink" href="https://docs.buildbot.net/0.9.15.post1/" target="_top">Buildbot Nine</a>
to drive its integration and testing.
Buildbot Nine has a plug-in interface that the Yocto Project
customizes using code from the
<code class="filename">yocto-autobuilder2</code> repository.
The resulting customized UI plug-in allows you to
visualize builds in a way suited to the project.
</p><p>
A "helper" layer provides configuration and job management
through scripts found in the
<code class="filename">yocto-autobuilder-helper</code> repository.
The helper layer contains the bulk of the build configuration
information and is release-specific, which makes it highly
customizable on a per-project basis.
The layer is CI system-agnostic and contains a number of helper
scripts that can generate build configurations from simple JSON
files.
</p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
It is possible to use the outer layers from another
Continuous Integration (CI) system such as
<a class="ulink" href="https://en.wikipedia.org/wiki/Jenkins_(software)" target="_top">Jenkins</a>
instead of Buildbot.
</div><p>
</p><p>
The following figure shows the Yocto Project Autobuilder stack
with a topology that includes a controller and a cluster of
workers:
</p><table border="0" summary="manufactured viewport for HTML img" cellspacing="0" cellpadding="0" width="720"><tr style="height: 540px"><td align="center"><img src="figures/ab-test-stack.png" align="middle" width="720" /></td></tr></table><p>
</p></div><div class="section" title="1.3. Yocto Project Tests - Type Overview"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-project-tests">1.3. Yocto Project Tests - Type Overview<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-project-tests">¶</a></span></h2></div></div></div><p>
Two kinds of tests exist within the Yocto Project:
</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
<span class="emphasis"><em>Build Testing:</em></span>
Tests whether specific configurations build by varying
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-MACHINE" target="_top"><code class="filename">MACHINE</code></a>,
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-DISTRO" target="_top"><code class="filename">DISTRO</code></a>,
and the specific target images being built (or world).
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Build Performance Testing:</em></span>
Tests whether or not commonly used steps during builds work
efficiently and avoid regressions.
</p></li></ul></div><p>
Beyond these types of testing, the Autobuilder tests different
pieces by using the following types of tests:
</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
<span class="emphasis"><em>Build Testing:</em></span>
Trigger builds of all the different test configurations
on the Autobuilder.
Builds usually cover each target for different
architectures, machines, and distributions.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Build Performance Testing:</em></span>
Tests to time commonly used usage scenarios are run through
<code class="filename">oe-build-perf-test</code>.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>eSDK Testing:</em></span>
Image tests initiated through the following command:
</p><pre class="literallayout">
$ bitbake <em class="replaceable"><code>image</code></em> -c testsdkext
</pre><p>
The tests utilize the <code class="filename">testsdkext</code>
class and the <code class="filename">do_testsdkext</code> task.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Feature Testing:</em></span>
Various scenario-based tests are run through the
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#testing-and-quality-assurance" target="_top">OpenEmbedded Self-Test</a>
(oe-selftest).
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Image Testing:</em></span>
Image tests initiated through the following command:
</p><pre class="literallayout">
$ bitbake <em class="replaceable"><code>image</code></em> -c testimage
</pre><p>
The tests utilize the
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-classes-testimage*" target="_top"><code class="filename">testimage*</code></a>
classes and the
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-tasks-testimage" target="_top"><code class="filename">do_testimage</code></a>
task.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Package Testing:</em></span>
A Package Test (ptest) runs tests against packages built
by the OpenEmbedded build system on the target machine.
See the
"<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/dev-manual/dev-manual.html#testing-packages-with-ptest" target="_top">Testing Packages With ptest</a>"
section in the Yocto Project Development Tasks Manual and
the
"<a class="ulink" href="https://wiki.yoctoproject.org/wiki/Ptest" target="_top">Ptest</a>"
Wiki page for more information on Ptest.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Sanity Checks During the Build Process:</em></span>
Tests initiated through the
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-classes-insane" target="_top"><code class="filename">insane</code></a>
class.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>SDK Testing:</em></span>
Image tests initiated through the following command:
</p><pre class="literallayout">
$ bitbake <em class="replaceable"><code>image</code></em> -c testsdk
</pre><p>
The tests utilize the
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-classes-testsdk" target="_top"><code class="filename">testsdk</code></a>
class and the <code class="filename">do_testsdk</code> task.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Unit Testing:</em></span>
Unit tests on various components of the system run
through <code class="filename">oe-selftest</code> and
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#testing-and-quality-assurance" target="_top"><code class="filename">bitbake-selftest</code></a>.
</p></li></ul></div><p>
</p></div><div class="section" title="1.4. How Tests Map to Areas of Code"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-test-mapping">1.4. How Tests Map to Areas of Code<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-test-mapping">¶</a></span></h2></div></div></div><p>
Tests map into the codebase as follows:
</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
<span class="emphasis"><em>bitbake-selftest:</em></span>
</p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
These tests are self-contained and test BitBake
as well as its APIs, which include the fetchers.
The tests are located in
<code class="filename">bitbake/lib/*/tests</code>.
</p></li><li class="listitem"><p>
From within the BitBake repository, run the
following:
</p><pre class="literallayout">
$ bitbake-selftest
</pre><p>
</p></li><li class="listitem"><p>
The tests are based on
<a class="ulink" href="https://docs.python.org/3/library/unittest.html" target="_top">Python unittest</a>.
</p></li></ul></div><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>oe-selftest:</em></span>
</p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
These tests use OE to test the workflows, which
include testing specific features, behaviors
of tasks, and API unit tests.
The tests take advantage of parallelism through
the "-j" option to run in multiple threads.
</p></li><li class="listitem"><p>
The tests are based on Python unittest.
</p></li><li class="listitem"><p>
The code for the tests resides in
<code class="filename">meta/lib/oeqa/selftest</code>.
</p></li><li class="listitem"><p>
To run all the test, enter the following command:
</p><pre class="literallayout">
$ oe-selftest -a
</pre><p>
</p></li><li class="listitem"><p>
To run a specific test, use the following command
form where <em class="replaceable"><code>testname</code></em> is
the name of the specific test:
</p><pre class="literallayout">
$ oe-selftest -r <em class="replaceable"><code>testname</code></em>
</pre><p>
</p></li></ul></div><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testimage:</em></span>
</p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
These tests build an image, boot it, and run tests
against the image's content.
</p></li><li class="listitem"><p>
The code for these tests resides in
<code class="filename">meta/lib/oeqa/runtime</code>.
</p></li><li class="listitem"><p>
You need to set the
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-IMAGE_CLASSES" target="_top"><code class="filename">IMAGE_CLASSES</code></a>
variable as follows:
</p><pre class="literallayout">
IMAGE_CLASSES += "testimage"
</pre><p>
</p></li><li class="listitem"><p>
Run the tests using the following command form:
</p><pre class="literallayout">
$ bitbake <em class="replaceable"><code>image</code></em> -c testimage
</pre><p>
</p></li></ul></div><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsdk:</em></span>
</p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
These tests build an SDK, install it, and then
run tests against that SDK.
</p></li><li class="listitem"><p>
The code for these tests resides in
<code class="filename">meta/lib/oeqa/sdk</code>.
</p></li><li class="listitem"><p>
Run the test using the following command form:
</p><pre class="literallayout">
$ bitbake <em class="replaceable"><code>image</code></em> -c testsdk
</pre><p>
</p></li></ul></div><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsdk_ext:</em></span>
</p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
These tests build an extended SDK (eSDK), install
that eSDK, and run tests against the eSDK.
</p></li><li class="listitem"><p>
The code for these tests resides in
<code class="filename">meta/lib/oeqa/esdk</code>.
</p></li><li class="listitem"><p>
To run the tests, use the following command form:
</p><pre class="literallayout">
$ bitbake <em class="replaceable"><code>image</code></em> -c testsdkext
</pre><p>
</p></li></ul></div><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>oe-build-perf-test:</em></span>
</p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
These tests run through commonly used usage
scenarios and measure the performance times.
</p></li><li class="listitem"><p>
The code for these tests resides in
NEED A DIRECTORY HERE.
</p></li><li class="listitem"><p>
NEED SOME INFORMATION ON HOW TO ENABLE THIS
TEST OR INCLUDE IT HERE.
</p><pre class="literallayout">
<em class="replaceable"><code>some setting</code></em>
</pre><p>
</p></li><li class="listitem"><p>
Run the tests using the following command form:
</p><pre class="literallayout">
$ <em class="replaceable"><code>some command</code></em>
</pre><p>
</p></li></ul></div><p>
</p></li></ul></div><p>
</p></div><div class="section" title="1.5. Test Examples"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-examples">1.5. Test Examples<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-examples">¶</a></span></h2></div></div></div><p>
This section provides example tests for each of the tests
listed in the
<a class="link" href="#test-test-mapping" title="1.4. How Tests Map to Areas of Code">How Tests Map to Areas of Code</a>"
section.
</p><div class="section" title="1.5.1. bitbake-selftest"><div class="titlepage"><div><div><h3 class="title" id="bitbake-selftest-example">1.5.1. <code class="filename">bitbake-selftest</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#bitbake-selftest-example">¶</a></span></h3></div></div></div><p>
Content here.
</p></div><div class="section" title="1.5.2. oe-selftest"><div class="titlepage"><div><div><h3 class="title" id="oe-selftest-example">1.5.2. <code class="filename">oe-selftest</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#oe-selftest-example">¶</a></span></h3></div></div></div><p>
NEED CONTENT HERE.
</p></div><div class="section" title="1.5.3. testimage"><div class="titlepage"><div><div><h3 class="title" id="testimage-example">1.5.3. <code class="filename">testimage</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#testimage-example">¶</a></span></h3></div></div></div><p>
NEED CONTENT HERE.
</p></div><div class="section" title="1.5.4. testsdk_ext"><div class="titlepage"><div><div><h3 class="title" id="testsdk_ext-example">1.5.4. <code class="filename">testsdk_ext</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#testsdk_ext-example">¶</a></span></h3></div></div></div><p>
NEED CONTENT HERE.
</p></div><div class="section" title="1.5.5. testsdk"><div class="titlepage"><div><div><h3 class="title" id="testsdk-example">1.5.5. <code class="filename">testsdk</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#testsdk-example">¶</a></span></h3></div></div></div><p>
NEED CONTENT HERE.
</p></div><div class="section" title="1.5.6. oe-build-perf-test"><div class="titlepage"><div><div><h3 class="title" id="oe-build-perf-test-example">1.5.6. <code class="filename">oe-build-perf-test</code><span class="permalink"><a alt="Permalink" title="Permalink" href="#oe-build-perf-test-example">¶</a></span></h3></div></div></div><p>
NEED CONTENT HERE.
</p></div></div><div class="section" title="1.6. New Section on the Periodic Builds"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="some-id">1.6. New Section on the Periodic Builds<span class="permalink"><a alt="Permalink" title="Permalink" href="#some-id">¶</a></span></h2></div></div></div><p>
The following is going to be the replacement content for the
section on "Nightly Builds".
Not sure what we are going to call these builds.
We need a name to replace "Nightly Builds".
</p><p>
Here is the content from Richards email:
</p><pre class="literallayout">
In 1.6, we actually dropped the "nightly" bit pretty much everywhere.
They are now named MACHINE or MACHINE-DISTRO, e.g. qemuarm or qemuarm-
lsb (which tests poky-lsb with qemuarm). We now parallelise not just
architecture but by machine so machine and real hardware are now
separate. The flow is therefore to build the images+sdks, then test the
images+sdks, trying to do as much as possible in parallel.
We have two types of build trigger, "quick" and "full". quick runs all
the things which commonly fail and one random oe-selftest. "full" runs
all our targets, runs oe-selftest on all distros and includes ptest and
build performance tests. Its slower but more complete and would be used
for release builds.
</pre><p>
</p></div><div class="section" title="1.7. Configuring and Triggering Autobuilder Helper Build Scripts"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-configuring-and-triggering-autobuilder-helper-build-scripts">1.7. Configuring and Triggering Autobuilder Helper Build Scripts<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-configuring-and-triggering-autobuilder-helper-build-scripts">¶</a></span></h2></div></div></div><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
This section is created from the information in the
<code class="filename">yocto-autobuilder2</code>
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md" target="_top"><code class="filename">README.md</code></a>
file.
I am making an assumption that we do not want to refer to the
Autobuilder stuff as "Autobuilder2".
My guess is that since this is the first documentation of any
automated test environment and process in the Yocto Project
user documentation, we will treat it as the start of things.
</div><p>
Automatic testing is based on the workers executing builds using
Buildbot Nine configured for specific build jobs triggered in an
automatic and regular fashion.
Worker Configuration and triggering is accomplished through
the
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/" target="_top">Yocto Project Autobuilder layer</a>
and a set of
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree" target="_top">helper scripts</a>.
</p><p>
The configuration and helper scripts have as little code and
as few custom Buildbot extensions as possible.
The configuration collects required input from the user to
furnish the helper scripts with the input needed for workers
to accomplish their builds.
The input consists of minimal user-customizable parameters
used to trigger the helper build scripts.
</p><p>
Each builder maps to a named configuration in the helper
scripts.
The configuration is created with the steps and properties
required to invoke the helper scripts for a worker's builds.
</p><p>
Each worker has a custom scheduler created for it and contains
parameters configured for the scheduler that can supply the custom
versions of the required values for the helper script parameters.
</p><p>
Following is the code layout for the Autobuilder:
</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/builders.py" target="_top"><code class="filename">builders.py</code></a>:</em></span>
Configures the builders with the minimal buildsteps
to invoke the Yocto Project Autobuilder helper scripts.
</p></li><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/lib/wiki.py" target="_top"><code class="filename">lib/wiki.py</code></a>:</em></span>
Implements functionality related to
<a class="ulink" href="https://www.mediawiki.org/wiki/MediaWiki" target="_top">MediaWiki</a>.
The <code class="filename">wikilog</code> plug-in uses this
functionality.
Effectively, this functionality provides helper functions
for the plug-in.
</p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
Much of this code can be replaced by porting the
plug-in so that it is implemented as a
<code class="filename">buildbot.util.service.HTTPClient</code>.
</div><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/reporters/wikilog.py" target="_top"><code class="filename">reporters/wikilog.py</code></a>:</em></span>
A custom plug-in that is a Buildbot service that listens for
build failures and then writes information about the
failure to the configured wiki page.
</p></li><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/steps/writelayerinfo.py" target="_top"><code class="filename">steps/writelayerinfo.py</code></a>:</em></span>
Implements a simple, custom buildset that iterates the
<code class="filename">repo_</code>, <code class="filename">branch_</code>,
and <code class="filename">commit_</code> properties, which are set
by the schedulers, and then writes a JSON file with the
user's values.
</p></li><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/config.py" target="_top"><code class="filename">config.py</code></a>:</em></span>
Contains all values that might need changing to redeploy
the Autobuilder code elsewhere.
</p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
The redeployment goal has not been currently met.
</div><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/master.cfg" target="_top"><code class="filename">master.cfg</code></a>:</em></span>
Performs most configuration by making calls into other
scripts.
Configuration specific for a worker cluster (i.e. a
Controller URL) resides here.
</p></li><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/schedulers.py" target="_top"><code class="filename">schedulers.py</code></a>:</em></span>
Sets up the force schedulers with controls for modifying
inputs for each worker.
</p></li><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/services.py" target="_top"><code class="filename">services.py</code></a>:</em></span>
Configures IRC, mail, and Wikilog reporters.
</p></li><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/workers.py" target="_top"><code class="filename">workers.py</code></a>:</em></span>
Configures the worker objects.
</p></li><li class="listitem"><p>
<span class="emphasis"><em><a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/www.py" target="_top"><code class="filename">www.py</code></a>:</em></span>
Sets up the Web User Interface.
</p></li></ul></div><p>
</p><p>
The goal is to keep custom code minimized throughout the
Autobuilder.
The few customizations implemented support the Yocto Project
Autobuilder Helper Script workflows and help replicate the
workflows established with the Yocto Autobuilder layer.
In particular, the following files accomplish this customization:
</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
<code class="filename">writelayerinfo.py</code>
</p></li><li class="listitem"><p>
<code class="filename">wikilog.py</code>
</p></li><li class="listitem"><p>
<code class="filename">wiki.py</code>
</p></li></ul></div><p>
</p></div><div class="section" title="1.8. Deploying Yocto Autobuilder"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-deploying-yocto-autobuilder">1.8. Deploying Yocto Autobuilder<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-deploying-yocto-autobuilder">¶</a></span></h2></div></div></div><p>
Steps to deploy the Yocto Project Autobuilder assume each target
system has a copy of Buildbot installed.
Additionally, various pieces of functionality require that a copy
of the Autobuilder Helper Scripts (i.e.
<code class="filename">yocto-autobuilder-helper</code>) is available
in the home directory at
<code class="filename">~/yocto-autobuilder-helper</code> of the user
running Buildbot.
</p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
If you are using a reverse proxy, be aware that modern
Buildbot uses a web socket for various communications between
the master and the web's User Interface.
Refer to the
<a class="ulink" href="http://docs.buildbot.net/latest/manual/cfg-www.html#reverse-proxy-configuration" target="_top">Buildbot documentation</a>
for information on how to correctly configure a reverse proxy.
</div><p>
</p><p>
The following sections provide steps for Yocto Autobuilder
deployment.
</p><div class="section" title="1.8.1. Upstream Autobuilder Deployment on the Controller"><div class="titlepage"><div><div><h3 class="title" id="test-upstream-autobuilder-deployment-on-the-controller">1.8.1. Upstream Autobuilder Deployment on the Controller<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-upstream-autobuilder-deployment-on-the-controller">¶</a></span></h3></div></div></div><p>
Follow these steps to deploy Yocto Autobuilder on an
upstream controller:
</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
<span class="emphasis"><em>Create the Master Yocto Controller</em></span>:
</p><pre class="literallayout">
$ buildbot create-master <em class="replaceable"><code>yocto-controller</code></em>
</pre><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Change Your Working Directory to the Master Yocto Controller</em></span>:
</p><pre class="literallayout">
$ cd <em class="replaceable"><code>yocto-controller</code></em>
</pre><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Create a Local Git Repository of the Yocto Project Autobuilder</em></span>:
</p><pre class="literallayout">
$ git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb
</pre><p>
In the previous command, the local repository is
created in a <code class="filename">yoctoabb</code>
directory inside the directory of the Master
Yocto Controller directory.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Change Your Working Directory Back to the Master Yocto Controller</em></span>:
</p><pre class="literallayout">
$ cd ..
</pre><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Create a Relative Symbolic Link to <code class="filename">master.cfg</code></em></span>:
</p><pre class="literallayout">
$ ln -rs <em class="replaceable"><code>yocto-controller</code></em>/yoctoabb/master.cfg <em class="replaceable"><code>yocto-controller</code></em>/master.cfg
</pre><p>
The previous command sets up a relative symbolic
link to the <code class="filename">master.cfg</code> using
a link of the same name.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Update the Buildbot URL in <code class="filename">master.cfg</code></em></span>:
Use your <code class="filename">$EDITOR</code> to edit the
Buildbot URL in the <code class="filename">master.cfg</code>
file.
Find the following line and replace the URL with
the URL for your Buildbot:
</p><pre class="literallayout">
c['buildbotURL'] = "https://autobuilder.yoctoproject.org/main/"
</pre><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Enable services in <code class="filename">services.py</code></em></span>:
Use your <code class="filename">$EDITOR</code> to edit the
<code class="filename">services.py</code> file.
Set appropriate configuration values to enable
desired services.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Enable Automatic Authorization (Autorisation) in <code class="filename">www.py</code></em></span>:
Use your <code class="filename">$EDITOR</code> to edit the
<code class="filename">www.py</code> file.
Configure autorisation if desired.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Modify Configuration Options in <code class="filename">config.py</code></em></span>:
Use your <code class="filename">$EDITOR</code> to edit the
<code class="filename">config.py</code> file.
Modify configuration options such as worker
configurations.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Start Buildbot</em></span>:
</p><pre class="literallayout">
$ buildbot start <em class="replaceable"><code>yocto-controller</code></em>
</pre><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Create a Local Git Repository of the Yocto Autobuilder Helper Scripts:</em></span>:
</p><pre class="literallayout">
Move up a directory so that you are above the
<em class="replaceable"><code>yocto-controller</code></em>
location and clone the directory:
$ cd ..
$ git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper
</pre><p>
</p></li></ol></div><p>
</p></div><div class="section" title="1.8.2. Upstream Autobuilder Deployment on the Worker"><div class="titlepage"><div><div><h3 class="title" id="test-upstream-autobuilder-deployment-on-the-worker">1.8.2. Upstream Autobuilder Deployment on the Worker<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-upstream-autobuilder-deployment-on-the-worker">¶</a></span></h3></div></div></div><p>
Follow these steps to deploy Yocto Autobuilder on an
upstream worker:
</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
<span class="emphasis"><em>Create the Worker</em></span>:
</p><pre class="literallayout">
$ buildbot-worker create-worker <em class="replaceable"><code>yocto-worker</code></em> <em class="replaceable"><code>localhost</code></em> <em class="replaceable"><code>example-worker</code></em> <em class="replaceable"><code>pass</code></em>
</pre><p>
</p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
You do not have to hard-code the third
parameter (i.e.
<em class="replaceable"><code>example-worker</code></em>).
For example, you can pass
<code class="filename">`hostname`</code> to use the
host's configured name.
</div><p>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Start the Worker</em></span>:
</p><pre class="literallayout">
$ buildbot-worker start <em class="replaceable"><code>yocto-worker</code></em>
</pre><p>
</p></li></ol></div><p>
</p></div><div class="section" title="1.8.3. Upstream Autobuilder Deployment No Upstream Users"><div class="titlepage"><div><div><h3 class="title" id="test-upstream-autobuilder-deployment-no-upstream-users">1.8.3. Upstream Autobuilder Deployment No Upstream Users<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-upstream-autobuilder-deployment-no-upstream-users">¶</a></span></h3></div></div></div><p>
This case has yet to be defined.
It requires a custom <code class="filename">config.json</code> file
for <code class="filename">yocto-autobuilder-helper</code>.
</p></div></div><div class="section" title="1.9. Setting Up Headless Sanity Tests"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-headless-sanity-tests">1.9. Setting Up Headless Sanity Tests<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-headless-sanity-tests">¶</a></span></h2></div></div></div><p>
If you plan on using the Yocto Project Autobuilder to run
headless sanity testing, you need to do the following:
</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
Install
<a class="link" href="#test-tight-vnc">TightVNC</a>
client and server.
</p></li><li class="listitem"><p>
Create a bank of tap network devices (tap devs)
by running the
<code class="filename">runqemu-gen-tapdevs</code> script
found in the
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#source-directory" target="_top">Source Directory</a>
at
<a class="ulink" href="https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts" target="_top">https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts</a>.
</p><p>You must disable interface control on these
new tap devices.
</p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
Some services include NetworkManager,
connman, or wicd.
</div><p>
</p></li><li class="listitem"><p>
Add "xterm*vt100*geometry: 80x50+10+10" to
<code class="filename">.Xdefaults</code>
</p></li><li class="listitem"><p>
Set up and start the TightVNC session as the
Autobuilder user.
</p></li><li class="listitem"><p>
Manually connect to the VNC session at least once
prior to running a QEMU sanity test.
</p><div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3>
Something is getting set during the initial
connection that has not been figured out yet.
Manually connecting seems to set up the session
correctly.
</div><p>
</p></li></ol></div><p>
</p></div><div class="section" title="1.10. Adding Additional Build Workers"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-adding-additional-build-workers">1.10. Adding Additional Build Workers<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-adding-additional-build-workers">¶</a></span></h2></div></div></div><p>
The production Yocto Autobuilder uses a cluster of build
workers.
The cluster shares the same
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-SSTATE_DIR" target="_top"><code class="filename">SSTATE_DIR</code></a>
and
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-DL_DIR" target="_top"><code class="filename">DL_DIR</code></a>
through an NFS4 mounted Network Attached Storage (NAS).
The main nightly trigger pre-populates the
<code class="filename">DL_DIR</code>, which allows the workers to not
have to deal with a lot of downloading.
In theory, you could also run your build workers with
<a class="ulink" href="http://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-NO_NETWORK" target="_top"><code class="filename">NO_NETWORK</code></a>
to enforce a single point for populating
<code class="filename">DL_DIR</code>.
</p><p>
Running multiple build workers is fairly simple, but does require
some setup:
</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
Ensure the settings in
<code class="filename">autobuilder.conf</code> are valid
for each worker.
Certain variables are set within this file that
work with the local configurations on each
worker.
</p></li><li class="listitem"><p>
Within
<code class="filename">yocto-controller/controller.cfg</code>,
add your worker to the
<code class="filename">c['workers']</code> list inside
the <code class="filename">BUILDWORKERS</code> section.
</p></li><li class="listitem"><p>
For each worker change the
<code class="filename">WORKER SETTINGS</code> section
of
<code class="filename">yocto-worker/buildbot.tac</code>
to match the settings in
<code class="filename">controller.cfg</code>.
</p></li><li class="listitem"><p>
Workers must reside in the same path as the
Build Controller, even if they are on
completely different machines.
</p></li></ol></div><p>
</p></div><div class="section" title="1.11. Setting Up Build History"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-setting-up-build-history">1.11. Setting Up Build History<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-setting-up-build-history">¶</a></span></h2></div></div></div><p>
Build History is used to track changes to packages and
images.
By default, the Autobuilder does not collect build history.
The production Autobuilder does have this functionality
enabled.
</p><p>
Setting up build history requires the following
steps:
</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>
Create an empty Git repository.
Make a single commit to it and then create and
push branches for each of the nightly core
architectures (i.e.. mips, ppc, x86...).
</p></li><li class="listitem"><p>
Find a central location to create a clone for the
repository created in the previous step.
This works best if you have a setup similar to
the production Autobuilder (i.e. NAS with many
workers).
</p></li><li class="listitem"><p>
Run the following:
</p><pre class="literallayout">
# This is an example of how to set up a local build history checkout. Paths
# obviously are situationally dependent.
$ mkdir /nas/buildhistory
$ cd /nas/buildhistory
$ git clone ssh://git@git.myproject.org/buildhistory
$ git clone ssh://git@git.myproject.org/buildhistory nightly-arm
$ git clone ssh://git@git.myproject.org/buildhistory nightly-x86
$ git clone ssh://git@git.myproject.org/buildhistory nightly-x86-64
$ git clone ssh://git@git.myproject.org/buildhistory nightly-ppc
$ git clone ssh://git@git.myproject.org/buildhistory nightly-mips
$ for x in `ls|grep nightly` do cd $x; git checkout $x; cd /nas/buildhistory; done
</pre><p>
</p></li><li class="listitem"><p>
Within the <code class="filename">autobuilder.conf</code>
of each worker, change the following:
</p><pre class="literallayout">
BUILD_HISTORY_COLLECT = True
BUILD_HISTORY_DIR = "/nas/buildhistory"
BUILD_HISTORY_REPO = "ssh://git@git.myproject.org/buildhistory"
</pre><p>
</p></li></ol></div><p>
</p></div><div class="section" title="1.12. Some More Notes"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-some-more-notes">1.12. Some More Notes<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-some-more-notes">¶</a></span></h2></div></div></div><p>
</p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><p>
<span class="emphasis"><em>Yocto Autobuilder</em></span>:
The Git repository is at
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/" target="_top">http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/</a>.
</p><p>Essentially an extension to the vanilla buildbot.
This extension mainly addresses configuration file handling
and Yocto-specific build steps.</p><p>For better maintainability, the Autobuilder (see
<code class="filename">Autobuilder.py</code> located at
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/lib/python2.7/site-packages/autobuilder" target="_top">http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/lib/python2.7/site-packages/autobuilder</a>),
handles configuration from multiple files.</p><p>Additional build steps such as
<code class="filename">CheckOutLayers.py</code> or
<code class="filename">CreateBBLayersConf</code> are Yocto-specific
and simplify the worker's configuration.
</p></li><li class="listitem"><p><a id="test-tight-vnc"></a>
<span class="emphasis"><em>TightVNC</em></span>:
Virtual Network Computing (VNC) is a client/server software
package that allows remote network access to graphical
desktops.
With VNC, you can access your machine from everywhere
provided that your machine is connected to the Internet.
VNC is free (released under the GNU General Public License)
and it is available on most platforms.</p><p>TightVNC is an enhanced version of VNC, which
includes new features, improvements, optimizations, and
bug fixes over the original VNC version.
See the list of features at
<a class="ulink" href="http://www.tightvnc.com/intro.php" target="_top">http://www.tightvnc.com/intro.php</a>.
</p><p>You need TightVNC in order to run headless sanity
tests.
See the bullet on
<a class="link" href="#test-headless-sanity-tests" title="1.9. Setting Up Headless Sanity Tests">headless sanity tests</a>
for more information.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>Files Used for Yocto-Autobuilder Configuration:</em></span>
</p><div class="itemizedlist"><ul class="itemizedlist" type="circle"><li class="listitem"><p>
<span class="emphasis"><em><code class="filename">config/autobuilder.conf</code></em></span>:
Used to set Autobuilder-wide parameters, such as
where various build artifacts are published
(e.g. <code class="filename">DL_DIR</code> and
<code class="filename">SSTATE_DIR</code>).
Another example is if build artifacts should be
published, which is necessary for production
Autobuilders but not desktop builders.
</p></li><li class="listitem"><p>
<span class="emphasis"><em><code class="filename">buildset-config/yoctoAB.conf</code></em></span>:
The main Yocto Project Autobuilder configuration
file.
Documentation for this file and its associated
format is in the
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/tree/README-NEW-AUTOBUILDER" target="_top"><code class="filename">README-NEW-AUTOBUILDER</code></a>
file.
</p></li></ul></div><p>
</p></li></ul></div><p>
</p></div><div class="section" title="1.13. Yocto Project Autobuilder Helper Scripts"><div class="titlepage"><div><div><h2 class="title" style="clear: both" id="test-yocto-project-helper">1.13. Yocto Project Autobuilder Helper Scripts<span class="permalink"><a alt="Permalink" title="Permalink" href="#test-yocto-project-helper">¶</a></span></h2></div></div></div><div class="note" title="WRITER NOTE" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">WRITER NOTE</h3>
Deferring this topic per Richard's suggestion.
It is placed here temporarily.
</div><p>
The helper scripts work in conjunction with the Yocto Project
Autobuilder.
These scripts do the actual build configuration and execution
for tests on a per release basis.
</p><p>
You can use <code class="filename">pre-commit-hook.sh</code> to verify
the JSON file before committing it.
Create a symbolic link as follows:
</p><pre class="literallayout">
$ ln -s ../../scripts/pre-commit-hook.sh .git/hooks/pre-commit
</pre><p>
</p><p>
Most users will have to customize the helper script repository
to meet their needs.
The repository is located at
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper" target="_top">http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper</a>.
The scripts themselves should be more generically reusable.
The <code class="filename">config.json</code> is less reusable as it
represents the Yocto Project Autobuilder test matrix.
</p><p>
Two customization options are possible: 1) variable substitution,
and 2) overlaying configuration files.
The standard <code class="filename">config.json</code> minimally attempts
to allow substitution of the paths.
The helper script repository includes a
<a class="ulink" href="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/local-example.json" target="_top"><code class="filename">local-example.json</code></a>
to show how you could override these from a separate configuration
file.
Pass the following into the environment of the autobuilder:
</p><pre class="literallayout">
ABHELPER_JSON="config.json local-example.json"
</pre><p>
As another example, you could also pass the following into the
environment:
</p><pre class="literallayout">
ABHELPER_JSON="config.json /some/location/local.json"
</pre><p>
</p></div></div>
</div></body></html>
|
