1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
|
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
<chapter id='usingpoky'>
<title>Using the Yocto Project</title>
<para>
This chapter describes common usage for the Yocto Project.
The information is introductory in nature as other manuals in the Yocto Project
documentation set provide more details on how to use the Yocto Project.
</para>
<section id='usingpoky-build'>
<title>Running a Build</title>
<para>
This section provides a summary of the build process and provides information
for less obvious aspects of the build process.
For general information on how to build an image using the OpenEmbedded build
system, see the
"<ulink url='&YOCTO_DOCS_QS_URL;#qs-building-images'>Building Images</ulink>"
section of the Yocto Project Quick Start.
</para>
<section id='build-overview'>
<title>Build Overview</title>
<para>
In the development environment you will need to build an image whenever you change hardware
support, add or change system libraries, or add or change services that have dependencies.
</para>
<mediaobject>
<imageobject>
<imagedata fileref="figures/building-an-image.png" format="PNG" align='center' scalefit='1'/>
</imageobject>
<caption>
<para>Building an Image</para>
</caption>
</mediaobject>
<para>
The first thing you need to do is set up the OpenEmbedded build
environment by sourcing an environment setup script
(i.e.
<link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>
or
<link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>).
Here is an example:
<literallayout class='monospaced'>
$ source &OE_INIT_FILE; [<replaceable>build_dir</replaceable>]
</literallayout>
</para>
<para>
The <replaceable>build_dir</replaceable> argument is optional and specifies the directory the
OpenEmbedded build system uses for the build -
the
<link linkend='build-directory'>Build Directory</link>.
If you do not specify a Build Directory, it defaults to a directory
named <filename>build</filename> in your current working directory.
A common practice is to use a different Build Directory for different targets.
For example, <filename>~/build/x86</filename> for a <filename>qemux86</filename>
target, and <filename>~/build/arm</filename> for a <filename>qemuarm</filename> target.
</para>
<para>
Once the build environment is set up, you can build a target using:
<literallayout class='monospaced'>
$ bitbake <replaceable>target</replaceable>
</literallayout>
<note>
<para>
If you experience a build error due to resources
temporarily being unavailable and it appears you
should not be having this issue, it might be due
to the combination of a 4.3+ Linux kernel and
<filename>systemd</filename> version 228+
(i.e. see this
<ulink url='http://unix.stackexchange.com/questions/253903/creating-threads-fails-with-resource-temporarily-unavailable-with-4-3-kernel'>link</ulink>
for information).
</para>
<para>
To work around this issue, you can try either
of the following:
<itemizedlist>
<listitem><para>
Try the build again.
</para></listitem>
<listitem><para>
Modify the "DefaultTasksMax"
<filename>systemd</filename> parameter
by uncommenting it and setting it to
"infinity".
You can find this parameter in the
<filename>system.conf</filename> file
located in
<filename>/etc/systemd</filename>
on most systems.
</para></listitem>
</itemizedlist>
</para>
</note>
</para>
<para>
The <replaceable>target</replaceable> is the name of the recipe you want to build.
Common targets are the images in <filename>meta/recipes-core/images</filename>,
<filename>meta/recipes-sato/images</filename>, etc. all found in the
<ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
Or, the target can be the name of a recipe for a specific piece of software such as
BusyBox.
For more details about the images the OpenEmbedded build system supports, see the
"<link linkend="ref-images">Images</link>" chapter.
</para>
<note>
Building an image without GNU General Public License Version
3 (GPLv3), or similarly licensed, components is supported for
only minimal and base images.
See the "<link linkend='ref-images'>Images</link>" chapter for more information.
</note>
</section>
<section id='building-an-image-using-gpl-components'>
<title>Building an Image Using GPL Components</title>
<para>
When building an image using GPL components, you need to maintain your original
settings and not switch back and forth applying different versions of the GNU
General Public License.
If you rebuild using different versions of GPL, dependency errors might occur
due to some components not being rebuilt.
</para>
</section>
</section>
<section id='usingpoky-install'>
<title>Installing and Using the Result</title>
<para>
Once an image has been built, it often needs to be installed.
The images and kernels built by the OpenEmbedded build system are placed in the
<link linkend='build-directory'>Build Directory</link> in
<filename class="directory">tmp/deploy/images</filename>.
For information on how to run pre-built images such as <filename>qemux86</filename>
and <filename>qemuarm</filename>, see the
<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-manual'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>.
For information about how to install these images, see the documentation for your
particular board or machine.
</para>
</section>
<section id='usingpoky-debugging-tools-and-techniques'>
<title>Debugging Tools and Techniques</title>
<para>
The exact method for debugging build failures depends on the nature of
the problem and on the system's area from which the bug originates.
Standard debugging practices such as comparison against the last
known working version with examination of the changes and the
re-application of steps to identify the one causing the problem are
valid for the Yocto Project just as they are for any other system.
Even though it is impossible to detail every possible potential failure,
this section provides some general tips to aid in debugging.
</para>
<para>
A useful feature for debugging is the error reporting tool.
Configuring the Yocto Project to use this tool causes the
OpenEmbedded build system to produce error reporting commands as
part of the console output.
You can enter the commands after the build completes
to log error information
into a common database, that can help you figure out what might be
going wrong.
For information on how to enable and use this feature, see the
"<ulink url='&YOCTO_DOCS_DEV_URL;#using-the-error-reporting-tool'>Using the Error Reporting Tool</ulink>"
section in the Yocto Project Development Manual.
</para>
<para>
For discussions on debugging, see the
"<ulink url='&YOCTO_DOCS_DEV_URL;#platdev-gdb-remotedebug'>Debugging With the GNU Project Debugger (GDB) Remotely</ulink>" section
in the Yocto Project Developer's Manual
and the
"<ulink url='&YOCTO_DOCS_SDK_URL;#adt-eclipse'>Working within Eclipse</ulink>"
section in the Yocto Project Software Development Kit (SDK) Developer's Guide.
</para>
<note>
The remainder of this section presents many examples of the
<filename>bitbake</filename> command.
You can learn about BitBake by reading the
<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>.
</note>
<section id='usingpoky-debugging-viewing-logs-from-failed-tasks'>
<title>Viewing Logs from Failed Tasks</title>
<para>
You can find the log for a task in the file
<filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}/temp/log.do_</filename><replaceable>taskname</replaceable>.
For example, the log for the
<link linkend='ref-tasks-compile'><filename>do_compile</filename></link>
task of the QEMU minimal image for the x86 machine
(<filename>qemux86</filename>) might be in
<filename>tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile</filename>.
To see the commands
<link linkend='bitbake-term'>BitBake</link> ran
to generate a log, look at the corresponding
<filename>run.do_</filename><replaceable>taskname</replaceable>
file in the same directory.
</para>
<para>
<filename>log.do_</filename><replaceable>taskname</replaceable> and
<filename>run.do_</filename><replaceable>taskname</replaceable>
are actually symbolic links to
<filename>log.do_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable>
and
<filename>log.run_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable>,
where <replaceable>pid</replaceable> is the PID the task had when
it ran.
The symlinks always point to the files corresponding to the most
recent run.
</para>
</section>
<section id='usingpoky-debugging-viewing-variable-values'>
<title>Viewing Variable Values</title>
<para>
BitBake's <filename>-e</filename> option is used to display
variable values after parsing.
The following command displays the variable values after the
configuration files (i.e. <filename>local.conf</filename>,
<filename>bblayers.conf</filename>,
<filename>bitbake.conf</filename> and so forth) have been
parsed:
<literallayout class='monospaced'>
$ bitbake -e
</literallayout>
The following command displays variable values after a specific
recipe has been parsed.
The variables include those from the configuration as well:
<literallayout class='monospaced'>
$ bitbake -e recipename
</literallayout>
<note><para>
Each recipe has its own private set of variables (datastore).
Internally, after parsing the configuration, a copy of the
resulting datastore is made prior to parsing each recipe.
This copying implies that variables set in one recipe will
not be visible to other recipes.</para>
<para>Likewise, each task within a recipe gets a private
datastore based on the recipe datastore, which means that
variables set within one task will not be visible to
other tasks.</para>
</note>
</para>
<para>
In the output of <filename>bitbake -e</filename>, each variable is
preceded by a description of how the variable got its value,
including temporary values that were later overriden.
This description also includes variable flags (varflags) set on
the variable.
The output can be very helpful during debugging.
</para>
<para>
Variables that are exported to the environment are preceded by
<filename>export</filename> in the output of
<filename>bitbake -e</filename>.
See the following example:
<literallayout class='monospaced'>
export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86"
</literallayout>
</para>
<para>
In addition to variable values, the output of the
<filename>bitbake -e</filename> and
<filename>bitbake -e</filename> <replaceable>recipe</replaceable>
commands includes the following information:
<itemizedlist>
<listitem><para>
The output starts with a tree listing all configuration
files and classes included globally, recursively listing
the files they include or inherit in turn.
Much of the behavior of the OpenEmbedded build system
(including the behavior of the
<link linkend='normal-recipe-build-tasks'>normal recipe build tasks</link>)
is implemented in the
<link linkend='ref-classes-base'><filename>base</filename></link>
class and the classes it inherits, rather than being built
into BitBake itself.
</para></listitem>
<listitem><para>
After the variable values, all functions appear in the
output.
For shell functions, variables referenced within the
function body are expanded.
If a function has been modified using overrides or
using override-style operators like
<filename>_append</filename> and
<filename>_prepend</filename>, then the final assembled
function body appears in the output.
</para></listitem>
</itemizedlist>
</para>
</section>
<section id='viewing-package-information-with-oe-pkgdata-util'>
<title>Viewing Package Information with <filename>oe-pkgdata-util</filename></title>
<para>
You can use the <filename>oe-pkgdata-util</filename> command-line
utility to query
<link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link>
and display various package-related information.
When you use the utility, you must use it to view information
on packages that have already been built.
</para>
<para>
Following are a few of the available
<filename>oe-pkgdata-util</filename> subcommands.
<note>
You can use the standard * and ? globbing wildcards as part of
package names and paths.
</note>
<itemizedlist>
<listitem><para>
<filename>oe-pkgdata-util list-pkgs [</filename><replaceable>pattern</replaceable><filename>]</filename>:
Lists all packages that have been built, optionally
limiting the match to packages that match
<replaceable>pattern</replaceable>.
</para></listitem>
<listitem><para>
<filename>oe-pkgdata-util list-pkg-files </filename><replaceable>package</replaceable><filename> ...</filename>:
Lists the files and directories contained in the given
packages.
<note>
<para>
A different way to view the contents of a package is
to look at the
<filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}/packages-split</filename>
directory of the recipe that generates the
package.
This directory is created by the
<link linkend='ref-tasks-package'><filename>do_package</filename></link>
task and has one subdirectory for each package the
recipe generates, which contains the files stored in
that package.</para>
<para>
If you want to inspect the
<filename>${WORKDIR}/packages-split</filename>
directory, make sure that
<link linkend='ref-classes-rm-work'><filename>rm_work</filename></link>
is not enabled when you build the recipe.
</para>
</note>
</para></listitem>
<listitem><para>
<filename>oe-pkgdata-util find-path </filename><replaceable>path</replaceable><filename> ...</filename>:
Lists the names of the packages that contain the given
paths.
For example, the following tells us that
<filename>/usr/share/man/man1/make.1</filename>
is contained in the <filename>make-doc</filename>
package:
<literallayout class='monospaced'>
$ oe-pkgdata-util find-path /usr/share/man/man1/make.1
make-doc: /usr/share/man/man1/make.1
</literallayout>
</para></listitem>
<listitem><para>
<filename>oe-pkgdata-util lookup-recipe </filename><replaceable>package</replaceable><filename> ...</filename>:
Lists the name of the recipes that
produce the given packages.
</para></listitem>
</itemizedlist>
</para>
<para>
For more information on the <filename>oe-pkgdata-util</filename>
command, use the help facility:
<literallayout class='monospaced'>
$ oe-pkgdata-util ‐‐help
$ oe-pkgdata-util <replaceable>subcommand</replaceable> --help
</literallayout>
</para>
</section>
<section id='usingpoky-viewing-dependencies-between-recipes-and-tasks'>
<title>Viewing Dependencies Between Recipes and Tasks</title>
<para>
Sometimes it can be hard to see why BitBake wants to build other
recipes before the one you have specified.
Dependency information can help you understand why a recipe is
built.
</para>
<para>
To generate dependency information for a recipe, run the following
command:
<literallayout class='monospaced'>
$ bitbake -g <replaceable>recipename</replaceable>
</literallayout>
This command writes the following files in the current directory:
<itemizedlist>
<listitem><para>
<filename>pn-buildlist</filename>: A list of
recipes/targets involved in building
<replaceable>recipename</replaceable>.
"Involved" here means that at least one task from the
recipe needs to run when building
<replaceable>recipename</replaceable> from scratch.
Targets that are in
<link linkend='var-ASSUME_PROVIDED'><filename>ASSUME_PROVIDED</filename></link>
are not listed.
</para></listitem>
<listitem><para>
<filename>task-depends.dot</filename>: A graph showing
dependencies between tasks.
</para></listitem>
</itemizedlist>
</para>
<para>
The graphs are in
<ulink url='https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29'>DOT</ulink>
format and can be converted to images (e.g. using the
<filename>dot</filename> tool from
<ulink url='http://www.graphviz.org/'>Graphviz</ulink>).
<note><title>Notes</title>
<itemizedlist>
<listitem><para>
DOT files use a plain text format.
The graphs generated using the
<filename>bitbake -g</filename> command are often so
large as to be difficult to read without special
pruning (e.g. with Bitbake's
<filename>-I</filename> option) and processing.
Despite the form and size of the graphs, the
corresponding <filename>.dot</filename> files can still
be possible to read and provide useful information.
</para>
<para>As an example, the
<filename>task-depends.dot</filename> file contains
lines such as the following:
<literallayout class='monospaced'>
"libxslt.do_configure" -> "libxml2.do_populate_sysroot"
</literallayout>
The above example line reveals that the
<link linkend='ref-tasks-configure'><filename>do_configure</filename></link>
task in <filename>libxslt</filename> depends on the
<link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link>
task in <filename>libxml2</filename>, which is a normal
<link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
dependency between the two recipes.
</para></listitem>
<listitem><para>
For an example of how <filename>.dot</filename> files
can be processed, see the
<filename>scripts/contrib/graph-tool</filename> Python
script, which finds and displays paths between graph
nodes.
</para></listitem>
</itemizedlist>
</note>
</para>
<para>
You can use a different method to view dependency information
by using the following command:
<literallayout class='monospaced'>
$ bitbake -g -u taskexp <replaceable>recipename</replaceable>
</literallayout>
This command displays a GUI window from which you can view
build-time and runtime dependencies for the recipes involved in
building <replaceable>recipename</replaceable>.
</para>
</section>
<section id='usingpoky-viewing-task-variable-dependencies'>
<title>Viewing Task Variable Dependencies</title>
<para>
As mentioned in the
"<ulink url='&YOCTO_DOCS_BB_URL;#checksums'>Checksums (Signatures)</ulink>"
section of the BitBake User Manual, BitBake tries to automatically
determine what variables a task depends on so that it can rerun
the task if any values of the variables change.
This determination is usually reliable.
However, if you do things like construct variable names at runtime,
then you might have to manually declare dependencies on those
variables using <filename>vardeps</filename> as described in the
"<ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'>Variable Flags</ulink>"
section of the BitBake User Manual.
</para>
<para>
If you are unsure whether a variable dependency is being picked up
automatically for a given task, you can list the variable
dependencies BitBake has determined by doing the following:
<orderedlist>
<listitem><para>
Build the recipe containing the task:
<literallayout class='monospaced'>
$ bitbake <replaceable>recipename</replaceable>
</literallayout>
</para></listitem>
<listitem><para>
Inside the
<link linkend='var-STAMPS_DIR'><filename>STAMPS_DIR</filename></link>
directory, find the signature data
(<filename>sigdata</filename>) file that corresponds to the
task.
The <filename>sigdata</filename> files contain a pickled
Python database of all the metadata that went into creating
the input checksum for the task.
As an example, for the
<link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link>
task of the <filename>db</filename> recipe, the
<filename>sigdata</filename> file might be found in the
following location:
<literallayout class='monospaced'>
${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
</literallayout>
For tasks that are accelerated through the shared state
(<link linkend='shared-state-cache'>sstate</link>)
cache, an additional <filename>siginfo</filename> file is
written into
<link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>
along with the cached task output.
The <filename>siginfo</filename> files contain exactly the
same information as <filename>sigdata</filename> files.
</para></listitem>
<listitem><para>
Run <filename>bitbake-dumpsig</filename> on the
<filename>sigdata</filename> or
<filename>siginfo</filename> file.
Here is an example:
<literallayout class='monospaced'>
$ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
</literallayout>
In the output of the above command, you will find a line
like the following, which lists all the (inferred) variable
dependencies for the task.
This list also includes indirect dependencies from
variables depending on other variables, recursively.
<literallayout class='monospaced'>
Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch']
</literallayout>
<note>
Functions (e.g. <filename>base_do_fetch</filename>)
also count as variable dependencies.
These functions in turn depend on the variables they
reference.
</note>
The output of <filename>bitbake-dumpsig</filename> also includes
the value each variable had, a list of dependencies for each
variable, and
<ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHBASE_WHITELIST'><filename>BB_HASHBASE_WHITELIST</filename></ulink>
information.
</para></listitem>
</orderedlist>
</para>
<para>
There is also a <filename>bitbake-diffsigs</filename> command for
comparing two <filename>siginfo</filename> or
<filename>sigdata</filename> files.
This command can be helpful when trying to figure out what changed
between two versions of a task.
If you call <filename>bitbake-diffsigs</filename> with just one
file, the command behaves like
<filename>bitbake-dumpsig</filename>.
</para>
<para>
You can also use BitBake to dump out the signature construction
information without executing tasks by using either of the
following BitBake command-line options:
<literallayout class='monospaced'>
‐‐dump-signatures=<replaceable>SIGNATURE_HANDLER</replaceable>
-S <replaceable>SIGNATURE_HANDLER</replaceable>
</literallayout>
<note>
Two common values for
<replaceable>SIGNATURE_HANDLER</replaceable> are "none" and
"printdiff", which dump only the signature or compare the
dumped signature with the cached one, respectively.
</note>
Using BitBake with either of these options causes BitBake to dump
out <filename>sigdata</filename> files in the
<filename>stamps</filename> directory for every task it would have
executed instead of building the specified target package.
</para>
</section>
<section id='usingpoky-debugging-taskrunning'>
<title>Running Specific Tasks</title>
<para>
Any given recipe consists of a set of tasks.
The standard BitBake behavior in most cases is:
<filename>do_fetch</filename>,
<filename>do_unpack</filename>,
<filename>do_patch</filename>, <filename>do_configure</filename>,
<filename>do_compile</filename>, <filename>do_install</filename>,
<filename>do_package</filename>,
<filename>do_package_write_*</filename>, and
<filename>do_build</filename>.
The default task is <filename>do_build</filename> and any tasks
on which it depends build first.
Some tasks, such as <filename>do_devshell</filename>, are not part
of the default build chain.
If you wish to run a task that is not part of the default build
chain, you can use the <filename>-c</filename> option in BitBake.
Here is an example:
<literallayout class='monospaced'>
$ bitbake matchbox-desktop -c devshell
</literallayout>
</para>
<para>
The <filename>-c</filename> option respects task dependencies,
which means that all other tasks (including tasks from other
recipes) that the specified task depends on will be run before the
task.
Even when you manually specify a task to run with
<filename>-c</filename>, BitBake will only run the task if it
considers it "out of date".
See the
"<link linkend='stamp-files-and-the-rerunning-of-tasks'>Stamp Files and the Rerunning of Tasks</link>"
section for how BitBake determines whether a task is "out of date".
</para>
<para>
If you want to force an up-to-date task to be rerun (e.g.
because you made manual modifications to the recipe's
<link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
that you want to try out), then you can use the
<filename>-f</filename> option.
<note>
The reason <filename>-f</filename> is never required when
running the
<link linkend='ref-tasks-devshell'><filename>do_devshell</filename></link>
task is because the
<filename>[</filename><ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink><filename>]</filename>
variable flag is already set for the task.
</note>
The following example shows one way you can use the
<filename>-f</filename> option:
<literallayout class='monospaced'>
$ bitbake matchbox-desktop
.
.
make some changes to the source code in the work directory
.
.
$ bitbake matchbox-desktop -c compile -f
$ bitbake matchbox-desktop
</literallayout>
</para>
<para>
This sequence first builds and then recompiles
<filename>matchbox-desktop</filename>.
The last command reruns all tasks (basically the packaging tasks)
after the compile.
BitBake recognizes that the <filename>do_compile</filename>
task was rerun and therefore understands that the other tasks
also need to be run again.
</para>
<para>
Another, shorter way to rerun a task and all
<link linkend='normal-recipe-build-tasks'>normal recipe build tasks</link>
that depend on it is to use the <filename>-C</filename>
option.
<note>
This option is upper-cased and is separate from the
<filename>-c</filename> option, which is lower-cased.
</note>
Using this option invalidates the given task and then runs the
<link linkend='ref-tasks-build'><filename>do_build</filename></link>
task, which is the default task if no task is given, and the
tasks on which it depends.
You could replace the final two commands in the previous example
with the following single command:
<literallayout class='monospaced'>
$ bitbake matchbox-desktop -C compile
</literallayout>
Internally, the <filename>-f</filename> and
<filename>-C</filename> options work by tainting (modifying) the
input checksum of the specified task.
This tainting indirectly causes the task and its
dependent tasks to be rerun through the normal task dependency
mechanisms.
<note>
BitBake explicitly keeps track of which tasks have been
tainted in this fashion, and will print warnings such as the
following for builds involving such tasks:
<literallayout class='monospaced'>
WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run
</literallayout>
The purpose of the warning is to let you know that the work
directory and build output might not be in the clean state they
would be in for a "normal" build, depending on what actions
you took.
To get rid of such warnings, you can remove the work directory
and rebuild the recipe, as follows:
<literallayout class='monospaced'>
$ bitbake matchbox-desktop -c clean
$ bitbake matchbox-desktop
</literallayout>
</note>
</para>
<para>
You can view a list of tasks in a given package by running the
<filename>do_listtasks</filename> task as follows:
<literallayout class='monospaced'>
$ bitbake matchbox-desktop -c listtasks
</literallayout>
The results appear as output to the console and are also in the
file <filename>${WORKDIR}/temp/log.do_listtasks</filename>.
</para>
</section>
<section id='usingpoky-debugging-bitbake'>
<title>General BitBake Problems</title>
<para>
You can see debug output from BitBake by using the <filename>-D</filename> option.
The debug output gives more information about what BitBake
is doing and the reason behind it.
Each <filename>-D</filename> option you use increases the logging level.
The most common usage is <filename>-DDD</filename>.
</para>
<para>
The output from <filename>bitbake -DDD -v</filename> <replaceable>targetname</replaceable> can reveal why
BitBake chose a certain version of a package or why BitBake
picked a certain provider.
This command could also help you in a situation where you think BitBake did something
unexpected.
</para>
</section>
<section id='development-host-system-issues'>
<title>Development Host System Issues</title>
<para>
Sometimes issues on the host development system can cause your
build to fail.
Following are known, host-specific problems.
Be sure to always consult the
<ulink url='&YOCTO_RELEASE_NOTES;'>Release Notes</ulink>
for a look at all release-related issues.
<itemizedlist>
<listitem><para><emphasis><filename>glibc-initial</filename> fails to build</emphasis>:
If your development host system has the unpatched
<filename>GNU Make 3.82</filename>,
the
<link linkend='ref-tasks-install'><filename>do_install</filename></link>
task fails for <filename>glibc-initial</filename> during
the build.</para>
<para>Typically, every distribution that ships
<filename>GNU Make 3.82</filename> as
the default already has the patched version.
However, some distributions, such as Debian, have
<filename>GNU Make 3.82</filename> as an option, which
is unpatched.
You will see this error on these types of distributions.
Switch to <filename>GNU Make 3.81</filename> or patch
your <filename>make</filename> to solve the problem.
</para></listitem>
</itemizedlist>
</para>
</section>
<section id='usingpoky-debugging-buildfile'>
<title>Building with No Dependencies</title>
<para>
To build a specific recipe (<filename>.bb</filename> file),
you can use the following command form:
<literallayout class='monospaced'>
$ bitbake -b <replaceable>somepath</replaceable>/<replaceable>somerecipe</replaceable>.bb
</literallayout>
This command form does not check for dependencies.
Consequently, you should use it
only when you know existing dependencies have been met.
<note>
You can also specify fragments of the filename.
In this case, BitBake checks for a unique match.
</note>
</para>
</section>
<section id='recipe-logging-mechanisms'>
<title>Recipe Logging Mechanisms</title>
<para>
The Yocto Project provides several logging functions for producing
debugging output and reporting errors and warnings.
For Python functions, the following logging functions exist.
All of these functions log to
<filename>${T}/log.do_</filename><replaceable>task</replaceable>,
and can also log to standard output (stdout) with the right
settings:
<itemizedlist>
<listitem><para>
<filename>bb.plain(</filename><replaceable>msg</replaceable><filename>)</filename>:
Writes <replaceable>msg</replaceable> as is to the log while
also logging to stdout.
</para></listitem>
<listitem><para>
<filename>bb.note(</filename><replaceable>msg</replaceable><filename>)</filename>:
Writes "NOTE: <replaceable>msg</replaceable>" to the log.
Also logs to stdout if BitBake is called with "-v".
</para></listitem>
<listitem><para>
<filename>bb.debug(</filename><replaceable>level</replaceable><filename>, </filename><replaceable>msg</replaceable><filename>)</filename>:
Writes "DEBUG: <replaceable>msg</replaceable>" to the log.
Also logs to stdout if the log level is greater than or
equal to <replaceable>level</replaceable>.
See the
"<ulink url='&YOCTO_DOCS_BB_URL;#usage-and-syntax'>-D</ulink>"
option in the BitBake User Manual for more information.
</para></listitem>
<listitem><para>
<filename>bb.warn(</filename><replaceable>msg</replaceable><filename>)</filename>:
Writes "WARNING: <replaceable>msg</replaceable>" to the log
while also logging to stdout.
</para></listitem>
<listitem><para>
<filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename>:
Writes "ERROR: <replaceable>msg</replaceable>" to the log
while also logging to stdout.
<note>
Calling this function does not cause the task to fail.
</note>
</para></listitem>
<listitem><para>
<filename>bb.fatal(</filename><replaceable>msg</replaceable><filename>)</filename>:
This logging function is similar to
<filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename>
but also causes the calling task to fail.
<note>
<filename>bb.fatal()</filename> raises an exception,
which means you do not need to put a "return"
statement after the function.
</note>
</para></listitem>
</itemizedlist>
</para>
<para>
The same logging functions are also available in shell functions,
under the names
<filename>bbplain</filename>, <filename>bbnote</filename>,
<filename>bbdebug</filename>, <filename>bbwarn</filename>,
<filename>bberror</filename>, and <filename>bbfatal</filename>.
The
<link linkend='ref-classes-logging'><filename>logging</filename></link>
class implements these functions.
See that class in the
<filename>meta/classes</filename> folder of the
<ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>
for information.
</para>
<section id='logging-with-python'>
<title>Logging With Python</title>
<para>
When creating recipes using Python and inserting code that handles build logs,
keep in mind the goal is to have informative logs while keeping the console as
"silent" as possible.
Also, if you want status messages in the log, use the "debug" loglevel.
</para>
<para>
Following is an example written in Python.
The code handles logging for a function that determines the
number of tasks needed to be run.
See the
"<link linkend='ref-tasks-listtasks'><filename>do_listtasks</filename></link>"
section for additional information:
<literallayout class='monospaced'>
python do_listtasks() {
bb.debug(2, "Starting to figure out the task list")
if noteworthy_condition:
bb.note("There are 47 tasks to run")
bb.debug(2, "Got to point xyz")
if warning_trigger:
bb.warn("Detected warning_trigger, this might be a problem later.")
if recoverable_error:
bb.error("Hit recoverable_error, you really need to fix this!")
if fatal_error:
bb.fatal("fatal_error detected, unable to print the task list")
bb.plain("The tasks present are abc")
bb.debug(2, "Finished figuring out the tasklist")
}
</literallayout>
</para>
</section>
<section id='logging-with-bash'>
<title>Logging With Bash</title>
<para>
When creating recipes using Bash and inserting code that handles build
logs, you have the same goals - informative with minimal console output.
The syntax you use for recipes written in Bash is similar to that of
recipes written in Python described in the previous section.
</para>
<para>
Following is an example written in Bash.
The code logs the progress of the <filename>do_my_function</filename> function.
<literallayout class='monospaced'>
do_my_function() {
bbdebug 2 "Running do_my_function"
if [ exceptional_condition ]; then
bbnote "Hit exceptional_condition"
fi
bbdebug 2 "Got to point xyz"
if [ warning_trigger ]; then
bbwarn "Detected warning_trigger, this might cause a problem later."
fi
if [ recoverable_error ]; then
bberror "Hit recoverable_error, correcting"
fi
if [ fatal_error ]; then
bbfatal "fatal_error detected"
fi
bbdebug 2 "Completed do_my_function"
}
</literallayout>
</para>
</section>
</section>
<section id='usingpoky-debugging-others'>
<title>Other Tips</title>
<para>
Here are some other tips that you might find useful:
<itemizedlist>
<listitem><para>
When adding new packages, it is worth watching for
undesirable items making their way into compiler command
lines.
For example, you do not want references to local system
files like
<filename>/usr/lib/</filename> or
<filename>/usr/include/</filename>.
</para></listitem>
<listitem><para>
If you want to remove the <filename>psplash</filename>
boot splashscreen,
add <filename>psplash=false</filename> to the kernel
command line.
Doing so prevents <filename>psplash</filename> from loading
and thus allows you to see the console.
It is also possible to switch out of the splashscreen by
switching the virtual console (e.g. Fn+Left or Fn+Right
on a Zaurus).
</para></listitem>
<listitem><para>
Removing
<link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
(usually <filename>tmp/</filename>, within the
<link linkend='build-directory'>Build Directory</link>)
can often fix temporary build issues.
Removing <filename>TMPDIR</filename> is usually a
relatively cheap operation, because task output will be
cached in
<link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>
(usually <filename>sstate-cache/</filename>, which is
also in the Build Directory).
<note>
Removing <filename>TMPDIR</filename> might be a
workaround rather than a fix.
Consequently, trying to determine the underlying cause
of an issue before removing the directory is a good
idea.
</note>
</para></listitem>
<listitem><para>
Understanding how a feature is used in practice within
existing recipes can be very helpful.
It is recommended that you configure some method that
allows you to quickly search through files.</para>
<para>Using GNU Grep, you can use the following shell
function to recursively search through common
recipe-related files, skipping binary files,
<filename>.git</filename> directories, and the
Build Directory (assuming its name starts with
"build"):
<literallayout class='monospaced'>
g() {
grep -Ir \
--exclude-dir=.git \
--exclude-dir='build*' \
--include='*.bb*' \
--include='*.inc*' \
--include='*.conf*' \
--include='*.py*' \
"$@"
}
</literallayout>
Following are some usage examples:
<literallayout class='monospaced'>
$ g FOO # Search recursively for "FOO"
$ g -i foo # Search recursively for "foo", ignoring case
$ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR"
</literallayout>
If figuring out how some feature works requires a lot of
searching, it might indicate that the documentation should
be extended or improved.
In such cases, consider filing a documentation bug using
the Yocto Project implementation of
<ulink url='https://bugzilla.yoctoproject.org/'>Bugzilla</ulink>.
For general information on how to submit a bug against
the Yocto Project, see the Yocto Project Bugzilla
<ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>wiki page</ulink>"
or the
<ulink url='&YOCTO_DOCS_DEV_URL;#submitting-a-defect-against-the-yocto-project'>Submitting a Defect Against the Yocto Project</ulink>"
section, which is in the Yocto Project Development Manual.
<note>
The manuals might not be the right place to document
variables that are purely internal and have a limited
scope (e.g. internal variables used to implement a
single <filename>.bbclass</filename> file).
</note>
</para></listitem>
</itemizedlist>
</para>
</section>
</section>
<section id='ref-quick-emulator-qemu'>
<title>Quick EMUlator (QEMU)</title>
<para>
The Yocto Project uses an implementation of the Quick EMUlator (QEMU)
Open Source project as part of the Yocto Project development "tool
set".
</para>
<para>
Within the context of the Yocto Project, QEMU is an
emulator and virtualization machine that allows you to run a complete
image you have built using the Yocto Project as just another task
on your build system.
QEMU is useful for running and testing images and applications on
supported Yocto Project architectures without having actual hardware.
Among other things, the Yocto Project uses QEMU to run automated
Quality Assurance (QA) tests on final images shipped with each
release.
<note>
This implementation is not the same as QEMU in general.
</note>
This section provides a brief reference for the Yocto Project
implementation of QEMU.
</para>
<para>
For official information and documentation on QEMU in general, see the
following references:
<itemizedlist>
<listitem><para>
<emphasis><ulink url='http://wiki.qemu.org/Main_Page'>QEMU Website</ulink>:</emphasis>
The official website for the QEMU Open Source project.
</para></listitem>
<listitem><para>
<emphasis><ulink url='http://wiki.qemu.org/Manual'>Documentation</ulink>:</emphasis>
The QEMU user manual.
</para></listitem>
</itemizedlist>
</para>
<para>
For information on how to use the Yocto Project implementation of
QEMU, see the
"<ulink url='&YOCTO_DOCS_DEV_URL;#dev-manual-qemu'>Using the Quick EMUlator (QEMU)</ulink>"
chapter in the Yocto Project Development Manual.
</para>
<section id='qemu-availability'>
<title>QEMU Availability</title>
<para>
QEMU is made available with the Yocto Project a number of ways.
One method is to install a Software Development Kit (SDK).
For more information on how to make sure you have
QEMU available, see
"<ulink url='&YOCTO_DOCS_SDK_URL;#the-qemu-emulator'>The QEMU Emulator</ulink>"
section in the Yocto Project Software Development Kit (SDK)
Developer's Guide.
</para>
</section>
<section id='qemu-performance'>
<title>QEMU Performance</title>
<para>
Using QEMU to emulate your hardware can result in speed issues
depending on the target and host architecture mix.
For example, using the <filename>qemux86</filename> image in the
emulator on an Intel-based 32-bit (x86) host machine is fast
because the target and host architectures match.
On the other hand, using the <filename>qemuarm</filename> image
on the same Intel-based host can be slower.
But, you still achieve faithful emulation of ARM-specific issues.
</para>
<para>
To speed things up, the QEMU images support using
<filename>distcc</filename> to call a cross-compiler outside the
emulated system.
If you used <filename>runqemu</filename> to start QEMU, and the
<filename>distccd</filename> application is present on the host
system, any BitBake cross-compiling toolchain available from the
build system is automatically used from within QEMU simply by
calling <filename>distcc</filename>.
You can accomplish this by defining the cross-compiler variable
(e.g. <filename>export CC="distcc"</filename>).
Alternatively, if you are using a suitable SDK image or the
appropriate stand-alone toolchain is present, the toolchain is
also automatically used.
</para>
<note>
Several mechanisms exist that let you connect to the system
running on the QEMU emulator:
<itemizedlist>
<listitem><para>
QEMU provides a framebuffer interface that makes standard
consoles available.
</para></listitem>
<listitem><para>
Generally, headless embedded devices have a serial port.
If so, you can configure the operating system of the
running image to use that port to run a console.
The connection uses standard IP networking.
</para></listitem>
<listitem><para>
SSH servers exist in some QEMU images.
The <filename>core-image-sato</filename> QEMU image has a
Dropbear secure shell (SSH) server that runs with the root
password disabled.
The <filename>core-image-full-cmdline</filename> and
<filename>core-image-lsb</filename> QEMU images
have OpenSSH instead of Dropbear.
Including these SSH servers allow you to use standard
<filename>ssh</filename> and <filename>scp</filename>
commands.
The <filename>core-image-minimal</filename> QEMU image,
however, contains no SSH server.
</para></listitem>
<listitem><para>
You can use a provided, user-space NFS server to boot
the QEMU session using a local copy of the root
filesystem on the host.
In order to make this connection, you must extract a
root filesystem tarball by using the
<filename>runqemu-extract-sdk</filename> command.
After running the command, you must then point the
<filename>runqemu</filename>
script to the extracted directory instead of a root
filesystem image file.
See the
"<ulink url='&YOCTO_DOCS_DEV_URL;#qemu-running-under-a-network-file-system-nfs-server'>Running Under a Network File System (NFS) Server</ulink>"
section in the Yocto Project Development Manual for more
information.
</para></listitem>
</itemizedlist>
</note>
</section>
<section id='qemu-command-line-syntax'>
<title>QEMU Command-Line Syntax</title>
<para>
The basic <filename>runqemu</filename> command syntax is as
follows:
<literallayout class='monospaced'>
$ runqemu [<replaceable>option</replaceable> ] [...]
</literallayout>
Based on what you provide on the command line,
<filename>runqemu</filename> does a good job of figuring out what
you are trying to do.
For example, by default, QEMU looks for the most recently built
image according to the timestamp when it needs to look for an
image.
Minimally, through the use of options, you must provide either
a machine name, a virtual machine image
(<filename>*.vmdk</filename>), or a kernel image
(<filename>*.bin</filename>).
</para>
<para>
Following is the command-line help output for the
<filename>runqemu</filename> command:
<literallayout class='monospaced'>
$ runqemu --help
Usage: you can run this script with any valid combination
of the following environment variables (in any order):
KERNEL - the kernel image file to use
ROOTFS - the rootfs image file or nfsroot directory to use
MACHINE - the machine name (optional, autodetected from KERNEL filename if unspecified)
Simplified QEMU command-line options can be passed with:
nographic - disable video console
serial - enable a serial console on /dev/ttyS0
slirp - enable user networking, no root privileges is required
kvm - enable KVM when running x86/x86_64 (VT-capable CPU required)
kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU required)
publicvnc - enable a VNC server open to all hosts
audio - enable audio
[*/]ovmf* - OVMF firmware file or base name for booting with UEFI
tcpserial=<port> - specify tcp serial port number
biosdir=<dir> - specify custom bios dir
biosfilename=<filename> - specify bios filename
qemuparams=<xyz> - specify custom parameters to QEMU
bootparams=<xyz> - specify custom kernel parameters during boot
help, -h, --help: print this text
Examples:
runqemu
runqemu qemuarm
runqemu tmp/deploy/images/qemuarm
runqemu tmp/deploy/images/qemux86/<qemuboot.conf>
runqemu qemux86-64 core-image-sato ext4
runqemu qemux86-64 wic-image-minimal wic
runqemu path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial
runqemu qemux86 iso/hddimg/vmdk/qcow2/vdi/ramfs/cpio.gz...
runqemu qemux86 qemuparams="-m 256"
runqemu qemux86 bootparams="psplash=false"
runqemu path/to/<image>-<machine>.vmdk
runqemu path/to/<image>-<machine>.wic
</literallayout>
</para>
</section>
<section id='runqemu-command-line-options'>
<title><filename>runqemu</filename> Command-Line Options</title>
<para>
Following is a description of <filename>runqemu</filename>
options you can provide on the command line:
<note><title>Tip</title>
If you do provide some "illegal" option combination or perhaps
you do not provide enough in the way of options,
<filename>runqemu</filename> provides appropriate error
messaging to help you correct the problem.
</note>
<itemizedlist>
<listitem><para>
<replaceable>QEMUARCH</replaceable>:
The QEMU machine architecture, which must be "qemuarm",
"qemuarm64", "qemumips", "qemumips64", "qemuppc",
"qemux86", or "qemux86-64".
</para></listitem>
<listitem><para>
<filename><replaceable>VM</replaceable></filename>:
The virtual machine image, which must be a
<filename>.vmdk</filename> file.
Use this option when you want to boot a
<filename>.vmdk</filename> image.
The image filename you provide must contain one of the
following strings: "qemux86-64", "qemux86", "qemuarm",
"qemumips64", "qemumips", "qemuppc", or "qemush4".
</para></listitem>
<listitem><para>
<replaceable>ROOTFS</replaceable>:
A root filesystem that has one of the following
filetype extensions: "ext2", "ext3", "ext4", "jffs2",
"nfs", or "btrfs".
If the filename you provide for this option uses “nfs”, it
must provide an explicit root filesystem path.
</para></listitem>
<listitem><para>
<replaceable>KERNEL</replaceable>:
A kernel image, which is a <filename>.bin</filename> file.
When you provide a <filename>.bin</filename> file,
<filename>runqemu</filename> detects it and assumes the
file is a kernel image.
</para></listitem>
<listitem><para>
<replaceable>MACHINE</replaceable>:
The architecture of the QEMU machine, which must be one
of the following: "qemux86",
"qemux86", "qemux86-64", "qemuarm", "qemuarm64",
"qemumips", “qemumips64", or "qemuppc".
The <replaceable>MACHINE</replaceable> and
<replaceable>QEMUARCH</replaceable> options are basically
identical.
If you do not provide a <replaceable>MACHINE</replaceable>
option, <filename>runqemu</filename> tries to determine
it based on other options.
</para></listitem>
<listitem><para>
<filename>ramfs</filename>:
Indicates you are booting an initial RAM disk (initramfs)
image, which means the <filename>FSTYPE</filename> is
<filename>cpio.gz</filename>.
</para></listitem>
<listitem><para>
<filename>iso</filename>:
Indicates you are booting an ISO image, which means the
<filename>FSTYPE</filename> is
<filename>.iso</filename>.
</para></listitem>
<listitem><para>
<filename>nographic</filename>:
Disables the video console, which sets the console to
"ttys0".
</para></listitem>
<listitem><para>
<filename>serial</filename>:
Enables a serial console on
<filename>/dev/ttyS0</filename>.
</para></listitem>
<listitem><para>
<filename>biosdir</filename>:
Establishes a custom directory for BIOS, VGA BIOS and
keymaps.
</para></listitem>
<listitem><para>
<filename>biosfilename</filename>:
Establishes a custom BIOS name.
</para></listitem>
<listitem><para>
<filename>qemuparams=\"<replaceable>xyz</replaceable>\"</filename>:
Specifies custom QEMU parameters.
Use this option to pass options other than the simple
"kvm" and "serial" options.
</para></listitem>
<listitem><para><filename>bootparams=\"<replaceable>xyz</replaceable>\"</filename>:
Specifies custom boot parameters for the kernel.
</para></listitem>
<listitem><para>
<filename>audio</filename>:
Enables audio in QEMU.
The <replaceable>MACHINE</replaceable> option must be
either "qemux86" or "qemux86-64" in order for audio to be
enabled.
Additionally, the <filename>snd_intel8x0</filename>
or <filename>snd_ens1370</filename> driver must be
installed in linux guest.
</para></listitem>
<listitem><para>
<filename>slirp</filename>:
Enables "slirp" networking, which is a different way
of networking that does not need root access
but also is not as easy to use or comprehensive
as the default.
</para></listitem>
<listitem><para id='kvm-cond'>
<filename>kvm</filename>:
Enables KVM when running "qemux86" or "qemux86-64"
QEMU architectures.
For KVM to work, all the following conditions must be met:
<itemizedlist>
<listitem><para>
Your <replaceable>MACHINE</replaceable> must be either
qemux86" or "qemux86-64".
</para></listitem>
<listitem><para>
Your build host has to have the KVM modules
installed, which are
<filename>/dev/kvm</filename>.
</para></listitem>
<listitem><para>
The build host <filename>/dev/kvm</filename>
directory has to be both writable and readable.
</para></listitem>
</itemizedlist>
</para></listitem>
<listitem><para>
<filename>kvm-vhost</filename>:
Enables KVM with VHOST support when running "qemux86"
or "qemux86-64" QEMU architectures.
For KVM with VHOST to work, the following conditions must
be met:
<itemizedlist>
<listitem><para>
<link linkend='kvm-cond'>kvm</link> option
conditions must be met.
</para></listitem>
<listitem><para>
Your build host has to have virtio net device, which
are <filename>/dev/vhost-net</filename>.
</para></listitem>
<listitem><para>
The build host <filename>/dev/vhost-net</filename>
directory has to be either readable or writable
and “slirp-enabled”.
</para></listitem>
</itemizedlist>
</para></listitem>
<listitem><para>
<filename>publicvnc</filename>:
Enables a VNC server open to all hosts.
</para></listitem>
</itemizedlist>
</para>
</section>
</section>
<section id='maintaining-build-output-quality'>
<title>Maintaining Build Output Quality</title>
<para>
Many factors can influence the quality of a build.
For example, if you upgrade a recipe to use a new version of an upstream software
package or you experiment with some new configuration options, subtle changes
can occur that you might not detect until later.
Consider the case where your recipe is using a newer version of an upstream package.
In this case, a new version of a piece of software might introduce an optional
dependency on another library, which is auto-detected.
If that library has already been built when the software is building,
the software will link to the built library and that library will be pulled
into your image along with the new software even if you did not want the
library.
</para>
<para>
The
<link linkend='ref-classes-buildhistory'><filename>buildhistory</filename></link>
class exists to help you maintain
the quality of your build output.
You can use the class to highlight unexpected and possibly unwanted
changes in the build output.
When you enable build history, it records information about the contents of
each package and image and then commits that information to a local Git
repository where you can examine the information.
</para>
<para>
The remainder of this section describes the following:
<itemizedlist>
<listitem><para>How you can enable and disable
build history</para></listitem>
<listitem><para>How to understand what the build history contains
</para></listitem>
<listitem><para>How to limit the information used for build history
</para></listitem>
<listitem><para>How to examine the build history from both a
command-line and web interface</para></listitem>
</itemizedlist>
</para>
<section id='enabling-and-disabling-build-history'>
<title>Enabling and Disabling Build History</title>
<para>
Build history is disabled by default.
To enable it, add the following <filename>INHERIT</filename>
statement and set the
<link linkend='var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT</filename></link>
variable to "1" at the end of your
<filename>conf/local.conf</filename> file found in the
<link linkend='build-directory'>Build Directory</link>:
<literallayout class='monospaced'>
INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "1"
</literallayout>
Enabling build history as previously described
causes the build process to collect build
output information and commit it to a local
<link linkend='git'>Git</link> repository.
<note>
Enabling build history increases your build times slightly,
particularly for images, and increases the amount of disk
space used during the build.
</note>
</para>
<para>
You can disable build history by removing the previous statements
from your <filename>conf/local.conf</filename> file.
</para>
</section>
<section id='understanding-what-the-build-history-contains'>
<title>Understanding What the Build History Contains</title>
<para>
Build history information is kept in
<filename>${</filename><link linkend='var-TOPDIR'><filename>TOPDIR</filename></link><filename>}/buildhistory</filename>
in the Build Directory as defined by the
<link linkend='var-BUILDHISTORY_DIR'><filename>BUILDHISTORY_DIR</filename></link>
variable.
The following is an example abbreviated listing:
<imagedata fileref="figures/buildhistory.png" align="center" width="6in" depth="4in" />
</para>
<para>
At the top level, there is a <filename>metadata-revs</filename> file
that lists the revisions of the repositories for the layers enabled
when the build was produced.
The rest of the data splits into separate
<filename>packages</filename>, <filename>images</filename> and
<filename>sdk</filename> directories, the contents of which are
described below.
</para>
<section id='build-history-package-information'>
<title>Build History Package Information</title>
<para>
The history for each package contains a text file that has
name-value pairs with information about the package.
For example, <filename>buildhistory/packages/i586-poky-linux/busybox/busybox/latest</filename>
contains the following:
<literallayout class='monospaced'>
PV = 1.22.1
PR = r32
RPROVIDES =
RDEPENDS = glibc (>= 2.20) update-alternatives-opkg
RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d
PKGSIZE = 540168
FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \
/etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \
/usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \
/usr/share/pixmaps /usr/share/applications /usr/share/idl \
/usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers
FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \
/etc/busybox.links.nosuid /etc/busybox.links.suid
</literallayout>
Most of these name-value pairs correspond to variables used
to produce the package.
The exceptions are <filename>FILELIST</filename>, which is the
actual list of files in the package, and
<filename>PKGSIZE</filename>, which is the total size of files
in the package in bytes.
</para>
<para>
There is also a file corresponding to the recipe from which the
package came (e.g.
<filename>buildhistory/packages/i586-poky-linux/busybox/latest</filename>):
<literallayout class='monospaced'>
PV = 1.22.1
PR = r32
DEPENDS = initscripts kern-tools-native update-rc.d-native \
virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \
virtual/libc virtual/update-alternatives
PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \
busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \
busybox-staticdev busybox-dev busybox-doc busybox-locale busybox
</literallayout>
</para>
<para>
Finally, for those recipes fetched from a version control
system (e.g., Git), a file exists that lists source revisions
that are specified in the recipe and lists the actual revisions
used during the build.
Listed and actual revisions might differ when
<link linkend='var-SRCREV'><filename>SRCREV</filename></link>
is set to
<filename>${<link linkend='var-AUTOREV'>AUTOREV</link>}</filename>.
Here is an example assuming
<filename>buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev</filename>):
<literallayout class='monospaced'>
# SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
# SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
</literallayout>
You can use the <filename>buildhistory-collect-srcrevs</filename>
command with the <filename>-a</filename> option to
collect the stored <filename>SRCREV</filename> values
from build history and report them in a format suitable for
use in global configuration (e.g.,
<filename>local.conf</filename> or a distro include file) to
override floating <filename>AUTOREV</filename> values to a
fixed set of revisions.
Here is some example output from this command:
<literallayout class='monospaced'>
$ buildhistory-collect-srcrevs -a
# i586-poky-linux
SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072"
SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072"
SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a"
SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"
# x86_64-linux
SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa"
SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf"
SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11"
SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072"
SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3"
SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca"
SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a"
SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff"
SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"
# qemux86-poky-linux
SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f"
# all-poky-linux
SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11"
</literallayout>
<note>
Here are some notes on using the
<filename>buildhistory-collect-srcrevs</filename> command:
<itemizedlist>
<listitem><para>By default, only values where the
<filename>SRCREV</filename> was
not hardcoded (usually when <filename>AUTOREV</filename>
was used) are reported.
Use the <filename>-a</filename> option to see all
<filename>SRCREV</filename> values.
</para></listitem>
<listitem><para>The output statements might not have any effect
if overrides are applied elsewhere in the build system
configuration.
Use the <filename>-f</filename> option to add the
<filename>forcevariable</filename> override to each output line
if you need to work around this restriction.
</para></listitem>
<listitem><para>The script does apply special handling when
building for multiple machines.
However, the script does place a
comment before each set of values that specifies
which triplet to which they belong as shown above
(e.g., <filename>i586-poky-linux</filename>).
</para></listitem>
</itemizedlist>
</note>
</para>
</section>
<section id='build-history-image-information'>
<title>Build History Image Information</title>
<para>
The files produced for each image are as follows:
<itemizedlist>
<listitem><para><filename>image-files:</filename>
A directory containing selected files from the root
filesystem.
The files are defined by
<link linkend='var-BUILDHISTORY_IMAGE_FILES'><filename>BUILDHISTORY_IMAGE_FILES</filename></link>.
</para></listitem>
<listitem><para><filename>build-id.txt:</filename>
Human-readable information about the build configuration
and metadata source revisions.
This file contains the full build header as printed
by BitBake.</para></listitem>
<listitem><para><filename>*.dot:</filename>
Dependency graphs for the image that are
compatible with <filename>graphviz</filename>.
</para></listitem>
<listitem><para><filename>files-in-image.txt:</filename>
A list of files in the image with permissions,
owner, group, size, and symlink information.
</para></listitem>
<listitem><para><filename>image-info.txt:</filename>
A text file containing name-value pairs with information
about the image.
See the following listing example for more information.
</para></listitem>
<listitem><para><filename>installed-package-names.txt:</filename>
A list of installed packages by name only.</para></listitem>
<listitem><para><filename>installed-package-sizes.txt:</filename>
A list of installed packages ordered by size.
</para></listitem>
<listitem><para><filename>installed-packages.txt:</filename>
A list of installed packages with full package
filenames.</para></listitem>
</itemizedlist>
<note>
Installed package information is able to be gathered and
produced even if package management is disabled for the final
image.
</note>
</para>
<para>
Here is an example of <filename>image-info.txt</filename>:
<literallayout class='monospaced'>
DISTRO = poky
DISTRO_VERSION = 1.7
USER_CLASSES = buildstats image-mklibs image-prelink
IMAGE_CLASSES = image_types
IMAGE_FEATURES = debug-tweaks
IMAGE_LINGUAS =
IMAGE_INSTALL = packagegroup-core-boot run-postinsts
BAD_RECOMMENDATIONS =
NO_RECOMMENDATIONS =
PACKAGE_EXCLUDE =
ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \
write_image_manifest ; buildhistory_list_installed_image ; \
buildhistory_get_image_installed ; ssh_allow_empty_password; \
postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ;
IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ;
IMAGESIZE = 6900
</literallayout>
Other than <filename>IMAGESIZE</filename>, which is the
total size of the files in the image in Kbytes, the
name-value pairs are variables that may have influenced the
content of the image.
This information is often useful when you are trying to determine
why a change in the package or file listings has occurred.
</para>
</section>
<section id='using-build-history-to-gather-image-information-only'>
<title>Using Build History to Gather Image Information Only</title>
<para>
As you can see, build history produces image information,
including dependency graphs, so you can see why something
was pulled into the image.
If you are just interested in this information and not
interested in collecting specific package or SDK information,
you can enable writing only image information without
any history by adding the following to your
<filename>conf/local.conf</filename> file found in the
<link linkend='build-directory'>Build Directory</link>:
<literallayout class='monospaced'>
INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "0"
BUILDHISTORY_FEATURES = "image"
</literallayout>
Here, you set the
<link linkend='var-BUILDHISTORY_FEATURES'><filename>BUILDHISTORY_FEATURES</filename></link>
variable to use the image feature only.
</para>
</section>
<section id='build-history-sdk-information'>
<title>Build History SDK Information</title>
<para>
Build history collects similar information on the contents
of SDKs
(e.g. <filename>bitbake -c populate_sdk imagename</filename>)
as compared to information it collects for images.
Furthermore, this information differs depending on whether an
extensible or standard SDK is being produced.
</para>
<para>
The following list shows the files produced for SDKs:
<itemizedlist>
<listitem><para><filename>files-in-sdk.txt:</filename>
A list of files in the SDK with permissions,
owner, group, size, and symlink information.
This list includes both the host and target parts
of the SDK.
</para></listitem>
<listitem><para><filename>sdk-info.txt:</filename>
A text file containing name-value pairs with information
about the SDK.
See the following listing example for more information.
</para></listitem>
<listitem><para><filename>sstate-task-sizes.txt:</filename>
A text file containing name-value pairs with information
about task group sizes
(e.g. <filename>do_populate_sysroot</filename> tasks
have a total size).
The <filename>sstate-task-sizes.txt</filename> file
exists only when an extensible SDK is created.
</para></listitem>
<listitem><para><filename>sstate-package-sizes.txt:</filename>
A text file containing name-value pairs with information
for the shared-state packages and sizes in the SDK.
The <filename>sstate-package-sizes.txt</filename> file
exists only when an extensible SDK is created.
</para></listitem>
<listitem><para><filename>sdk-files:</filename>
A folder that contains copies of the files mentioned in
<filename>BUILDHISTORY_SDK_FILES</filename> if the
files are present in the output.
Additionally, the default value of
<filename>BUILDHISTORY_SDK_FILES</filename> is specific
to the extensible SDK although you can set it
differently if you would like to pull in specific files
from the standard SDK.</para>
<para>The default files are
<filename>conf/local.conf</filename>,
<filename>conf/bblayers.conf</filename>,
<filename>conf/auto.conf</filename>,
<filename>conf/locked-sigs.inc</filename>, and
<filename>conf/devtool.conf</filename>.
Thus, for an extensible SDK, these files get copied
into the <filename>sdk-files</filename> directory.
</para></listitem>
<listitem><para>The following information appears under
each of the <filename>host</filename>
and <filename>target</filename> directories
for the portions of the SDK that run on the host and
on the target, respectively:
<note>
The following files for the most part are empty
when producing an extensible SDK because this
type of SDK is not constructed from packages as is
the standard SDK.
</note>
<itemizedlist>
<listitem><para><filename>depends.dot:</filename>
Dependency graph for the SDK that is
compatible with <filename>graphviz</filename>.
</para></listitem>
<listitem><para><filename>installed-package-names.txt:</filename>
A list of installed packages by name only.
</para></listitem>
<listitem><para><filename>installed-package-sizes.txt:</filename>
A list of installed packages ordered by size.
</para></listitem>
<listitem><para><filename>installed-packages.txt:</filename>
A list of installed packages with full package
filenames.</para></listitem>
</itemizedlist>
</para></listitem>
</itemizedlist>
</para>
<para>
Here is an example of <filename>sdk-info.txt</filename>:
<literallayout class='monospaced'>
DISTRO = poky
DISTRO_VERSION = 1.3+snapshot-20130327
SDK_NAME = poky-glibc-i686-arm
SDK_VERSION = 1.3+snapshot
SDKMACHINE =
SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs
BAD_RECOMMENDATIONS =
SDKSIZE = 352712
</literallayout>
Other than <filename>SDKSIZE</filename>, which is the
total size of the files in the SDK in Kbytes, the
name-value pairs are variables that might have influenced the
content of the SDK.
This information is often useful when you are trying to
determine why a change in the package or file listings
has occurred.
</para>
</section>
<section id='examining-build-history-information'>
<title>Examining Build History Information</title>
<para>
You can examine build history output from the command line or
from a web interface.
</para>
<para>
To see any changes that have occurred (assuming you have
<link linkend='var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT = "1"</filename></link>),
you can simply
use any Git command that allows you to view the history of
a repository.
Here is one method:
<literallayout class='monospaced'>
$ git log -p
</literallayout>
You need to realize, however, that this method does show
changes that are not significant (e.g. a package's size
changing by a few bytes).
</para>
<para>
A command-line tool called <filename>buildhistory-diff</filename>
does exist, though, that queries the Git repository and prints just
the differences that might be significant in human-readable form.
Here is an example:
<literallayout class='monospaced'>
$ ~/poky/poky/scripts/buildhistory-diff . HEAD^
Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):
/etc/anotherpkg.conf was added
/sbin/anotherpkg was added
* (installed-package-names.txt):
* anotherpkg was added
Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt):
anotherpkg was added
packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras"
* PR changed from "r0" to "r1"
* PV changed from "0.1.10" to "0.1.12"
packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%)
* PR changed from "r0" to "r1"
* PV changed from "0.1.10" to "0.1.12"
</literallayout>
<note>
The <filename>buildhistory-diff</filename> tool requires
the <filename>GitPython</filename> package.
Be sure to install it using Pip3 as follows:
<literallayout class='monospaced'>
$ pip3 install GitPython --user
</literallayout>
Alternatively, you can install
<filename>python3-git</filename> using the appropriate
distribution package manager (e.g.
<filename>apt-get</filename>, <filename>dnf</filename>, or
<filename>zipper</filename>).
</note>
</para>
<para>
To see changes to the build history using a web interface, follow
the instruction in the <filename>README</filename> file here.
<ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/'></ulink>.
</para>
<para>
Here is a sample screenshot of the interface:
<imagedata fileref="figures/buildhistory-web.png" align="center" scalefit="1" width="130%" contentdepth="130%" />
</para>
</section>
</section>
</section>
<section id='speeding-up-the-build'>
<title>Speeding Up the Build</title>
<para>
Build time can be an issue.
By default, the build system uses simple controls to try and maximize
build efficiency.
In general, the default settings for all the following variables
result in the most efficient build times when dealing with single
socket systems (i.e. a single CPU).
If you have multiple CPUs, you might try increasing the default
values to gain more speed.
See the descriptions in the glossary for each variable for more
information:
<itemizedlist>
<listitem><para>
<link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename>:</link>
The maximum number of threads BitBake simultaneously executes.
</para></listitem>
<listitem><para>
<ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename>:</ulink>
The number of threads BitBake uses during parsing.
</para></listitem>
<listitem><para>
<link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename>:</link>
Extra options passed to the <filename>make</filename> command
during the
<link linkend='ref-tasks-compile'><filename>do_compile</filename></link>
task in order to specify parallel compilation on the
local build host.
</para></listitem>
<listitem><para>
<link linkend='var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename>:</link>
Extra options passed to the <filename>make</filename> command
during the
<link linkend='ref-tasks-install'><filename>do_install</filename></link>
task in order to specify parallel installation on the
local build host.
</para></listitem>
</itemizedlist>
As mentioned, these variables all scale to the number of processor
cores available on the build system.
For single socket systems, this auto-scaling ensures that the build
system fundamentally takes advantage of potential parallel operations
during the build based on the build machine's capabilities.
</para>
<para>
Following are additional factors that can affect build speed:
<itemizedlist>
<listitem><para>
File system type:
The file system type that the build is being performed on can
also influence performance.
Using <filename>ext4</filename> is recommended as compared
to <filename>ext2</filename> and <filename>ext3</filename>
due to <filename>ext4</filename> improved features
such as extents.
</para></listitem>
<listitem><para>
Disabling the updating of access time using
<filename>noatime</filename>:
The <filename>noatime</filename> mount option prevents the
build system from updating file and directory access times.
</para></listitem>
<listitem><para>
Setting a longer commit:
Using the "commit=" mount option increases the interval
in seconds between disk cache writes.
Changing this interval from the five second default to
something longer increases the risk of data loss but decreases
the need to write to the disk, thus increasing the build
performance.
</para></listitem>
<listitem><para>
Choosing the packaging backend:
Of the available packaging backends, IPK is the fastest.
Additionally, selecting a singular packaging backend also
helps.
</para></listitem>
<listitem><para>
Using <filename>tmpfs</filename> for
<link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
as a temporary file system:
While this can help speed up the build, the benefits are
limited due to the compiler using
<filename>-pipe</filename>.
The build system goes to some lengths to avoid
<filename>sync()</filename> calls into the
file system on the principle that if there was a significant
failure, the
<link linkend='build-directory'>Build Directory</link>
contents could easily be rebuilt.
</para></listitem>
<listitem><para>
Inheriting the
<link linkend='ref-classes-rm-work'><filename>rm_work</filename></link>
class:
Inheriting this class has shown to speed up builds due to
significantly lower amounts of data stored in the data
cache as well as on disk.
Inheriting this class also makes cleanup of
<link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
faster, at the expense of being easily able to dive into the
source code.
File system maintainers have recommended that the fastest way
to clean up large numbers of files is to reformat partitions
rather than delete files due to the linear nature of partitions.
This, of course, assumes you structure the disk partitions and
file systems in a way that this is practical.
</para></listitem>
</itemizedlist>
Aside from the previous list, you should keep some trade offs in
mind that can help you speed up the build:
<itemizedlist>
<listitem><para>
Remove items from
<link linkend='var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></link>
that you might not need.
</para></listitem>
<listitem><para>
Exclude debug symbols and other debug information:
If you do not need these symbols and other debug information,
disabling the <filename>*-dbg</filename> package generation
can speed up the build.
You can disable this generation by setting the
<link linkend='var-INHIBIT_PACKAGE_DEBUG_SPLIT'><filename>INHIBIT_PACKAGE_DEBUG_SPLIT</filename></link>
variable to "1".
</para></listitem>
<listitem><para>
Disable static library generation for recipes derived from
<filename>autoconf</filename> or <filename>libtool</filename>:
Following is an example showing how to disable static
libraries and still provide an override to handle exceptions:
<literallayout class='monospaced'>
STATICLIBCONF = "--disable-static"
STATICLIBCONF_sqlite3-native = ""
EXTRA_OECONF += "${STATICLIBCONF}"
</literallayout>
<note><title>Notes</title>
<itemizedlist>
<listitem><para>
Some recipes need static libraries in order to work
correctly (e.g. <filename>pseudo-native</filename>
needs <filename>sqlite3-native</filename>).
Overrides, as in the previous example, account for
these kinds of exceptions.
</para></listitem>
<listitem><para>
Some packages have packaging code that assumes the
presence of the static libraries.
If so, you might need to exclude them as well.
</para></listitem>
</itemizedlist>
</note>
</para></listitem>
</itemizedlist>
</para>
</section>
</chapter>
<!--
vim: expandtab tw=80 ts=4
-->
|