summaryrefslogtreecommitdiffstats
path: root/documentation
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2014-11-10 15:35:59 -0600
committerRichard Purdie <richard.purdie@linuxfoundation.org>2014-12-09 15:09:05 +0000
commit146d4bb4ddd801580c27d38066a5ecbf508fc2ff (patch)
tree6351b48e1d96a285db58da03c7c1426b0ed87697 /documentation
parent62a67c813a9fe7e3c3fef6f5dda2d1699af55c69 (diff)
downloadpoky-146d4bb4ddd801580c27d38066a5ecbf508fc2ff.tar.gz
ref-manual: WIP - Edits for the speed section.
(From yocto-docs rev: 3a158dbefe32fb1e87718558462c0562077052c8) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation')
-rw-r--r--documentation/ref-manual/usingpoky.xml281
1 files changed, 142 insertions, 139 deletions
diff --git a/documentation/ref-manual/usingpoky.xml b/documentation/ref-manual/usingpoky.xml
index 030448df0b..41f872c07e 100644
--- a/documentation/ref-manual/usingpoky.xml
+++ b/documentation/ref-manual/usingpoky.xml
@@ -89,145 +89,6 @@
89 </section> 89 </section>
90</section> 90</section>
91 91
92<section id='speeding-up-the-build'>
93 <title>Speeding Up the Build</title>
94
95 <para>
96 Build time can be an issue.
97 By default, the build system uses three simple controls to try and
98 maximize build efficiency:
99 <itemizedlist>
100 <listitem><para>
101 <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>
102 </para></listitem>
103 <listitem><para>
104 <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename></ulink>
105 </para></listitem>
106 <listitem><para>
107 <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>
108 </para></listitem>
109 </itemizedlist>
110 These three variables all scale to the number of processor cores
111 available on the build system.
112 This auto-scaling ensures that the build system fundamentally takes
113 advantage of potential parallel operations during the build
114 based on the build machine's capabilities.
115 </para>
116
117 <para>
118 If you need to achieve even faster builds than what the build system
119 produces by default, you can consider and implement some of the
120 following:
121 <itemizedlist>
122 <listitem><para>
123 <filename>BB_NUMBER_THREADS</filename>,
124 <filename>BB_NUMBER_PARSE_THREADS</filename>, and
125 <filename>PARALLEL_MAKE</filename>:
126 As previously mentioned, the build system scales the values
127 for these variables.
128 However, you can manually override them in your
129 <filename>local.conf</filename> file if you are not satisfied
130 with the defaults.
131 </para></listitem>
132 <listitem><para>
133 File system type:
134 The file system type that the build is being performed on can
135 also influence performance.
136 Using <filename>ext4</filename> is recommended as compared
137 to <filename>ext2</filename> and <filename>ext3</filename>
138 due to <filename>ext4</filename> improved features
139 such as extents.
140 </para></listitem>
141 <listitem><para>
142 Disabling the updating of access time using
143 <filename>noatime</filename>:
144 The <filename>noatime</filename> mount option prevents the
145 build system from updating file and directory access times.
146 </para></listitem>
147 <listitem><para>
148 Setting a longer commit:
149 Richard - I need some sort of general summary here about this.
150 I don't know the context.
151 </para></listitem>
152 <listitem><para>
153 The packaging backend:
154 Of the available packaging backends, IPK is the fastest.
155 Additionally, selecting a singular packaging backend also
156 helps.
157 </para></listitem>
158 <listitem><para>
159 Using <filename>/tmp</filename> as a temporary file system:
160 While this can help speed up the build, the benefits are
161 limited due to the compiler using
162 <filename>-pipe</filename>.
163 The build system goes to some lengths to avoid
164 <filename>sync()</filename> calls into the
165 file system on the principle that if there was a significant
166 failure, the
167 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
168 contents could easily be rebuilt.
169 </para></listitem>
170 <listitem><para>
171 Inheriting the
172 <link linkend='ref-classes-rm-work'><filename>rm_work</filename></link>
173 class:
174 Inheriting this class has shown to speed up builds due to
175 significantly lower amounts of data stored in the data
176 cache as well as on disk.
177 Inheriting this class also makes cleanup of
178 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
179 faster, at the expense of being easily able to dive into the
180 source code.
181 File system maintainers have recommended that the fastest way
182 to clean up large numbers of files is to reformat partitions
183 rather than delete files due to the linear nature of partitions.
184 This, of course, assumes you structure the disk partitions and
185 file systems in a way that this is practical.
186 </para></listitem>
187 </itemizedlist>
188 Aside from the previous list, you should keep some trade offs in
189 mind that can help you speed up the build:
190 <itemizedlist>
191 <listitem><para>
192 Exclude debug symbols and other debug information:
193 If you do not need these symbols and other debug information,
194 disabling the <filename>*-dbg</filename> package generation
195 can speed up the build.
196 You can disable this generation by setting the
197 <link linkend='var-INHIBIT_PACKAGE_DEBUG_SPLIT'><filename>INHIBIT_PACKAGE_DEBUG_SPLIT</filename></link>
198 variable to "1".
199 </para></listitem>
200 <listitem><para>
201 Disable static library generation for recipes derived from
202 <filename>autoconf</filename> or <filename>libtool</filename>:
203 Following is an example showing how to disable static
204 libraries and still provide an override to handle exceptions:
205 <literallayout class='monospaced'>
206 STATICLIBCONF = "--disable-static"
207 STATICLIBCONF_sqlite3-native = ""
208 EXTRA_OECONF += "${STATICLIBCONF}"
209 </literallayout>
210 <note><title>Notes</title>
211 <itemizedlist>
212 <listitem><para>
213 Some recipes need static libraries in order to work
214 correctly (e.g. <filename>pseudo-native</filename>
215 needs <filename>sqlite3-native</filename>).
216 Overrides, as in the previous example, account for
217 these kinds of exceptions.
218 </para></listitem>
219 <listitem><para>
220 Some packages have packaging code that assumes the
221 presence of the static libraries.
222 If so, you might need to exclude them as well.
223 </para></listitem>
224 </itemizedlist>
225 </note>
226 </para></listitem>
227 </itemizedlist>
228 </para>
229</section>
230
231<section id='usingpoky-install'> 92<section id='usingpoky-install'>
232 <title>Installing and Using the Result</title> 93 <title>Installing and Using the Result</title>
233 94
@@ -1027,6 +888,148 @@
1027 </section> 888 </section>
1028</section> 889</section>
1029 890
891<section id='speeding-up-the-build'>
892 <title>Speeding Up the Build</title>
893
894 <para>
895 Build time can be an issue.
896 By default, the build system uses three simple controls to try and
897 maximize build efficiency:
898 <itemizedlist>
899 <listitem><para>
900 <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>
901 </para></listitem>
902 <listitem><para>
903 <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename></ulink>
904 </para></listitem>
905 <listitem><para>
906 <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>
907 </para></listitem>
908 </itemizedlist>
909 These three variables all scale to the number of processor cores
910 available on the build system.
911 This auto-scaling ensures that the build system fundamentally takes
912 advantage of potential parallel operations during the build
913 based on the build machine's capabilities.
914 </para>
915
916 <para>
917 If you need to achieve even faster builds than what the build system
918 produces by default, you can consider and implement some of the
919 following:
920 <itemizedlist>
921 <listitem><para>
922 <filename>BB_NUMBER_THREADS</filename>,
923 <filename>BB_NUMBER_PARSE_THREADS</filename>, and
924 <filename>PARALLEL_MAKE</filename>:
925 As previously mentioned, the build system scales the values
926 for these variables.
927 However, you can manually override them in your
928 <filename>local.conf</filename> file if you are not satisfied
929 with the defaults.
930 </para></listitem>
931 <listitem><para>
932 File system type:
933 The file system type that the build is being performed on can
934 also influence performance.
935 Using <filename>ext4</filename> is recommended as compared
936 to <filename>ext2</filename> and <filename>ext3</filename>
937 due to <filename>ext4</filename> improved features
938 such as extents.
939 </para></listitem>
940 <listitem><para>
941 Disabling the updating of access time using
942 <filename>noatime</filename>:
943 The <filename>noatime</filename> mount option prevents the
944 build system from updating file and directory access times.
945 </para></listitem>
946 <listitem><para>
947 Setting a longer commit:
948 Using the "commit=" mount option increases the interval
949 in seconds between disk cache writes.
950 Changing this interval from the five second default to
951 something longer increases the risk of data loss but decreases
952 the need to write to the disk, thus increasing the build
953 performance.
954 </para></listitem>
955 <listitem><para>
956 Choosing the packaging backend:
957 Of the available packaging backends, IPK is the fastest.
958 Additionally, selecting a singular packaging backend also
959 helps.
960 </para></listitem>
961 <listitem><para>
962 Using <filename>/tmp</filename> as a temporary file system:
963 While this can help speed up the build, the benefits are
964 limited due to the compiler using
965 <filename>-pipe</filename>.
966 The build system goes to some lengths to avoid
967 <filename>sync()</filename> calls into the
968 file system on the principle that if there was a significant
969 failure, the
970 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
971 contents could easily be rebuilt.
972 </para></listitem>
973 <listitem><para>
974 Inheriting the
975 <link linkend='ref-classes-rm-work'><filename>rm_work</filename></link>
976 class:
977 Inheriting this class has shown to speed up builds due to
978 significantly lower amounts of data stored in the data
979 cache as well as on disk.
980 Inheriting this class also makes cleanup of
981 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
982 faster, at the expense of being easily able to dive into the
983 source code.
984 File system maintainers have recommended that the fastest way
985 to clean up large numbers of files is to reformat partitions
986 rather than delete files due to the linear nature of partitions.
987 This, of course, assumes you structure the disk partitions and
988 file systems in a way that this is practical.
989 </para></listitem>
990 </itemizedlist>
991 Aside from the previous list, you should keep some trade offs in
992 mind that can help you speed up the build:
993 <itemizedlist>
994 <listitem><para>
995 Exclude debug symbols and other debug information:
996 If you do not need these symbols and other debug information,
997 disabling the <filename>*-dbg</filename> package generation
998 can speed up the build.
999 You can disable this generation by setting the
1000 <link linkend='var-INHIBIT_PACKAGE_DEBUG_SPLIT'><filename>INHIBIT_PACKAGE_DEBUG_SPLIT</filename></link>
1001 variable to "1".
1002 </para></listitem>
1003 <listitem><para>
1004 Disable static library generation for recipes derived from
1005 <filename>autoconf</filename> or <filename>libtool</filename>:
1006 Following is an example showing how to disable static
1007 libraries and still provide an override to handle exceptions:
1008 <literallayout class='monospaced'>
1009 STATICLIBCONF = "--disable-static"
1010 STATICLIBCONF_sqlite3-native = ""
1011 EXTRA_OECONF += "${STATICLIBCONF}"
1012 </literallayout>
1013 <note><title>Notes</title>
1014 <itemizedlist>
1015 <listitem><para>
1016 Some recipes need static libraries in order to work
1017 correctly (e.g. <filename>pseudo-native</filename>
1018 needs <filename>sqlite3-native</filename>).
1019 Overrides, as in the previous example, account for
1020 these kinds of exceptions.
1021 </para></listitem>
1022 <listitem><para>
1023 Some packages have packaging code that assumes the
1024 presence of the static libraries.
1025 If so, you might need to exclude them as well.
1026 </para></listitem>
1027 </itemizedlist>
1028 </note>
1029 </para></listitem>
1030 </itemizedlist>
1031 </para>
1032</section>
1030</chapter> 1033</chapter>
1031<!-- 1034<!--
1032vim: expandtab tw=80 ts=4 1035vim: expandtab tw=80 ts=4
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-05 07:14:19 +0000