1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
|
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<chapter>
<title>File Download Support</title>
<para>
BitBake's <filename>fetch</filename> and
<filename>fetch2</filename> modules support downloading
files.
This chapter provides an overview of the fetching process
and also presents sections on each of the fetchers BitBake
supports.
<note>
The original <filename>fetch</filename> code, for all
practical purposes, has been replaced by
<filename>fetch2</filename> code.
Consequently, the information in this chapter does not
apply to <filename>fetch</filename>.
</note>
</para>
<section id='file-download-overview'>
<title>Overview</title>
<para>
When BitBake starts to execute, the very first thing
it does is to fetch the source files needed.
This section overviews the process.
</para>
<para>
When BitBake goes looking for source files, it follows a search
order:
<orderedlist>
<listitem><para><emphasis>Pre-mirror Sites:</emphasis>
BitBake first uses pre-mirrors to try and find source
files.
These locations are defined using the
<link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link>
variable.
</para></listitem>
<listitem><para><emphasis>Source URI:</emphasis>
If pre-mirrors fail, BitBake uses
<link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>.
</para></listitem>
<listitem><para><emphasis>Mirror Sites:</emphasis>
If fetch failures occur using <filename>SRC_URI</filename>,
BitBake next uses mirror location as defined by the
<link linkend='var-MIRRORS'><filename>MIRRORS</filename></link>
variable.
</para></listitem>
</orderedlist>
</para>
<para>
Because cross-URLs are supported, it is possible to mirror
a Git repository on an HTTP server as a tarball.
Here are some examples that show commonly used mirror
definitions:
<literallayout class='monospaced'>
PREMIRRORS ?= "\
bzr://.*/.* http://somemirror.org/sources/ \n \
cvs://.*/.* http://somemirror.org/sources/ \n \
git://.*/.* http://somemirror.org/sources/ \n \
hg://.*/.* http://somemirror.org/sources/ \n \
osc://.*/.* http://somemirror.org/sources/ \n \
p4://.*/.* http://somemirror.org/sources/ \n \
svk://.*/.* http://somemirror.org/sources/ \n \
svn://.*/.* http://somemirror.org/sources/ \n"
MIRRORS =+ "\
ftp://.*/.* http://somemirror.org/sources/ \n \
http://.*/.* http://somemirror.org/sources/ \n \
https://.*/.* http://somemirror.org/sources/ \n"
</literallayout>
</para>
<para>
Any source files that are not local (i.e. downloaded from
the Internet) are placed into the download directory,
which is specified by
<link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>.
</para>
<para>
For non-local archive downloads, the fetcher code can verify
sha256 and md5 checksums to ensure
the archives have been downloaded correctly.
You can specify these checksums by using the
<filename>SRC_URI</filename> variable with the appropriate
varflags as follows:
<literallayout class='monospaced'>
SRC_URI[md5sum]
SRC_URI[sha256sum]
</literallayout>
You can also specify the checksums as parameters on the
<filename>SRC_URI</filename> as shown below:
<literallayout class='monospaced'>
SRC_URI="http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d"
</literallayout>
If
<link linkend='var-BB_STRICT_CHECKSUM'><filename>BB_STRICT_CHECKSUM</filename></link>
is set, any download without a checksum triggers an error message.
In cases where multiple files are listed using
<filename>SRC_URI</filename>, the name parameter is used
assign names to the URLs and these are then specified
in the checksums using the following form:
<literallayout class='monospaced'>
SRC_URI[name.sha256sum]
</literallayout>
</para>
</section>
<section id='bb-fetchers'>
<title>Fetchers</title>
<para>
As mentioned in the previous section, the
<filename>SRC_URI</filename> is normally used to
tell BitBake which files to fetch.
And, the fetcher BitBake uses depends on the how
<filename>SRC_URI</filename> is set.
</para>
<para>
These next few sections describe the available fetchers and
their options.
Each fetcher honors a set of variables URI parameters,
which are separated by semi-colon characters and consist
of a key and a value.
The semantics of the variables and parameters are
defined by the fetcher.
BitBake tries to have consistent semantics between the
different fetchers.
</para>
<section id='local-file-fetcher'>
<title>Local file fetcher</title>
<para>
The URN for the local file fetcher is file.
</para>
<para>
The filename can be either absolute or relative.
If the filename is relative,
<link linkend='var-FILESPATH'><filename>FILESPATH</filename></link>
is used.
Failing that,
<link linkend='var-FILESDIR'><filename>FILESDIR</filename></link>
is used to find the appropriate relative file.
</para>
<para>
The metadata usually extend these variables to include
variations of the values in
<link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>.
Single files and complete directories can be specified.
<literallayout class='monospaced'>
SRC_URI = "file://relativefile.patch"
SRC_URI = "file://relativefile.patch;this=ignored"
SRC_URI = "file:///Users/ich/very_important_software"
</literallayout>
</para>
</section>
<section id='cvs-fetcher'>
<title>CVS fetcher</title>
<para>
The URN for the CVS fetcher is cvs.
</para>
<para>
This fetcher honors the variables <filename>CVSDIR</filename>,
<filename>SRCDATE</filename>, <filename>FETCHCOMMAND_cvs</filename>,
<filename>UPDATECOMMAND_cvs</filename>.
The
<link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
variable specifies where a
temporary checkout is saved.
The
<link linkend='var-SRCDATE'><filename>SRCDATE</filename></link>
variable specifies which date to
use when doing the fetching.
The special value of "now" causes the checkout to be
updated on every build.
The <filename>FETCHCOMMAND</filename> and
<filename>UPDATECOMMAND</filename> variables specify the executables
to use for the CVS checkout or update.
</para>
<para>
The supported parameters are as follows:
<itemizedlist>
<listitem><para><emphasis>"module":</emphasis>
Specifies the module to check out.
</para></listitem>
<listitem><para><emphasis>"tag":</emphasis>
Describes which CVS TAG should be used for
the checkout.
By default, the TAG is empty.
</para></listitem>
<listitem><para><emphasis>"date":</emphasis>
Specifies a date.
If no "date" is specified, the
<link linkend='var-SRCDATE'><filename>SRCDATE</filename></link>
of the configuration is used to checkout a specific date.
The special value of "now" causes the checkout to be
updated on every build.
</para></listitem>
<listitem><para><emphasis>"method":</emphasis>
By default <filename>pserver</filename>.
If "method" is set to "ext", BitBake examines the "rsh"
parameter and sets <filename>CVS_RSH</filename>.
</para></listitem>
<listitem><para><emphasis>"localdir":</emphasis>
Used to checkout force into a special
directory relative to <filename>CVSDIR</filename>.
</para></listitem>
<listitem><para><emphasis>"rsh"</emphasis>
Used in conjunction with the "method" parameter.
</para></listitem>
<listitem><para><emphasis>"scmdata":</emphasis>
I need a description for this.
</para></listitem>
</itemizedlist>
Following are two examples using cvs:
<literallayout class='monospaced'>
SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext"
SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat"
</literallayout>
</para>
</section>
<section id='http-ftp-fetcher'>
<title>HTTP/FTP fetcher</title>
<para>
The URNs for the HTTP/FTP fetcher are http, https, and ftp.
</para>
<para>
This fetcher honors the variables
<filename>FETCHCOMMAND_wget</filename>.
The <filename>FETCHCOMMAND</filename> variable
contains the command used for fetching.
“${URI}” and “${FILES}” are replaced by the URI and
the base name of the file to be fetched.
<literallayout class='monospaced'>
SRC_URI = "http://oe.handhelds.org/not_there.aac"
SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac"
SRC_URI = "ftp://you@oe.handheld.sorg/home/you/secret.plan"
</literallayout>
</para>
</section>
<section id='svn-fetcher'>
<title>SVN Fetcher</title>
<para>
The URN for the SVN fetcher is svn.
</para>
<para>
This fetcher honors the variables
<filename>FETCHCOMMAND_svn</filename>,
<filename>SVNDIR</filename>,
and
<link linkend='var-SRCREV'><filename>SRCREV</filename></link>.
The <filename>FETCHCOMMAND</filename> variable contains the
<filename>subversion</filename> command.
The <filename>SRCREV</filename> variable specifies which revision
to use when doing the fetching.
</para>
<para>
The supported parameters are as follows:
<itemizedlist>
<listitem><para><emphasis>"proto":</emphasis>
The Subversion protocol.
</para></listitem>
<listitem><para><emphasis>"rev":</emphasis>
The Subversion revision.
</para></listitem>
<listitem><para><emphasis>"scmdata":</emphasis>
Set to "keep" causes the “.svn” directories
to be available during compile-time.
</para></listitem>
</itemizedlist>
Following are two examples using svn:
<literallayout class='monospaced'>
SRC_URI = "svn://svn.oe.handhelds.org/svn;module=vip;proto=http;rev=667"
SRC_URI = "svn://svn.oe.handhelds.org/svn/;module=opie;proto=svn+ssh;date=20060126"
</literallayout>
</para>
</section>
<section id='git-fetcher'>
<title>GIT Fetcher</title>
<para>
The URN for the Git Fetcher is git.
</para>
<para>
The variable <filename>GITDIR</filename> is used as the
base directory in which the Git tree is cloned.
</para>
<para>
The supported parameters are as follows:
<itemizedlist>
<listitem><para><emphasis>"tag":</emphasis>
The Git tag.
The default is "master".
</para></listitem>
<listitem><para><emphasis>"protocol":</emphasis>
The Git protocol.
The default is "git" when a hostname is set.
If a hostname is not set, the Git protocol is "file".
</para></listitem>
<listitem><para><emphasis>"scmdata":</emphasis>
When set to “keep”, the “.git” directory is available
during compile-time.
</para></listitem>
</itemizedlist>
Following are two examples using git:
<literallayout class='monospaced'>
SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1"
SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http"
</literallayout>
</para>
</section>
</section>
</chapter>
|
