diff options
| author | Richard Purdie <richard.purdie@linuxfoundation.org> | 2014-04-11 17:38:18 +0100 |
|---|---|---|
| committer | Richard Purdie <richard.purdie@linuxfoundation.org> | 2014-04-21 23:04:51 +0100 |
| commit | 72532539726c6129f51c206009bdbdd7da069e5c (patch) | |
| tree | 9b7b8131d75843797a7751cf9262a28ebcd7df56 /bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml | |
| parent | cddb415f7276ef5c0b4e974a27af3892ccdc0283 (diff) | |
| download | poky-72532539726c6129f51c206009bdbdd7da069e5c.tar.gz | |
bitbake: doc: Rename user-manual -> bitbake-user-manual
This manual gets combined with other manuals and in that context, it helps
a lot if its seen as the Bitbake User Manual. Renames are a pain but
this is worthwhile so that other docs can correctly be combined with this
one. This also clarifies things like google search results which is helpful.
(Bitbake rev: 452a62ae0c2793e281d6769fd3e45500a74898d6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml')
| -rw-r--r-- | bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml | 622 |
1 files changed, 622 insertions, 0 deletions
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml new file mode 100644 index 0000000000..5aa53defc4 --- /dev/null +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml | |||
| @@ -0,0 +1,622 @@ | |||
| 1 | <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" | ||
| 2 | "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> | ||
| 3 | |||
| 4 | <chapter> | ||
| 5 | <title>File Download Support</title> | ||
| 6 | |||
| 7 | <para> | ||
| 8 | BitBake's fetch module is a standalone piece of library code | ||
| 9 | that deals with the intricacies of downloading source code | ||
| 10 | and files from remote systems. | ||
| 11 | Fetching source code is one of the corner stones of building software. | ||
| 12 | As such, this module forms an important part of BitBake. | ||
| 13 | </para> | ||
| 14 | |||
| 15 | <para> | ||
| 16 | The current fetch module is called "fetch2" and refers to the | ||
| 17 | fact that it is the second major version of the API. | ||
| 18 | The original version is obsolete and removed from the codebase. | ||
| 19 | Thus, in all cases, "fetch" refers to "fetch2" in this | ||
| 20 | manual. | ||
| 21 | </para> | ||
| 22 | |||
| 23 | <section id='the-download-fetch'> | ||
| 24 | <title>The Download (Fetch)</title> | ||
| 25 | |||
| 26 | <para> | ||
| 27 | BitBake takes several steps when fetching source code or files. | ||
| 28 | The fetcher codebase deals with two distinct processes in order: | ||
| 29 | obtaining the files from somewhere (cached or otherwise) | ||
| 30 | and then unpacking those files into a specific location and | ||
| 31 | perhaps in a specific way. | ||
| 32 | Getting and unpacking the files is often optionally followed | ||
| 33 | by patching. | ||
| 34 | Patching, however, is not covered by this module. | ||
| 35 | </para> | ||
| 36 | |||
| 37 | <para> | ||
| 38 | The code to execute the first part of this process, a fetch, | ||
| 39 | looks something like the following: | ||
| 40 | <literallayout class='monospaced'> | ||
| 41 | src_uri = (d.getVar('SRC_URI', True) or "").split() | ||
| 42 | fetcher = bb.fetch2.Fetch(src_uri, d) | ||
| 43 | fetcher.download() | ||
| 44 | </literallayout> | ||
| 45 | This code sets up an instance of the fetch class. | ||
| 46 | The instance uses a space-separated list of URLs from the | ||
| 47 | <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> | ||
| 48 | variable and then calls the <filename>download</filename> | ||
| 49 | method to download the files. | ||
| 50 | </para> | ||
| 51 | |||
| 52 | <para> | ||
| 53 | The instantiation of the fetch class is usually followed by: | ||
| 54 | <literallayout class='monospaced'> | ||
| 55 | rootdir = l.getVar('WORKDIR', True) | ||
| 56 | fetcher.unpack(rootdir) | ||
| 57 | </literallayout> | ||
| 58 | This code unpacks the downloaded files to the | ||
| 59 | specified by <filename>WORKDIR</filename>. | ||
| 60 | <note> | ||
| 61 | For convenience, the naming in these examples matches | ||
| 62 | the variables used by OpenEmbedded. | ||
| 63 | </note> | ||
| 64 | The <filename>SRC_URI</filename> and <filename>WORKDIR</filename> | ||
| 65 | variables are not coded into the fetcher. | ||
| 66 | They variables can (and are) called with different variable names. | ||
| 67 | In OpenEmbedded for example, the shared state (sstate) code uses | ||
| 68 | the fetch module to fetch the sstate files. | ||
| 69 | </para> | ||
| 70 | |||
| 71 | <para> | ||
| 72 | When the <filename>download()</filename> method is called, | ||
| 73 | BitBake tries to fulfill the URLs by looking for source files | ||
| 74 | in a specific search order: | ||
| 75 | <itemizedlist> | ||
| 76 | <listitem><para><emphasis>Pre-mirror Sites:</emphasis> | ||
| 77 | BitBake first uses pre-mirrors to try and find source files. | ||
| 78 | These locations are defined using the | ||
| 79 | <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link> | ||
| 80 | variable. | ||
| 81 | </para></listitem> | ||
| 82 | <listitem><para><emphasis>Source URI:</emphasis> | ||
| 83 | If pre-mirrors fail, BitBake uses the original URL (e.g from | ||
| 84 | <filename>SRC_URI</filename>). | ||
| 85 | </para></listitem> | ||
| 86 | <listitem><para><emphasis>Mirror Sites:</emphasis> | ||
| 87 | If fetch failures occur, BitBake next uses mirror location as | ||
| 88 | defined by the | ||
| 89 | <link linkend='var-MIRRORS'><filename>MIRRORS</filename></link> | ||
| 90 | variable. | ||
| 91 | </para></listitem> | ||
| 92 | </itemizedlist> | ||
| 93 | </para> | ||
| 94 | |||
| 95 | <para> | ||
| 96 | For each URL passed to the fetcher, the fetcher | ||
| 97 | calls the submodule that handles that particular URL type. | ||
| 98 | This behavior can be the source of some confusion when you | ||
| 99 | are providing URLs for the <filename>SRC_URI</filename> | ||
| 100 | variable. | ||
| 101 | Consider the following two URLs: | ||
| 102 | <literallayout class='monospaced'> | ||
| 103 | http://git.yoctoproject.org/git/poky;protocol=git | ||
| 104 | git://git.yoctoproject.org/git/poky;protocol=http | ||
| 105 | </literallayout> | ||
| 106 | In the former case, the URL is passed to the | ||
| 107 | <filename>wget</filename> fetcher, which does not | ||
| 108 | understand "git". | ||
| 109 | Therefore, the latter case is the correct form since the | ||
| 110 | Git fetcher does know how to use HTTP as a transport. | ||
| 111 | </para> | ||
| 112 | |||
| 113 | <para> | ||
| 114 | Here are some examples that show commonly used mirror | ||
| 115 | definitions: | ||
| 116 | <literallayout class='monospaced'> | ||
| 117 | PREMIRRORS ?= "\ | ||
| 118 | bzr://.*/.* http://somemirror.org/sources/ \n \ | ||
| 119 | cvs://.*/.* http://somemirror.org/sources/ \n \ | ||
| 120 | git://.*/.* http://somemirror.org/sources/ \n \ | ||
| 121 | hg://.*/.* http://somemirror.org/sources/ \n \ | ||
| 122 | osc://.*/.* http://somemirror.org/sources/ \n \ | ||
| 123 | p4://.*/.* http://somemirror.org/sources/ \n \ | ||
| 124 | svn://.*/.* http://somemirror.org/sources/ \n" | ||
| 125 | |||
| 126 | MIRRORS =+ "\ | ||
| 127 | ftp://.*/.* http://somemirror.org/sources/ \n \ | ||
| 128 | http://.*/.* http://somemirror.org/sources/ \n \ | ||
| 129 | https://.*/.* http://somemirror.org/sources/ \n" | ||
| 130 | </literallayout> | ||
| 131 | It is useful to note that BitBake supports | ||
| 132 | cross-URLs. | ||
| 133 | It is possible to mirror a Git repository on an HTTP | ||
| 134 | server as a tarball. | ||
| 135 | This is what the <filename>git://</filename> mapping in | ||
| 136 | the previous example does. | ||
| 137 | </para> | ||
| 138 | |||
| 139 | <para> | ||
| 140 | Since network accesses are slow, Bitbake maintains a | ||
| 141 | cache of files downloaded from the network. | ||
| 142 | Any source files that are not local (i.e. | ||
| 143 | downloaded from the Internet) are placed into the download | ||
| 144 | directory, which is specified by the | ||
| 145 | <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> | ||
| 146 | variable. | ||
| 147 | </para> | ||
| 148 | |||
| 149 | <para> | ||
| 150 | File integrity is of key importance for reproducing builds. | ||
| 151 | For non-local archive downloads, the fetcher code can verify | ||
| 152 | sha256 and md5 checksums to ensure the archives have been | ||
| 153 | downloaded correctly. | ||
| 154 | You can specify these checksums by using the | ||
| 155 | <filename>SRC_URI</filename> variable with the appropriate | ||
| 156 | varflags as follows: | ||
| 157 | <literallayout class='monospaced'> | ||
| 158 | SRC_URI[md5sum] = "value" | ||
| 159 | SRC_URI[sha256sum] = "value" | ||
| 160 | </literallayout> | ||
| 161 | You can also specify the checksums as parameters on the | ||
| 162 | <filename>SRC_URI</filename> as shown below: | ||
| 163 | <literallayout class='monospaced'> | ||
| 164 | SRC_URI = "http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d" | ||
| 165 | </literallayout> | ||
| 166 | If multiple URIs exist, you can specify the checksums either | ||
| 167 | directly as in the previous example, or you can name the URLs. | ||
| 168 | The following syntax shows how you name the URIs: | ||
| 169 | <literallayout class='monospaced'> | ||
| 170 | SRC_URI = "http://example.com/foobar.tar.bz2;name=foo" | ||
| 171 | SRC_URI[foo.md5sum] = 4a8e0f237e961fd7785d19d07fdb994d | ||
| 172 | </literallayout> | ||
| 173 | After a file has been downloaded and has had its checksum checked, | ||
| 174 | a ".done" stamp is placed in <filename>DL_DIR</filename>. | ||
| 175 | BitBake uses this stamp during subsequent builds to avoid | ||
| 176 | downloading or comparing a checksum for the file again. | ||
| 177 | <note> | ||
| 178 | It is assumed that local storage is safe from data corruption. | ||
| 179 | If this were not the case, there would be bigger issues to worry about. | ||
| 180 | </note> | ||
| 181 | </para> | ||
| 182 | |||
| 183 | <para> | ||
| 184 | If | ||
| 185 | <link linkend='var-BB_STRICT_CHECKSUM'><filename>BB_STRICT_CHECKSUM</filename></link> | ||
| 186 | is set, any download without a checksum triggers an | ||
| 187 | error message. | ||
| 188 | The | ||
| 189 | <link linkend='var-BB_NO_NETWORK'><filename>BB_NO_NETWORK</filename></link> | ||
| 190 | variable can be used to make any attempted network access a fatal | ||
| 191 | error, which is useful for checking that mirrors are complete | ||
| 192 | as well as other things. | ||
| 193 | </para> | ||
| 194 | </section> | ||
| 195 | |||
| 196 | <section id='bb-the-unpack'> | ||
| 197 | <title>The Unpack</title> | ||
| 198 | |||
| 199 | <para> | ||
| 200 | The unpack process usually immediately follows the download. | ||
| 201 | For all URLs except Git URLs, BitBake uses the common | ||
| 202 | <filename>unpack</filename> method. | ||
| 203 | </para> | ||
| 204 | |||
| 205 | <para> | ||
| 206 | A number of parameters exist that you can specify within the | ||
| 207 | URL to govern the behavior of the unpack stage: | ||
| 208 | <itemizedlist> | ||
| 209 | <listitem><para><emphasis>unpack:</emphasis> | ||
| 210 | Controls whether the URL components are unpacked. | ||
| 211 | If set to "1", which is the default, the components | ||
| 212 | are unpacked. | ||
| 213 | If set to "0", the unpack stage leaves the file alone. | ||
| 214 | This parameter is useful when you want an archive to be | ||
| 215 | copied in and not be unpacked. | ||
| 216 | </para></listitem> | ||
| 217 | <listitem><para><emphasis>dos:</emphasis> | ||
| 218 | Applies to <filename>.zip</filename> and | ||
| 219 | <filename>.jar</filename> files and specifies whether to | ||
| 220 | use DOS line ending conversion on text files. | ||
| 221 | </para></listitem> | ||
| 222 | <listitem><para><emphasis>basepath:</emphasis> | ||
| 223 | Instructs the unpack stage to strip the specified | ||
| 224 | directories from the source path when unpacking. | ||
| 225 | </para></listitem> | ||
| 226 | <listitem><para><emphasis>subdir:</emphasis> | ||
| 227 | Unpacks the specific URL to the specified subdirectory | ||
| 228 | within the root directory. | ||
| 229 | </para></listitem> | ||
| 230 | </itemizedlist> | ||
| 231 | The unpack call automatically decompresses and extracts files | ||
| 232 | with ".Z", ".z", ".gz", ".xz", ".zip", ".jar", ".ipk", ".rpm". | ||
| 233 | ".srpm", ".deb" and ".bz2" extensions as well as various combinations | ||
| 234 | of tarball extensions. | ||
| 235 | </para> | ||
| 236 | |||
| 237 | <para> | ||
| 238 | As mentioned, the Git fetcher has its own unpack method that | ||
| 239 | is optimized to work with Git trees. | ||
| 240 | Basically, this method works by cloning the tree into the final | ||
| 241 | directory. | ||
| 242 | The process is completed using references so that there is | ||
| 243 | only one central copy of the Git metadata needed. | ||
| 244 | </para> | ||
| 245 | </section> | ||
| 246 | |||
| 247 | <section id='bb-fetchers'> | ||
| 248 | <title>Fetchers</title> | ||
| 249 | |||
| 250 | <para> | ||
| 251 | As mentioned earlier, the URL prefix determines which | ||
| 252 | fetcher submodule BitBake uses. | ||
| 253 | Each submodule can support different URL parameters, | ||
| 254 | which are described in the following sections. | ||
| 255 | </para> | ||
| 256 | |||
| 257 | <section id='local-file-fetcher'> | ||
| 258 | <title>Local file fetcher (<filename>file://</filename>)</title> | ||
| 259 | |||
| 260 | <para> | ||
| 261 | This submodule handles URLs that begin with | ||
| 262 | <filename>file://</filename>. | ||
| 263 | The filename you specify with in the URL can | ||
| 264 | either be an absolute or relative path to a file. | ||
| 265 | If the filename is relative, the contents of the | ||
| 266 | <link linkend='var-FILESPATH'><filename>FILESPATH</filename></link> | ||
| 267 | variable is used in the same way | ||
| 268 | <filename>PATH</filename> is used to find executables. | ||
| 269 | Failing that, | ||
| 270 | <link linkend='var-FILESDIR'><filename>FILESDIR</filename></link> | ||
| 271 | is used to find the appropriate relative file. | ||
| 272 | <note> | ||
| 273 | <filename>FILESDIR</filename> is deprecated and can | ||
| 274 | be replaced with <filename>FILESPATH</filename>. | ||
| 275 | Because <filename>FILESDIR</filename> is likely to be | ||
| 276 | removed, you should not use this variable in any new code. | ||
| 277 | </note> | ||
| 278 | If the file cannot be found, it is assumed that it is available in | ||
| 279 | <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> | ||
| 280 | by the time the <filename>download()</filename> method is called. | ||
| 281 | </para> | ||
| 282 | |||
| 283 | <para> | ||
| 284 | If you specify a directory, the entire directory is | ||
| 285 | unpacked. | ||
| 286 | </para> | ||
| 287 | |||
| 288 | <para> | ||
| 289 | Here are some example URLs: | ||
| 290 | <literallayout class='monospaced'> | ||
| 291 | SRC_URI = "file://relativefile.patch" | ||
| 292 | SRC_URI = "file://relativefile.patch;this=ignored" | ||
| 293 | SRC_URI = "file:///Users/ich/very_important_software" | ||
| 294 | </literallayout> | ||
| 295 | </para> | ||
| 296 | </section> | ||
| 297 | |||
| 298 | <section id='cvs-fetcher'> | ||
| 299 | <title>CVS fetcher (<filename>(cvs://</filename>)</title> | ||
| 300 | |||
| 301 | <para> | ||
| 302 | This submodule handles checking out files from the | ||
| 303 | CVS version control system. | ||
| 304 | You can configure it using a number of different variables: | ||
| 305 | <itemizedlist> | ||
| 306 | <listitem><para><emphasis><filename>FETCHCMD_cvs</filename>:</emphasis> | ||
| 307 | The name of the executable to use when running | ||
| 308 | the <filename>cvs</filename> command. | ||
| 309 | This name is usually "cvs". | ||
| 310 | </para></listitem> | ||
| 311 | <listitem><para><emphasis><filename>SRCDATE</filename>:</emphasis> | ||
| 312 | The date to use when fetching the CVS source code. | ||
| 313 | A special value of "now" causes the checkout to | ||
| 314 | be updated on every build. | ||
| 315 | </para></listitem> | ||
| 316 | <listitem><para><emphasis><filename>CVSDIR</filename>:</emphasis> | ||
| 317 | Specifies where a temporary checkout is saved. | ||
| 318 | The location is often <filename>DL_DIR/cvs</filename>. | ||
| 319 | </para></listitem> | ||
| 320 | <listitem><para><emphasis><filename>CVS_PROXY_HOST</filename>:</emphasis> | ||
| 321 | The name to use as a "proxy=" parameter to the | ||
| 322 | <filename>cvs</filename> command. | ||
| 323 | </para></listitem> | ||
| 324 | <listitem><para><emphasis><filename>CVS_PROXY_PORT</filename>:</emphasis> | ||
| 325 | The port number to use as a "proxyport=" parameter to | ||
| 326 | the <filename>cvs</filename> command. | ||
| 327 | </para></listitem> | ||
| 328 | </itemizedlist> | ||
| 329 | As well as the standard username and password URL syntax, | ||
| 330 | you can also configure the fetcher with various URL parameters: | ||
| 331 | </para> | ||
| 332 | |||
| 333 | <para> | ||
| 334 | The supported parameters are as follows: | ||
| 335 | <itemizedlist> | ||
| 336 | <listitem><para><emphasis>"method":</emphasis> | ||
| 337 | The protocol over which to communicate with the cvs server. | ||
| 338 | By default, this protocol is "pserver". | ||
| 339 | If "method" is set to "ext", BitBake examines the | ||
| 340 | "rsh" parameter and sets <filename>CVS_RSH</filename>. | ||
| 341 | You can use "dir" for local directories. | ||
| 342 | </para></listitem> | ||
| 343 | <listitem><para><emphasis>"module":</emphasis> | ||
| 344 | Specifies the module to check out. | ||
| 345 | You must supply this parameter. | ||
| 346 | </para></listitem> | ||
| 347 | <listitem><para><emphasis>"tag":</emphasis> | ||
| 348 | Describes which CVS TAG should be used for | ||
| 349 | the checkout. | ||
| 350 | By default, the TAG is empty. | ||
| 351 | </para></listitem> | ||
| 352 | <listitem><para><emphasis>"date":</emphasis> | ||
| 353 | Specifies a date. | ||
| 354 | If no "date" is specified, the | ||
| 355 | <link linkend='var-SRCDATE'><filename>SRCDATE</filename></link> | ||
| 356 | of the configuration is used to checkout a specific date. | ||
| 357 | The special value of "now" causes the checkout to be | ||
| 358 | updated on every build. | ||
| 359 | </para></listitem> | ||
| 360 | <listitem><para><emphasis>"localdir":</emphasis> | ||
| 361 | Used to rename the module. | ||
| 362 | Effectively, you are renaming the output directory | ||
| 363 | to which the module is unpacked. | ||
| 364 | You are forcing the module into a special | ||
| 365 | directory relative to <filename>CVSDIR</filename>. | ||
| 366 | </para></listitem> | ||
| 367 | <listitem><para><emphasis>"rsh"</emphasis> | ||
| 368 | Used in conjunction with the "method" parameter. | ||
| 369 | </para></listitem> | ||
| 370 | <listitem><para><emphasis>"scmdata":</emphasis> | ||
| 371 | Causes the CVS metadata to be maintained in the tarball | ||
| 372 | the fetcher creates when set to "keep". | ||
| 373 | The tarball is expanded into the work directory. | ||
| 374 | By default, the CVS metadata is removed. | ||
| 375 | </para></listitem> | ||
| 376 | <listitem><para><emphasis>"fullpath":</emphasis> | ||
| 377 | Controls whether the resulting checkout is at the | ||
| 378 | module level, which is the default, or is at deeper | ||
| 379 | paths. | ||
| 380 | </para></listitem> | ||
| 381 | <listitem><para><emphasis>"norecurse":</emphasis> | ||
| 382 | Causes the fetcher to only checkout the specified | ||
| 383 | directory with no recurse into any subdirectories. | ||
| 384 | </para></listitem> | ||
| 385 | <listitem><para><emphasis>"port":</emphasis> | ||
| 386 | The port to which the CVS server connects. | ||
| 387 | </para></listitem> | ||
| 388 | </itemizedlist> | ||
| 389 | Some example URLs are as follows: | ||
| 390 | <literallayout class='monospaced'> | ||
| 391 | SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext" | ||
| 392 | SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat" | ||
| 393 | </literallayout> | ||
| 394 | </para> | ||
| 395 | </section> | ||
| 396 | |||
| 397 | <section id='http-ftp-fetcher'> | ||
| 398 | <title>HTTP/FTP wget fetcher (<filename>http://</filename>, <filename>ftp://</filename>, <filename>https://</filename>)</title> | ||
| 399 | |||
| 400 | <para> | ||
| 401 | This fetcher obtains files from web and FTP servers. | ||
| 402 | Internally, the fetcher uses the wget utility. | ||
| 403 | </para> | ||
| 404 | |||
| 405 | <para> | ||
| 406 | The executable and parameters used are specified by the | ||
| 407 | <filename>FETCHCMD_wget</filename> variable, which defaults | ||
| 408 | to a sensible values. | ||
| 409 | The fetcher supports a parameter "downloadfilename" that | ||
| 410 | allows the name of the downloaded file to be specified. | ||
| 411 | Specifying the name of the downloaded file is useful | ||
| 412 | for avoiding collisions in | ||
| 413 | <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> | ||
| 414 | when dealing with multiple files that have the same name. | ||
| 415 | </para> | ||
| 416 | |||
| 417 | <para> | ||
| 418 | Some example URLs are as follows: | ||
| 419 | <literallayout class='monospaced'> | ||
| 420 | SRC_URI = "http://oe.handhelds.org/not_there.aac" | ||
| 421 | SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac" | ||
| 422 | SRC_URI = "ftp://you@oe.handheld.sorg/home/you/secret.plan" | ||
| 423 | </literallayout> | ||
| 424 | </para> | ||
| 425 | </section> | ||
| 426 | |||
| 427 | <section id='svn-fetcher'> | ||
| 428 | <title>Subversion (SVN) Fetcher (<filename>svn://</filename>)</title> | ||
| 429 | |||
| 430 | <para> | ||
| 431 | This fetcher submodule fetches code from the | ||
| 432 | Subversion source control system. | ||
| 433 | The executable used is specified by | ||
| 434 | <filename>FETCHCMD_svn</filename>, which defaults | ||
| 435 | to "svn". | ||
| 436 | The fetcher's temporary working directory is set | ||
| 437 | by <filename>SVNDIR</filename>, which is usually | ||
| 438 | <filename>DL_DIR/svn</filename>. | ||
| 439 | </para> | ||
| 440 | |||
| 441 | <para> | ||
| 442 | The supported parameters are as follows: | ||
| 443 | <itemizedlist> | ||
| 444 | <listitem><para><emphasis>"module":</emphasis> | ||
| 445 | The name of the svn module to checkout. | ||
| 446 | You must provide this parameter. | ||
| 447 | You can think of this parameter as the top-level | ||
| 448 | directory of the repository data you want. | ||
| 449 | </para></listitem> | ||
| 450 | <listitem><para><emphasis>"protocol":</emphasis> | ||
| 451 | The protocol to use, which defaults to "svn". | ||
| 452 | Other options are "svn+ssh" and "rsh". | ||
| 453 | For "rsh", the "rsh" parameter is also used. | ||
| 454 | </para></listitem> | ||
| 455 | <listitem><para><emphasis>"rev":</emphasis> | ||
| 456 | The revision of the source code to checkout. | ||
| 457 | </para></listitem> | ||
| 458 | <listitem><para><emphasis>"date":</emphasis> | ||
| 459 | The date of the source code to checkout. | ||
| 460 | Specific revisions are generally much safer to checkout | ||
| 461 | rather than by date as they do not involve timezones | ||
| 462 | (e.g. they are much more deterministic). | ||
| 463 | </para></listitem> | ||
| 464 | <listitem><para><emphasis>"scmdata":</emphasis> | ||
| 465 | Causes the “.svn” directories to be available during | ||
| 466 | compile-time when set to "keep". | ||
| 467 | By default, these directories are removed. | ||
| 468 | </para></listitem> | ||
| 469 | </itemizedlist> | ||
| 470 | Following are two examples using svn: | ||
| 471 | <literallayout class='monospaced'> | ||
| 472 | SRC_URI = "svn://svn.oe.handhelds.org/svn;module=vip;proto=http;rev=667" | ||
| 473 | SRC_URI = "svn://svn.oe.handhelds.org/svn/;module=opie;proto=svn+ssh;date=20060126" | ||
| 474 | </literallayout> | ||
| 475 | </para> | ||
| 476 | </section> | ||
| 477 | |||
| 478 | <section id='git-fetcher'> | ||
| 479 | <title>GIT Fetcher (<filename>git://</filename>)</title> | ||
| 480 | |||
| 481 | <para> | ||
| 482 | This fetcher submodule fetches code from the Git | ||
| 483 | source control system. | ||
| 484 | The fetcher works by creating a bare clone of the | ||
| 485 | remote into <filename>GITDIR</filename>, which is | ||
| 486 | usually <filename>DL_DIR/git</filename>. | ||
| 487 | This bare clone is then cloned into the work directory during the | ||
| 488 | unpack stage when a specific tree is checked out. | ||
| 489 | This is done using alternates and by reference to | ||
| 490 | minimize the amount of duplicate data on the disk and | ||
| 491 | make the unpack process fast. | ||
| 492 | The executable used can be set with | ||
| 493 | <filename>FETCHCMD_git</filename>. | ||
| 494 | </para> | ||
| 495 | |||
| 496 | <para> | ||
| 497 | This fetcher supports the following parameters: | ||
| 498 | <itemizedlist> | ||
| 499 | <listitem><para><emphasis>"protocol":</emphasis> | ||
| 500 | The protocol used to fetch the files. | ||
| 501 | The default is "git" when a hostname is set. | ||
| 502 | If a hostname is not set, the Git protocol is "file". | ||
| 503 | You can also use "http", "https", "ssh" and "rsync". | ||
| 504 | </para></listitem> | ||
| 505 | <listitem><para><emphasis>"nocheckout":</emphasis> | ||
| 506 | Tells the fetcher to not checkout source code when | ||
| 507 | unpacking when set to "1". | ||
| 508 | Set this option for the URL where there is a custom | ||
| 509 | routine to checkout code. | ||
| 510 | The default is "0". | ||
| 511 | </para></listitem> | ||
| 512 | <listitem><para><emphasis>"rebaseable":</emphasis> | ||
| 513 | Indicates that the upstream Git repository can be rebased. | ||
| 514 | You should set this parameter to "1" if | ||
| 515 | revisions can become detached from branches. | ||
| 516 | In this case, the source mirror tarball is done per | ||
| 517 | revision, which has a loss of efficiency. | ||
| 518 | Rebasing the upstream Git repository could cause the | ||
| 519 | current revision to disappear from the upstream repository. | ||
| 520 | This option reminds the fetcher to preserve the local cache | ||
| 521 | carefully for future use. | ||
| 522 | The default value for this parameter is "0". | ||
| 523 | </para></listitem> | ||
| 524 | <listitem><para><emphasis>"nobranch":</emphasis> | ||
| 525 | Tells the fetcher to not check the SHA validation | ||
| 526 | for the branch when set to "1". | ||
| 527 | The default is "0". | ||
| 528 | Set this option for the recipe that refers to | ||
| 529 | the commit that is valid for a tag instead of | ||
| 530 | the branch. | ||
| 531 | </para></listitem> | ||
| 532 | <listitem><para><emphasis>"bareclone":</emphasis> | ||
| 533 | Tells the fetcher to clone a bare clone into the | ||
| 534 | destination directory without checking out a working tree. | ||
| 535 | Only the raw Git metadata is provided. | ||
| 536 | This parameter implies the "nocheckout" parameter as well. | ||
| 537 | </para></listitem> | ||
| 538 | <listitem><para><emphasis>"branch":</emphasis> | ||
| 539 | The branch(es) of the Git tree to clone. | ||
| 540 | If unset, this is assumed to be "master". | ||
| 541 | The number of branch parameters much match the number of | ||
| 542 | name parameters. | ||
| 543 | </para></listitem> | ||
| 544 | <listitem><para><emphasis>"rev":</emphasis> | ||
| 545 | The revision to use for the checkout. | ||
| 546 | The default is "master". | ||
| 547 | </para></listitem> | ||
| 548 | <listitem><para><emphasis>"tag":</emphasis> | ||
| 549 | Specifies a tag to use for the checkout. | ||
| 550 | To correctly resolve tags, BitBake must access the | ||
| 551 | network. | ||
| 552 | For that reason, tags are often not used. | ||
| 553 | As far as Git is concerned, the "tag" parameter behaves | ||
| 554 | effectively the same as the "revision" parameter. | ||
| 555 | </para></listitem> | ||
| 556 | <listitem><para><emphasis>"subpath":</emphasis> | ||
| 557 | Limits the checkout to a specific subpath of the tree. | ||
| 558 | By default, the whole tree is checked out. | ||
| 559 | </para></listitem> | ||
| 560 | <listitem><para><emphasis>"destsuffix":</emphasis> | ||
| 561 | The name of the path in which to place the checkout. | ||
| 562 | By default, the path is <filename>git/</filename>. | ||
| 563 | </para></listitem> | ||
| 564 | </itemizedlist> | ||
| 565 | Here are some example URLs: | ||
| 566 | <literallayout class='monospaced'> | ||
| 567 | SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1" | ||
| 568 | SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http" | ||
| 569 | </literallayout> | ||
| 570 | </para> | ||
| 571 | </section> | ||
| 572 | |||
| 573 | <section id='other-fetchers'> | ||
| 574 | <title>Other Fetchers</title> | ||
| 575 | |||
| 576 | <para> | ||
| 577 | Fetch submodules also exist for the following: | ||
| 578 | <itemizedlist> | ||
| 579 | <listitem><para> | ||
| 580 | Bazaar (<filename>bzr://</filename>) | ||
| 581 | </para></listitem> | ||
| 582 | <listitem><para> | ||
| 583 | Perforce (<filename>p4://</filename>) | ||
| 584 | </para></listitem> | ||
| 585 | <listitem><para> | ||
| 586 | Git Submodules (<filename>gitsm://</filename>) | ||
| 587 | </para></listitem> | ||
| 588 | <listitem><para> | ||
| 589 | Trees using Git Annex (<filename>gitannex://</filename>) | ||
| 590 | </para></listitem> | ||
| 591 | <listitem><para> | ||
| 592 | Secure FTP (<filename>sftp://</filename>) | ||
| 593 | </para></listitem> | ||
| 594 | <listitem><para> | ||
| 595 | Secure Shell (<filename>ssh://</filename>) | ||
| 596 | </para></listitem> | ||
| 597 | <listitem><para> | ||
| 598 | Repo (<filename>repo://</filename>) | ||
| 599 | </para></listitem> | ||
| 600 | <listitem><para> | ||
| 601 | OSC (<filename>osc://</filename>) | ||
| 602 | </para></listitem> | ||
| 603 | <listitem><para> | ||
| 604 | Mercurial (<filename>hg://</filename>) | ||
| 605 | </para></listitem> | ||
| 606 | </itemizedlist> | ||
| 607 | No documentation currently exists for these lesser used | ||
| 608 | fetcher submodules. | ||
| 609 | However, you might find the code helpful and readable. | ||
| 610 | </para> | ||
| 611 | </section> | ||
| 612 | </section> | ||
| 613 | |||
| 614 | <section id='auto-revisions'> | ||
| 615 | <title>Auto Revisions</title> | ||
| 616 | |||
| 617 | <para> | ||
| 618 | We need to document <filename>AUTOREV</filename> and | ||
| 619 | <filename>SRCREV_FORMAT</filename> here. | ||
| 620 | </para> | ||
| 621 | </section> | ||
| 622 | </chapter> | ||
