From f3caa2b27e7ac4edf8e6e4c196c07f3fbe9e3e27 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Tue, 11 Feb 2014 20:00:02 -0600 Subject: bitbake: user-manual-fetching.xml: Re-write of "File Download Support" chapter. Basic re-write to clean up text and flow. (Bitbake rev: 0f82dc4c22ce015bef87fd5aae0b6fa3e429016e) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- bitbake/doc/user-manual/user-manual-fetching.xml | 432 ++++++++++++++--------- 1 file changed, 271 insertions(+), 161 deletions(-) (limited to 'bitbake/doc') diff --git a/bitbake/doc/user-manual/user-manual-fetching.xml b/bitbake/doc/user-manual/user-manual-fetching.xml index cdca64c801..499d4f5198 100644 --- a/bitbake/doc/user-manual/user-manual-fetching.xml +++ b/bitbake/doc/user-manual/user-manual-fetching.xml @@ -2,84 +2,110 @@ "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> -File download support +File Download Support + + + BitBake's fetch and + fetch2 modules support downloading + files. + This chapter provides an overview of the fetching process + and also presents sections on each of the fetchers BitBake + supports. + + The original fetch code, for all + practical purposes, has been replaced by + fetch2 code. + Consequently, the information in this chapter does not + apply to fetch. + +
Overview - BitBake provides support to download files. - This procedure is called fetching and is handled by the - fetch and fetch2 modules. - At this point, the original fetch code is considered to - be replaced by fetch2 and this manual is only related - to the fetch2 codebase. + When BitBake starts to execute, the very first thing + it does is to fetch the source files needed. + This section overviews the process. + For some additional information on fetching source files, see the + "Source Fetching" + section. - The SRC_URI is normally used to - tell BitBake which files to fetch. - The next sections describe the available fetchers and - their options. - Each fetcher honors a set of variables and per - URI parameters separated by a “;” consisting of a key and - a value. - The semantics of the variables and parameters are - defined by the fetcher. - BitBake tries to have consistent semantics between the - different fetchers. + When BitBake goes looking for source files, it follows a search + order: + + Pre-mirror Sites: + BitBake first uses pre-mirrors to try and find source + files. + These locations are defined using the + PREMIRRORS + variable. + + Source URI: + If pre-mirrors fail, BitBake uses + SRC_URI. + + Mirror Sites: + If fetch failures occur using SRC_URI, + BitBake next uses mirror location as defined by the + MIRRORS + variable. + + - The overall fetch process first attempts to fetch from - PREMIRRORS. - If these fail, the original SRC_URI - is attempted. - If that fails, BitBake falls back to - MIRRORS. Because cross-URLs are supported, it is possible to mirror a Git repository on an HTTP server as a tarball. Here are some examples that show commonly used mirror definitions: -PREMIRRORS ?= "\ - bzr://.*/.* http://somemirror.org/sources/ \n \ - cvs://.*/.* http://somemirror.org/sources/ \n \ - git://.*/.* http://somemirror.org/sources/ \n \ - hg://.*/.* http://somemirror.org/sources/ \n \ - osc://.*/.* http://somemirror.org/sources/ \n \ - p4://.*/.* http://somemirror.org/sources/ \n \ - svk://.*/.* http://somemirror.org/sources/ \n \ - svn://.*/.* http://somemirror.org/sources/ \n" - -MIRRORS =+ "\ - ftp://.*/.* http://somemirror.org/sources/ \n \ - http://.*/.* http://somemirror.org/sources/ \n \ - https://.*/.* http://somemirror.org/sources/ \n" + PREMIRRORS ?= "\ + bzr://.*/.* http://somemirror.org/sources/ \n \ + cvs://.*/.* http://somemirror.org/sources/ \n \ + git://.*/.* http://somemirror.org/sources/ \n \ + hg://.*/.* http://somemirror.org/sources/ \n \ + osc://.*/.* http://somemirror.org/sources/ \n \ + p4://.*/.* http://somemirror.org/sources/ \n \ + svk://.*/.* http://somemirror.org/sources/ \n \ + svn://.*/.* http://somemirror.org/sources/ \n" + + MIRRORS =+ "\ + ftp://.*/.* http://somemirror.org/sources/ \n \ + http://.*/.* http://somemirror.org/sources/ \n \ + https://.*/.* http://somemirror.org/sources/ \n" - Non-local downloaded output is placed - into the directory specified by the - DL_DIR variable. - For non local archive downloads, the code can verify - sha256 and md5 checksums for the download to ensure - the file has been downloaded correctly. - These can be specified in the following forms - for md5 and sha256 checksums, respectively: + Any source files that are not local (i.e. downloaded from + the Internet) are placed into the download directory, + which is specified by + DL_DIR. + + + + For non-local archive downloads, the fetcher code can verify + sha256 and md5 checksums to ensure + the archives have been downloaded correctly. + You can specify these checksums by using the + SRC_URI variable with the appropriate + varflags as follows: SRC_URI[md5sum] SRC_URI[sha256sum] - You can also specify them as parameters on the - SRC_URI: + You can also specify the checksums as parameters on the + SRC_URI as shown below: SRC_URI="http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d" - If BB_STRICT_CHECKSUM is set, any download - without a checksum will trigger an error message. - In cases where multiple files are listed in + If + BB_STRICT_CHECKSUM + is set, any download without a checksum triggers an error message. + In cases where multiple files are listed using SRC_URI, the name parameter is used assign names to the URLs and these are then specified in the checksums using the following form: @@ -89,142 +115,226 @@ MIRRORS =+ "\
-
- Local file fetcher +
+ Fetchers - The URN for the local file fetcher is file. - The filename can be either absolute or relative. - If the filename is relative, - FILESPATH and failing that - FILESDIR will be used to find the - appropriate relative file. - The metadata usually extend these variables to include - variations of the values in OVERRIDES. - Single files and complete directories can be specified. - + As mentioned in the previous section, the + SRC_URI is normally used to + tell BitBake which files to fetch. + And, the fetcher BitBake uses depends on the how + SRC_URI is set. + + + + These next few sections describe the available fetchers and + their options. + Each fetcher honors a set of variables URI parameters, + which are separated by semi-colon characters and consist + of a key and a value. + The semantics of the variables and parameters are + defined by the fetcher. + BitBake tries to have consistent semantics between the + different fetchers. + + +
+ Local file fetcher + + + The URN for the local file fetcher is file. + + + + The filename can be either absolute or relative. + If the filename is relative, + FILESPATH + is used. + Failing that, + FILESDIR + is used to find the appropriate relative file. + + + + The metadata usually extend these variables to include + variations of the values in + OVERRIDES. + Single files and complete directories can be specified. + SRC_URI = "file://relativefile.patch" SRC_URI = "file://relativefile.patch;this=ignored" SRC_URI = "file:///Users/ich/very_important_software" - - -
+ + +
-
- CVS fetcher +
+ CVS fetcher - - The URN for the CVS fetcher is cvs. - This fetcher honors the variables CVSDIR, - SRCDATE, FETCHCOMMAND_cvs, - UPDATECOMMAND_cvs. - DL_DIR specifies where a - temporary checkout is saved. - SRCDATE specifies which date to - use when doing the fetching (the special value of "now" - will cause the checkout to be updated on every build). - FETCHCOMMAND and - UPDATECOMMAND specify which executables - to use for the CVS checkout or update. - + + The URN for the CVS fetcher is cvs. + - - The supported parameters are module, tag, date, - method, localdir, rshand scmdata. - The module specifies which module to check out, - the tag describes which CVS TAG should be used for - the checkout. - By default, the TAG is empty. - A date can be specified to override the - SRCDATE of the - configuration to checkout a specific date. - The special value of "now" will cause the checkout to be - updated on every build. - method is by default pserver. - If ext is used the rsh parameter will be evaluated - and CVS_RSH will be set. - Finally, localdir is used to checkout into a special - directory relative to CVSDIR. - + + This fetcher honors the variables CVSDIR, + SRCDATE, FETCHCOMMAND_cvs, + UPDATECOMMAND_cvs. + The + DL_DIR + variable specifies where a + temporary checkout is saved. + The + SRCDATE + variable specifies which date to + use when doing the fetching. + The special value of "now" causes the checkout to be + updated on every build. + The FETCHCOMMAND and + UPDATECOMMAND variables specify the executables + to use for the CVS checkout or update. + + + + The supported parameters are as follows: + + "module": + Specifies the module to check out. + + "tag": + Describes which CVS TAG should be used for + the checkout. + By default, the TAG is empty. + + "date": + Specifies a date. + If no "date" is specified, the + SRCDATE + of the configuration is used to checkout a specific date. + The special value of "now" causes the checkout to be + updated on every build. + + "method": + By default pserver. + If "method" is set to "ext", BitBake examines the "rsh" + parameter and sets CVS_RSH. + + "localdir": + Used to checkout force into a special + directory relative to CVSDIR. + + "rsh" + Used in conjunction with the "method" parameter. + + "scmdata": + I need a description for this. + + + Following are two examples using cvs: + SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext" SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat" - - -
+ + +
-
- HTTP/FTP fetcher +
+ HTTP/FTP fetcher - - The URNs for the HTTP/FTP fetcher are http, https, and ftp. - This fetcher honors the variables - FETCHCOMMAND_wget. - FETCHCOMMAND contains the command used - for fetching. - “${URI}” and “${FILES}” will be replaced by the URI and - basename of the file to be fetched. - + + The URNs for the HTTP/FTP fetcher are http, https, and ftp. + + + + This fetcher honors the variables + FETCHCOMMAND_wget. + The FETCHCOMMAND variable + contains the command used for fetching. + “${URI}” and “${FILES}” are replaced by the URI and + the base name of the file to be fetched. + SRC_URI = "http://oe.handhelds.org/not_there.aac" SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac" SRC_URI = "ftp://you@oe.handheld.sorg/home/you/secret.plan" - - -
+ + +
-
- SVN Fetcher +
+ SVN Fetcher - - The URN for the SVN fetcher is svn. - + + The URN for the SVN fetcher is svn. + - - This fetcher honors the variables - FETCHCOMMAND_svn, - SVNDIR, - and SRCREV. - FETCHCOMMAND contains the - subversion command. - SRCREV specifies which revision - to use when doing the fetching. - + + This fetcher honors the variables + FETCHCOMMAND_svn, + SVNDIR, + and + SRCREV. + The FETCHCOMMAND variable contains the + subversion command. + The SRCREV variable specifies which revision + to use when doing the fetching. + - - The supported parameters are proto, rev and scmdata. - proto is the Subversion protocol, rev is the - Subversion revision. - If scmdata is set to “keep”, the “.svn” directories will - be available during compile-time. - + + The supported parameters are as follows: + + "proto": + The Subversion protocol. + + "rev": + The Subversion revision. + + "scmdata": + Set to "keep" causes the “.svn” directories + to be available during compile-time. + + + Following are two examples using svn: + SRC_URI = "svn://svn.oe.handhelds.org/svn;module=vip;proto=http;rev=667" SRC_URI = "svn://svn.oe.handhelds.org/svn/;module=opie;proto=svn+ssh;date=20060126" - - -
+ + +
-
- GIT Fetcher +
+ GIT Fetcher - - The URN for the GIT Fetcher is git. - + + The URN for the Git Fetcher is git. + - - The variable GITDIR will be used as the - base directory where the Git tree is cloned to. - + + The variable GITDIR is used as the + base directory in which the Git tree is cloned. + - - The parameters are tag, protocol, and scmdata. - The tag parameter is a Git tag, the default is “master”. - The protocol tag is the Git protocol to use and defaults to “git” - if a hostname is set, otherwise it is “file”. - If scmdata is set to “keep”, the “.git” directory will be available - during compile-time. - + + The supported parameters are as follows: + + "tag": + The Git tag. + The default is "master". + + "protocol": + The Git protocol. + The default is "git" when a hostname is set. + If a hostname is not set, the Git protocol is "file". + + "scmdata": + When set to “keep”, the “.git” directory is available + during compile-time. + + + Following are two examples using git: + SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1" SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http" - - + + +
-- cgit v1.2.3-54-g00ecf