summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch2
Commit message (Collapse)AuthorAgeFilesLines
* fetcher2: retry mirror if upstream checksum mismatchYu Ke2011-07-131-11/+20
| | | | | | | | | | | | | | | This patch is for [YOCTO #1085] fix. If the upstream fails a checksum, retry from the MIRROR before giving up. This will add more robust fetching if an upstream serves a bad file or webpage. fetching of distcc prior to the move from samba -> googlecode is a good example of this. (Bitbake rev: b631e922257de52bf2247c01152d9856c870e7d0) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: Tweak git fetcher to handling repo updates correctlyRichard Purdie2011-07-081-2/+2
| | | | | | | | | | | | | | | | | | Currently the git fetcher can malfunction when branches change in remote repositories since whilst the update code updates the "origin" remote, this isn't linked to the local heads. By passing the --mirror option to 'git clone' and 'git remote add', linkage between the local heads and remote heads is created with a 1:1 mapping, hence all the appropriate heads are then updated correctly. This fixes some issues which have been seen with the Yocto autobuilder mirrors. (Bitbake rev: 3725602ec53df116dc108b3197a426b86ca43d5f) Signed-off-by: Richard Purdie <richard.purdie@linux-foundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git.py: improve error reporting when an invalid protocol is usedScott Garman2011-06-221-0/+3
| | | | | | | | | | | | | | | | | | | | When an invalid 'protocol' parameter is used in a git SRC_URI, the error reported was not helpful: ERROR: Function 'Fetcher failure for URL: 'None'. <environment dump> fatal: Could not make temporary directory: No such file or directory So instead check that ud.proto is set to something valid, and if not raise a meaningful ParameterError which explains that the protocol type is the source of the problem. This fixes bug [YOCTO #1142] (Bitbake rev: a2a29b72275ab03a263f4479a590b92111a0d6a8) Signed-off-by: Scott Garman <scott.a.garman@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* make exception handling syntax consistentScott Garman2011-06-151-2/+2
| | | | | | | | | | Update exception handling syntax to use the modern style: except ExcType as localvar (Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d) Signed-off-by: Scott Garman <scott.a.garman@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: When replacing URLs in mirror handling mask out empty entriesRichard Purdie2011-06-131-1/+4
| | | | | | | | | | | | | The symptom of this problem is something like a cvs url which specifies a username where the username is then passed through to something like an http mirror. This patch fixes things by ensuring empty entries are preserved in the new URL. (Bitbake rev: c1d978d7bd1ac8eb1e2d50029ab2384be9f72fb4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: ensure network check log matches actual commandDarren Hart2011-05-271-6/+10
| | | | | | | | | | The git command string logged via check_network_access() does not match the actual command executed in a few places. Ensure that it does. (Bitbake rev: 10f3ca52dc274cd8b240987cfd7cd003aeda7ab1) Signed-off-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: use logging.debug() and clarify messagesDarren Hart2011-05-271-2/+5
| | | | | | | | | | Replace a call to print() with logging.debug() and flesh out the message to clarify the state being reported. (Bitbake rev: 9a28f7744e2f4224e7c097b8c4c1d49731b9a47e) Signed-off-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: enhance the unpack by using "git checkout"Yu Ke2011-05-271-2/+5
| | | | | | | | | | | | | | | current git fetcher unpack method only checkout index and working tree, but did not did not update the git branch in ref/heads, so user may not get right info in ${S} by using git. this patch enhance the unpack by using git checkout to fix this issue. Fix bug [YOCTO #1089] (Bitbake rev: c0eb89054aef4957966f98b44e7f3cce14fb337a) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: add document for git fetcher supported optionsYu Ke2011-05-271-3/+35
| | | | | | | (Bitbake rev: d424ecd751f80a7aecec26e1a0cbd2a1b38e076b) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: unify the nocheckout option formatYu Ke2011-05-271-3/+1
| | | | | | | | | | | | | make the nocheckout option format to be: default is "0", use nocheckou=1 to set this option with this patch, the format will be consistant with other bitbake options like rebaseable (Bitbake rev: bd51659f5ee521cb8e6631d5f26792ab573e6b30) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: change default protocol from rsync to gitYu Ke2011-05-271-1/+1
| | | | | | | (Bitbake rev: e912122706b402f99848d50f721f179491644d0e) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* git fetcher: add support for rebaseable git repoYu Ke2011-05-171-8/+20
| | | | | | | | | | | | | | | | | | | | | | Some upstream git repo may rebase in the future, which means current revision may disappear from the upstream repo after the rebase. current git fetcher can not handle this case, because the git mirror tar ball is per repo, and may also change in the rebase and lost the current revision info. To fix this issue, this patch - add rebaseable tag in the SRC_URI - for rebaseable repo, make git mirror tar ball per revision, in this case, even upstream rebase, the git mirror still has the current revision info. - for rebaseable repo, generate mirror tar ball by default, since the repo may change in the future. (Bitbake rev: 92701d4c5372db48847c70da4ebd0736d79fd54b) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* persist_data: implement comparison, same as dictChris Larson2011-05-061-9/+6
| | | | | | | (Bitbake rev: 1190406c526c7bb7cf415867be83e0403812a7dd) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* persist_data: raise KeyError on missing elementsChris Larson2011-05-061-8/+7
| | | | | | | (Bitbake rev: a4f62433845c29f98c6a9746d5d2847bf9506ea5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Initial work on getting bitbake working under pypyChris Larson2011-05-061-17/+15
| | | | | | | | | | | - use os.chmod, not os.fchmod, as the latter is missing under pypy - rearrange our imports a bit - don't die if sqlite3 is missing shared cache support (Bitbake rev: f229824dc9c453adf6067500e2bf6761536e4f2f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* git fetcher: make tag back to work, fix Yocto bug 972Yu Ke2011-05-031-0/+1
| | | | | | | | | | | | In current git fetcher, tag does not work due to commit http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=5920e85c561624e657c126df58f5c378a8950bbc. Tag is not in sha256 form, so it will be treated invalid, and silently replaced by latest revision. To fix it, this patch treat tag name as branches name, thus it will be handled correctly later. Thanks Richard for reviewing and proposing the better approach. Fix [YOCTO #972] CC: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Yu Ke <ke.yu@intel.com>
* bitbake/fetch2: Fix the problems introduced by the git fetcher AUTOREV fixRichard Purdie2011-04-205-13/+17
| | | | | | | | | The ordering constrains on the urldata_init functions are not straight forward. To avoid further problems, create a helper function to setup the source revisions which the init functions can all at the appropriate point. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2/git: Fix a bug where AUTOREV and the git fetcher interact badlyRichard Purdie2011-04-192-6/+10
| | | | | | | | | | | | Fix a bug where ud.branches were being referenced before it was set by the git fetcher when using AUTOREV. To do this some ordering needed to be changed. This fixes errors like: ERROR: Error parsing /recipes-kernel/linux/rt-tests_git.bb: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception AttributeError: 'FetchData' object has no attribute 'branches' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix more incorrect usages of 'is'Chris Larson2011-03-314-12/+12
| | | | | | | (Bitbake rev: a26a2f548419af0e971ad21ec0a29e5245fe307f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2/local: Fix inverted update required logicRichard Purdie2011-03-091-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Allow local file:// urls to be found on mirrorsRichard Purdie2011-03-082-9/+19
| | | | | | | | | | | | With the current implementation, file:// urls as used by sstate don't access the mirror code, breaking sstate mirror support. This change enables the usual mirror handling. To do this, we remove the localfile special case, using the basename paramemter instead. We also ensure the downloads directory is checked for files. The drawback of this change is that file urls containing "*" globing require special casing in the core. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch, fetch2: Get rid of DeprecationWarning noticeKhem Raj2011-03-0310-11/+11
| | | | | | | | | | | | | * This patch fixes a cosmetic issue currently we get with master WARNING: /home/kraj/work/bitbake/lib/bb/fetch2/__init__.py:733: DeprecationWarning: Call to deprecated function bb.mkdirhier: Please use bb.utils.mkdirhier instead. bb.mkdirhier("%s/%s" % (rootdir, destdir)) (Bitbake rev: 36fe59ce314c295d239b76de34c8714def2c32d5) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure SRCREV_pn-PN is checked for a revision when the ↵Richard Purdie2011-02-231-1/+3
| | | | | | SRC_URI is unnamed Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fecth2: Ensure BB_FETCH_PREMIRRORONLY being set as false is handled ↵Richard Purdie2011-02-161-1/+2
| | | | | | correctly Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Revert part of the unpack change until the issues with it ↵Richard Purdie2011-02-131-2/+2
| | | | | | are resolved Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: unpack revisionMark Hatle2011-02-121-39/+53
| | | | | | | | | | | | Revise the unpack function to have a way to disable the unpack. This is based on the work from "Andreas Oberritter <obi@opendreambox.org>", see http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=2bdfe8519eda8067845019a699acdf19a21ba380 In addition, the to_boolean function comes from the work of "Chris Larson <chris_larson@mentor.com>", see http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=900cc29b603691eb3a077cb660545ead3715ed54 Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
* bitbake/fetch2: Add explict network access exception and handling to give ↵Richard Purdie2011-02-119-17/+33
| | | | | | users usable error messages Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix message about md5 checksums when sha256 is incorrectRichard Purdie2011-02-111-1/+9
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix patch merge errorRichard Purdie2011-02-111-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Add some debug output so its clear when PREMIRRORS, upstream ↵Richard Purdie2011-02-111-0/+3
| | | | | | and MIRRORS are being used Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure network access status is reset for each loop ↵Richard Purdie2011-02-111-0/+4
| | | | | | iteration in download() Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2/wget: Fix missing string parameter referenceRichard Purdie2011-02-111-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure failed fetch attempts are logged in the debug logsRichard Purdie2011-02-111-2/+4
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Correctly handle git mirror tarball fetchingRichard Purdie2011-02-111-4/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Pass over malformatted (empty) mirror url linesRichard Purdie2011-02-111-1/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Correct the clean() mechanism for the fetcher2 codeSaul Wold2011-02-105-0/+63
| | | | | | | This create a clean() method in each of the fetcher modules and correctly cleans the .done stamp file and lock files Signed-off-by: Saul Wold <sgw@linux.intel.com>
* bitbake/fetch2/git: Write mirror tarballs if enabled and they don't exist, ↵Richard Purdie2011-02-101-11/+11
| | | | | | also set a default value for mirror tarball generation Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix negated if check for BB_FETCH_PREMIRRORONLYRichard Purdie2011-02-091-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Add SRPM knowledgeMark Hatle2011-02-081-24/+38
| | | | | | | | | | | | | | | | Enable the fetcher to be able to unpack and SRPM. By default the system will unpack the contents of the SRPM into the WORKDIR. A new syntax "unpack=file" was developed for the SRC_URI, to allow for a recipe to extract a specific file within an SRPM. An unpack operation will then be executed on the extracted file. In order to apply extracted patches (or unpack files not specified with unpack), you must specify the path using WORKDIR, i.e.: file://${WORKDIR}/mypatch.patch Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
* bitbake/fetch2: Ignore UnboundLocalError in exception handler in try_mirrorRichard Purdie2011-02-081-2/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Move symlink handling into try_mirror where is belongs ↵Richard Purdie2011-02-081-12/+21
| | | | | | instead of the main download function Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: When using BB_FETCH_PREMIRRORONLY, set BB_NO_NETWORK after ↵Richard Purdie2011-02-081-13/+15
| | | | | | premirrors as there could be data processing needed by the real fetcher Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure we only remove files, not directories when fetch ↵Richard Purdie2011-02-081-3/+5
| | | | | | failures occur Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure original ud is preserved in try_mirrorRichard Purdie2011-02-081-4/+4
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2/git: Fix broken variable referenceRichard Purdie2011-02-081-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2/git: Ensure unresolved branches are translated into revisionsRichard Purdie2011-02-081-1/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: add try/finally to ensure lockfile is unlocked on failureSaul Wold2011-02-071-42/+44
| | | | Signed-off-by: Saul Wold <sgw@linux.intel.com>
* fetch2: add runfetchcmd to import for fetchersSaul Wold2011-02-076-2/+15
| | | | Signed-off-by: Saul Wold <sgw@linux.intel.com>
* bitbake/fetch2:Fetch Make using the fn based cache optionalRichard Purdie2011-02-071-3/+4
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix pickling issues with fetcher exceptionsRichard Purdie2011-02-071-14/+21
| | | | | | | See the problems in http://bugs.python.org/issue1692335, need to set self.args correctly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-26 17:52:05 +0000