summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch2/__init__.py
Commit message (Collapse)AuthorAgeFilesLines
* Hob: add fadeout display effection for recipe view include pageLiming An2012-04-111-3/+4
| | | | | | | | | | | | | | As UI request, in recipes selection page, if user exclude a item, the related depends recipes will be excluded together,so the view clearly to add it. [YOCTO #2100] (Bitbake rev: c9eed04c6275ef2c694f89e047f85c7de76f89b6) Signed-off-by: Liming An <limingx.l.an@intel.com> Signed-off-by: Shane Wang <shane.wang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure mirror tarballs have .done stamps so we don't ↵Richard Purdie2012-03-231-0/+1
| | | | | | | | | | redownload and corrupt them [YOCTO #2154] (Bitbake rev: 521d6b14151d3bf934b9597557f7ae46e50a3d7a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix typo which broke use of tarballsGary Thomas2012-03-161-1/+1
| | | | | | | (Bitbake rev: 5a1e62a8cdd4f0253b0dc59046b3b4c3186af461) Signed-off-by: Gary Thomas <gary@mlbassoc.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch: Fix uri_replaceRichard Purdie2012-03-151-6/+7
| | | | | | | | | | | | | We were seeing duplicate endings to urls: DEBUG: For url ['http', 'www.apache.org', '/dist/subversion/subversion-1.7.1.tar.bz2', '', '', {}] comparing ['http', 'www.apache.org', '/dist', '', '', {}] to ['http', 'archive.apache.org', '/dist', '', '', {}] DEBUG: For url http://www.apache.org/dist/subversion/subversion-1.7.1.tar.bz2 returning http://archive.apache.org/dist/subversion/subversion-1.7.1.tar.bz2/subversion-1.7.1.tar.bz2 This patch addresses this by only performing substitutions when really needed. (Bitbake rev: a17473d51f54a9dfb4eeaa5ad516c4851d5ac142) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* stderr would previously be appended to stdout, corrupting the result when ↵Adrien Bustany2012-03-021-23/+17
| | | | | | | | | something was outputed to stderr but exit code was still 0 (non-fatal warning messages). This commit makes the code parse only stdout, but output stderr if an error happened. (Bitbake rev: 4a480a052f450c4ee061ab0e60a495a45f140cf9) Signed-off-by: Adrien Bustany <adrien.bustany@nokia.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: allow resuming interrupted fetchPaul Eggleton2012-01-201-3/+3
| | | | | | | | | | | | | | | | | A lot of our checks in the fetcher code assume that if the downloaded file exists, the download finished successfully; this meant that if BitBake was interrupted in the middle of a fetch then it would not resume the fetch the next time, but instead attempt to use the half-fetched file and usually fail as a result. Since we're already writing a ".done" stamp file when a fetch completes, just check for its existence and assume the download didn't complete successfully if it isn't present. (Bitbake rev: 721e986624529aedef96dd06c9fe6243f2897b10) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: reduce output for fetch failuresPaul Eggleton2012-01-151-1/+6
| | | | | | | | | | | | | | | | | | | Make the warning for the initial fetch failure a single line - we don't need the full command and output here yet, but write it into the log in full as a debug message. However, if fetching from mirrors fails as well then print out the full details for the first error that occurred as an ERROR rather than a WARNING. Since this is logged as an ERROR, combined with an earlier patch it suppresses the full log which does make the output much more readable for any fetch error. Fixes [YOCTO #1832]. (Bitbake rev: 6bbdc7d259c0cc041b62dbdb26cfc3ec6edcb6f3) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: correctly decode exit signal/statusPaul Eggleton2012-01-151-4/+7
| | | | | | | | | | | | | The termination signal and exit code of the fetch process were not being decoded correctly, resulting in bitbake reporting that the process terminated with a signal of the exit code (if it was under 255). There are functions in the Python os module to do this decoding correctly (for Unix at least), so let's use them. (Bitbake rev: 50aea9a76e40cf71cc3f1462c88298e4846a031c) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: improve error formatting for fetcher errorsPaul Eggleton2012-01-151-4/+7
| | | | | | | | | | | | | | | * The "name" argument to FuncFailed is rarely used as a name in actual usage within bitbake, so don't treat it as one in the output. * Don't print URL for FetchError if it was not specified (i.e. don't output "Fetcher failure for URL 'None'") * Don't include URL in "unable to fetch from any source" message since we supply it to FetchError and it will be printed anyway. * Don't include URL in "checksum failed" message for the same reason (Bitbake rev: 86811bd85e2e453ee92a05fe60160d9b49ac69e8) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: fix Exception parameters when BB_STRICT_CHECKSUM enabledJoshua Lock2011-12-051-3/+3
| | | | | | | | | | | | BB_STRICT_CHECKSUM caused a backtrace as the FetchError parameters are incorrectly specified such that FetchError is being passed 8 params when it's expecting 3. This fixes the parameters so we're passing a formatted string and the url. (Bitbake rev: b8b2f4287c9125542b18a294c0a94ed89a7e73a8) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-271-39/+39
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Improve uri_replace to handle paths with no trailing '/'Richard Purdie2011-11-241-3/+10
| | | | | | | | | | | | | | | | | | | | | | Currently if you specify a mirror like: file://.* http://linux.freescale.net/yocto/sstate-cache it won't work as you expect whilst: file://.* http://linux.freescale.net/yocto/sstate-cache/ will since it has the trailing slash. This patch handles both cases correctly. It also adds some debug to the uri_replace function since its near impossible to debug it without some kind of output. [YOCTO #1578] (Bitbake rev: a0246bf09c93bb657eaf6ba61d090b247ed33640) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: enable checksum definition as SRC_URI parameterJoshua Lock2011-10-241-2/+8
| | | | | | | | | | | | | | | | URI parameters should be able to be defined as a parameter of the SRC_URI, this patch enables thus for checksums. An example; SRC_URI = "http://pkgconfig.freedesktop.org/releases/pkg-config-${PV}.tar.gz;md5sum=a3270bab3f4b69b7dc6dbdacbcae9745;sha256sum=3ba691ee2431f32ccb8efa131e59bf23e37f122dc66791309023ca6dcefcd10e" Addresses the remainder of [YOCTO #1399] (Bitbake rev: 5f8f923b76722c9b6c7ffbe19e94df50f900155f) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: improve usability of checksumsJoshua Lock2011-10-241-34/+38
| | | | | | | | | | | | | | | | | | | | This patch improves the usability of checksums by only requiring one checksum be defined. Further, checksum verification will provide as much information as possible at, rather than a bit at a time. No longer will you need to run fetch, see an md5sum mismatch, fix it, run fetch, seen an sha256sum mismatch, fix it and fetch again. If neither checksum is defined we now report both missing sums at once - rather than one after the other. Finally, if both sums are incorrect, we'll report both incorrect sums at the same time. Fixes part of [YOCTO #1399] (Bitbake rev: 30ca41e596955fcbbe1ae6f4436ecf14cef96f8d) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Export additional variables to the fetchersMatthew McClintock2011-10-061-1/+2
| | | | | | | | | | git could need these environment variables when working behind a proxy (Bitbake rev: dca46cc2e1c75b6add2c4801e2994a4812745f5b) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetcher2: show warning message when checksum is mismatchYu Ke2011-08-101-1/+1
| | | | | | | | | | | | | | | | Currently, if checksum mismatch, fetcher will try mirror, and if mirror fetching fail, it will show error "Unable to fetch URL xxx from from any source" which actually hide the real reason and make user confuse. so showing warning message will make it more clear. Fix [YOCTO #1256] (Bitbake rev: b3e924297d670963714343d02f7898798fec84fd) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetcher2: retry mirror if upstream checksum mismatchYu Ke2011-07-131-11/+20
| | | | | | | | | | | | | | | This patch is for [YOCTO #1085] fix. If the upstream fails a checksum, retry from the MIRROR before giving up. This will add more robust fetching if an upstream serves a bad file or webpage. fetching of distcc prior to the move from samba -> googlecode is a good example of this. (Bitbake rev: b631e922257de52bf2247c01152d9856c870e7d0) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* make exception handling syntax consistentScott Garman2011-06-151-2/+2
| | | | | | | | | | Update exception handling syntax to use the modern style: except ExcType as localvar (Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d) Signed-off-by: Scott Garman <scott.a.garman@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: When replacing URLs in mirror handling mask out empty entriesRichard Purdie2011-06-131-1/+4
| | | | | | | | | | | | | The symptom of this problem is something like a cvs url which specifies a username where the username is then passed through to something like an http mirror. This patch fixes things by ensuring empty entries are preserved in the new URL. (Bitbake rev: c1d978d7bd1ac8eb1e2d50029ab2384be9f72fb4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* persist_data: implement comparison, same as dictChris Larson2011-05-061-9/+6
| | | | | | | (Bitbake rev: 1190406c526c7bb7cf415867be83e0403812a7dd) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* persist_data: raise KeyError on missing elementsChris Larson2011-05-061-8/+7
| | | | | | | (Bitbake rev: a4f62433845c29f98c6a9746d5d2847bf9506ea5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Initial work on getting bitbake working under pypyChris Larson2011-05-061-17/+15
| | | | | | | | | | | - use os.chmod, not os.fchmod, as the latter is missing under pypy - rearrange our imports a bit - don't die if sqlite3 is missing shared cache support (Bitbake rev: f229824dc9c453adf6067500e2bf6761536e4f2f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix the problems introduced by the git fetcher AUTOREV fixRichard Purdie2011-04-201-12/+9
| | | | | | | | | The ordering constrains on the urldata_init functions are not straight forward. To avoid further problems, create a helper function to setup the source revisions which the init functions can all at the appropriate point. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2/git: Fix a bug where AUTOREV and the git fetcher interact badlyRichard Purdie2011-04-191-2/+5
| | | | | | | | | | | | Fix a bug where ud.branches were being referenced before it was set by the git fetcher when using AUTOREV. To do this some ordering needed to be changed. This fixes errors like: ERROR: Error parsing /recipes-kernel/linux/rt-tests_git.bb: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception AttributeError: 'FetchData' object has no attribute 'branches' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Allow local file:// urls to be found on mirrorsRichard Purdie2011-03-081-9/+6
| | | | | | | | | | | | With the current implementation, file:// urls as used by sstate don't access the mirror code, breaking sstate mirror support. This change enables the usual mirror handling. To do this, we remove the localfile special case, using the basename paramemter instead. We also ensure the downloads directory is checked for files. The drawback of this change is that file urls containing "*" globing require special casing in the core. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch, fetch2: Get rid of DeprecationWarning noticeKhem Raj2011-03-031-2/+2
| | | | | | | | | | | | | * This patch fixes a cosmetic issue currently we get with master WARNING: /home/kraj/work/bitbake/lib/bb/fetch2/__init__.py:733: DeprecationWarning: Call to deprecated function bb.mkdirhier: Please use bb.utils.mkdirhier instead. bb.mkdirhier("%s/%s" % (rootdir, destdir)) (Bitbake rev: 36fe59ce314c295d239b76de34c8714def2c32d5) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure SRCREV_pn-PN is checked for a revision when the ↵Richard Purdie2011-02-231-1/+3
| | | | | | SRC_URI is unnamed Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fecth2: Ensure BB_FETCH_PREMIRRORONLY being set as false is handled ↵Richard Purdie2011-02-161-1/+2
| | | | | | correctly Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Revert part of the unpack change until the issues with it ↵Richard Purdie2011-02-131-2/+2
| | | | | | are resolved Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: unpack revisionMark Hatle2011-02-121-39/+53
| | | | | | | | | | | | Revise the unpack function to have a way to disable the unpack. This is based on the work from "Andreas Oberritter <obi@opendreambox.org>", see http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=2bdfe8519eda8067845019a699acdf19a21ba380 In addition, the to_boolean function comes from the work of "Chris Larson <chris_larson@mentor.com>", see http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=900cc29b603691eb3a077cb660545ead3715ed54 Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
* bitbake/fetch2: Add explict network access exception and handling to give ↵Richard Purdie2011-02-111-2/+18
| | | | | | users usable error messages Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix message about md5 checksums when sha256 is incorrectRichard Purdie2011-02-111-1/+9
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix patch merge errorRichard Purdie2011-02-111-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Add some debug output so its clear when PREMIRRORS, upstream ↵Richard Purdie2011-02-111-0/+3
| | | | | | and MIRRORS are being used Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure network access status is reset for each loop ↵Richard Purdie2011-02-111-0/+4
| | | | | | iteration in download() Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure failed fetch attempts are logged in the debug logsRichard Purdie2011-02-111-2/+4
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Correctly handle git mirror tarball fetchingRichard Purdie2011-02-111-4/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Pass over malformatted (empty) mirror url linesRichard Purdie2011-02-111-1/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Correct the clean() mechanism for the fetcher2 codeSaul Wold2011-02-101-0/+34
| | | | | | | This create a clean() method in each of the fetcher modules and correctly cleans the .done stamp file and lock files Signed-off-by: Saul Wold <sgw@linux.intel.com>
* bitbake/fetch2: Fix negated if check for BB_FETCH_PREMIRRORONLYRichard Purdie2011-02-091-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Add SRPM knowledgeMark Hatle2011-02-081-24/+38
| | | | | | | | | | | | | | | | Enable the fetcher to be able to unpack and SRPM. By default the system will unpack the contents of the SRPM into the WORKDIR. A new syntax "unpack=file" was developed for the SRC_URI, to allow for a recipe to extract a specific file within an SRPM. An unpack operation will then be executed on the extracted file. In order to apply extracted patches (or unpack files not specified with unpack), you must specify the path using WORKDIR, i.e.: file://${WORKDIR}/mypatch.patch Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
* bitbake/fetch2: Ignore UnboundLocalError in exception handler in try_mirrorRichard Purdie2011-02-081-2/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Move symlink handling into try_mirror where is belongs ↵Richard Purdie2011-02-081-12/+21
| | | | | | instead of the main download function Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: When using BB_FETCH_PREMIRRORONLY, set BB_NO_NETWORK after ↵Richard Purdie2011-02-081-13/+15
| | | | | | premirrors as there could be data processing needed by the real fetcher Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure we only remove files, not directories when fetch ↵Richard Purdie2011-02-081-3/+5
| | | | | | failures occur Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure original ud is preserved in try_mirrorRichard Purdie2011-02-081-4/+4
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: add try/finally to ensure lockfile is unlocked on failureSaul Wold2011-02-071-42/+44
| | | | Signed-off-by: Saul Wold <sgw@linux.intel.com>
* bitbake/fetch2:Fetch Make using the fn based cache optionalRichard Purdie2011-02-071-3/+4
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix pickling issues with fetcher exceptionsRichard Purdie2011-02-071-14/+21
| | | | | | | See the problems in http://bugs.python.org/issue1692335, need to set self.args correctly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Update mirror processing to ensure we look for mirror tarballsRichard Purdie2011-02-071-16/+21
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-10 14:22:39 +0000