summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch2/__init__.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake/fetch2: remove references to ChecksumError classMatthew McClintock2012-08-211-6/+2
| | | | | | | | | | | | | | | | From: Matthew McClintock <msm@freescale.com> When merging fetch2 improvements from master into denzil, there were too many dependencies to pull in the entire ChecksumError class, so this patch removes references to ChecksumError for compatability. Fixes this issue: NameError: global name 'ChecksumError' is not defined Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Scott Garman <scott.a.garman@intel.com>
* bitbake: fetch2: Handle errors orruring when building mirror urlsRichard Purdie2012-08-211-2/+11
| | | | | | | | | | | | When we build the mirror urls, its possible an error will occur. If it does, it should just mean we don't attempt this mirror url. The current code actually aborts *all* the mirrors, not just the failed url. This patch catches and logs the exception allowing things to continue. (Bitbake rev: c35cbd1a1403865cf4f59ec88e1881669868103c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Improve mirror looping to consider more casesRichard Purdie2012-08-211-12/+29
| | | | | | | | | | | | | | | | Currently we only consider one pass through the mirror list. This doesn't catch cases where for example you might want to setup a mirror of a mirror and allow multiple redirection. There is no reason we can't support this and the patch loops through the list recursively now. As a safeguard, it will stop if any duplicate urls are found, hence avoiding circular dependency looping. (From Poky rev: 0ec0a4412865e54495c07beea1ced8355da58073) (Bitbake rev: e585730e931e6abdb15ba8a3849c5fd22845b891) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Explicitly check for mirror tarballs in mirror handling codeRichard Purdie2012-08-211-1/+2
| | | | | | | | | | | | With support for things like git:// -> git:// urls, we need to be more explicity about the mirrortarball check since we need to fall through to the following code in other cases. (From Poky rev: 28e858cd6f7509468ef3e527a86820b9e06044db) (Bitbake rev: a2459f5ca2f517964287f9a7c666a6856434e631) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Split try_mirrors into two partsRichard Purdie2012-08-211-49/+59
| | | | | | | | | | | | | | | | There are no functionality changes in this change (From Poky rev: d222ebb7c75d74fde4fd04ea6feb27e10a862bae) (Bitbake rev: db62e109cc36380ff8b8918628c9dea14ac9afbc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Conflicts: bitbake/lib/bb/fetch2/__init__.py Signed-off-by: Khem Raj <kraj@juniper.net>
* bitbake: fetch2: Ensure when downloading we are consistently in the same ↵Richard Purdie2012-08-211-1/+5
| | | | | | | | | | | | | | directory This assists with build reproducuility. It also avoids errors if cwd happens not to exist when we call into the fetcher. That situation would be unusual but I hit it with the unit tests. (From Poky rev: 86517af9e066c2da1d580fa66b7c7f0340f3403e) (Bitbake rev: b886c6c15a58643e06ca5ad7a3ff1f7766e4f48c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Only cache data if fn is set, its pointless caching it ↵Richard Purdie2012-08-211-2/+2
| | | | | | | | | | against a None value (From Poky rev: c2df30bf6d1f8c263a38c45866936c1bf496ece5) (Bitbake rev: f4b59cc6e1c3ddc168a1678ce39ff402ea1ff4cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix error handling in uri_replace()Richard Purdie2012-08-211-1/+2
| | | | | | | | (From Poky rev: 1bfba28a583cb167f60e05ecdf34d0786dc1eec5) (Bitbake rev: aa7467a764ddcbc7d65af99e88cf093b6ec6d24e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/__init__: Make it clearer when uri_replace doesn't return a ↵Richard Purdie2012-08-211-2/+4
| | | | | | | | | | match (From Poky rev: dc9976331c5cbb0983adb54f6deb97b9203bacbc) (Bitbake rev: eb96609864dec95a516e6e687dd6a2f31d523acf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Hob: add fadeout display effection for recipe view include pageLiming An2012-04-111-3/+4
| | | | | | | | | | | | | | As UI request, in recipes selection page, if user exclude a item, the related depends recipes will be excluded together,so the view clearly to add it. [YOCTO #2100] (Bitbake rev: c9eed04c6275ef2c694f89e047f85c7de76f89b6) Signed-off-by: Liming An <limingx.l.an@intel.com> Signed-off-by: Shane Wang <shane.wang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure mirror tarballs have .done stamps so we don't ↵Richard Purdie2012-03-231-0/+1
| | | | | | | | | | redownload and corrupt them [YOCTO #2154] (Bitbake rev: 521d6b14151d3bf934b9597557f7ae46e50a3d7a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix typo which broke use of tarballsGary Thomas2012-03-161-1/+1
| | | | | | | (Bitbake rev: 5a1e62a8cdd4f0253b0dc59046b3b4c3186af461) Signed-off-by: Gary Thomas <gary@mlbassoc.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch: Fix uri_replaceRichard Purdie2012-03-151-6/+7
| | | | | | | | | | | | | We were seeing duplicate endings to urls: DEBUG: For url ['http', 'www.apache.org', '/dist/subversion/subversion-1.7.1.tar.bz2', '', '', {}] comparing ['http', 'www.apache.org', '/dist', '', '', {}] to ['http', 'archive.apache.org', '/dist', '', '', {}] DEBUG: For url http://www.apache.org/dist/subversion/subversion-1.7.1.tar.bz2 returning http://archive.apache.org/dist/subversion/subversion-1.7.1.tar.bz2/subversion-1.7.1.tar.bz2 This patch addresses this by only performing substitutions when really needed. (Bitbake rev: a17473d51f54a9dfb4eeaa5ad516c4851d5ac142) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* stderr would previously be appended to stdout, corrupting the result when ↵Adrien Bustany2012-03-021-23/+17
| | | | | | | | | something was outputed to stderr but exit code was still 0 (non-fatal warning messages). This commit makes the code parse only stdout, but output stderr if an error happened. (Bitbake rev: 4a480a052f450c4ee061ab0e60a495a45f140cf9) Signed-off-by: Adrien Bustany <adrien.bustany@nokia.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: allow resuming interrupted fetchPaul Eggleton2012-01-201-3/+3
| | | | | | | | | | | | | | | | | A lot of our checks in the fetcher code assume that if the downloaded file exists, the download finished successfully; this meant that if BitBake was interrupted in the middle of a fetch then it would not resume the fetch the next time, but instead attempt to use the half-fetched file and usually fail as a result. Since we're already writing a ".done" stamp file when a fetch completes, just check for its existence and assume the download didn't complete successfully if it isn't present. (Bitbake rev: 721e986624529aedef96dd06c9fe6243f2897b10) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: reduce output for fetch failuresPaul Eggleton2012-01-151-1/+6
| | | | | | | | | | | | | | | | | | | Make the warning for the initial fetch failure a single line - we don't need the full command and output here yet, but write it into the log in full as a debug message. However, if fetching from mirrors fails as well then print out the full details for the first error that occurred as an ERROR rather than a WARNING. Since this is logged as an ERROR, combined with an earlier patch it suppresses the full log which does make the output much more readable for any fetch error. Fixes [YOCTO #1832]. (Bitbake rev: 6bbdc7d259c0cc041b62dbdb26cfc3ec6edcb6f3) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: correctly decode exit signal/statusPaul Eggleton2012-01-151-4/+7
| | | | | | | | | | | | | The termination signal and exit code of the fetch process were not being decoded correctly, resulting in bitbake reporting that the process terminated with a signal of the exit code (if it was under 255). There are functions in the Python os module to do this decoding correctly (for Unix at least), so let's use them. (Bitbake rev: 50aea9a76e40cf71cc3f1462c88298e4846a031c) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: improve error formatting for fetcher errorsPaul Eggleton2012-01-151-4/+7
| | | | | | | | | | | | | | | * The "name" argument to FuncFailed is rarely used as a name in actual usage within bitbake, so don't treat it as one in the output. * Don't print URL for FetchError if it was not specified (i.e. don't output "Fetcher failure for URL 'None'") * Don't include URL in "unable to fetch from any source" message since we supply it to FetchError and it will be printed anyway. * Don't include URL in "checksum failed" message for the same reason (Bitbake rev: 86811bd85e2e453ee92a05fe60160d9b49ac69e8) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: fix Exception parameters when BB_STRICT_CHECKSUM enabledJoshua Lock2011-12-051-3/+3
| | | | | | | | | | | | BB_STRICT_CHECKSUM caused a backtrace as the FetchError parameters are incorrectly specified such that FetchError is being passed 8 params when it's expecting 3. This fixes the parameters so we're passing a formatted string and the url. (Bitbake rev: b8b2f4287c9125542b18a294c0a94ed89a7e73a8) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-271-39/+39
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Improve uri_replace to handle paths with no trailing '/'Richard Purdie2011-11-241-3/+10
| | | | | | | | | | | | | | | | | | | | | | Currently if you specify a mirror like: file://.* http://linux.freescale.net/yocto/sstate-cache it won't work as you expect whilst: file://.* http://linux.freescale.net/yocto/sstate-cache/ will since it has the trailing slash. This patch handles both cases correctly. It also adds some debug to the uri_replace function since its near impossible to debug it without some kind of output. [YOCTO #1578] (Bitbake rev: a0246bf09c93bb657eaf6ba61d090b247ed33640) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: enable checksum definition as SRC_URI parameterJoshua Lock2011-10-241-2/+8
| | | | | | | | | | | | | | | | URI parameters should be able to be defined as a parameter of the SRC_URI, this patch enables thus for checksums. An example; SRC_URI = "http://pkgconfig.freedesktop.org/releases/pkg-config-${PV}.tar.gz;md5sum=a3270bab3f4b69b7dc6dbdacbcae9745;sha256sum=3ba691ee2431f32ccb8efa131e59bf23e37f122dc66791309023ca6dcefcd10e" Addresses the remainder of [YOCTO #1399] (Bitbake rev: 5f8f923b76722c9b6c7ffbe19e94df50f900155f) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: improve usability of checksumsJoshua Lock2011-10-241-34/+38
| | | | | | | | | | | | | | | | | | | | This patch improves the usability of checksums by only requiring one checksum be defined. Further, checksum verification will provide as much information as possible at, rather than a bit at a time. No longer will you need to run fetch, see an md5sum mismatch, fix it, run fetch, seen an sha256sum mismatch, fix it and fetch again. If neither checksum is defined we now report both missing sums at once - rather than one after the other. Finally, if both sums are incorrect, we'll report both incorrect sums at the same time. Fixes part of [YOCTO #1399] (Bitbake rev: 30ca41e596955fcbbe1ae6f4436ecf14cef96f8d) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Export additional variables to the fetchersMatthew McClintock2011-10-061-1/+2
| | | | | | | | | | git could need these environment variables when working behind a proxy (Bitbake rev: dca46cc2e1c75b6add2c4801e2994a4812745f5b) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetcher2: show warning message when checksum is mismatchYu Ke2011-08-101-1/+1
| | | | | | | | | | | | | | | | Currently, if checksum mismatch, fetcher will try mirror, and if mirror fetching fail, it will show error "Unable to fetch URL xxx from from any source" which actually hide the real reason and make user confuse. so showing warning message will make it more clear. Fix [YOCTO #1256] (Bitbake rev: b3e924297d670963714343d02f7898798fec84fd) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetcher2: retry mirror if upstream checksum mismatchYu Ke2011-07-131-11/+20
| | | | | | | | | | | | | | | This patch is for [YOCTO #1085] fix. If the upstream fails a checksum, retry from the MIRROR before giving up. This will add more robust fetching if an upstream serves a bad file or webpage. fetching of distcc prior to the move from samba -> googlecode is a good example of this. (Bitbake rev: b631e922257de52bf2247c01152d9856c870e7d0) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* make exception handling syntax consistentScott Garman2011-06-151-2/+2
| | | | | | | | | | Update exception handling syntax to use the modern style: except ExcType as localvar (Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d) Signed-off-by: Scott Garman <scott.a.garman@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: When replacing URLs in mirror handling mask out empty entriesRichard Purdie2011-06-131-1/+4
| | | | | | | | | | | | | The symptom of this problem is something like a cvs url which specifies a username where the username is then passed through to something like an http mirror. This patch fixes things by ensuring empty entries are preserved in the new URL. (Bitbake rev: c1d978d7bd1ac8eb1e2d50029ab2384be9f72fb4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* persist_data: implement comparison, same as dictChris Larson2011-05-061-9/+6
| | | | | | | (Bitbake rev: 1190406c526c7bb7cf415867be83e0403812a7dd) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* persist_data: raise KeyError on missing elementsChris Larson2011-05-061-8/+7
| | | | | | | (Bitbake rev: a4f62433845c29f98c6a9746d5d2847bf9506ea5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Initial work on getting bitbake working under pypyChris Larson2011-05-061-17/+15
| | | | | | | | | | | - use os.chmod, not os.fchmod, as the latter is missing under pypy - rearrange our imports a bit - don't die if sqlite3 is missing shared cache support (Bitbake rev: f229824dc9c453adf6067500e2bf6761536e4f2f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix the problems introduced by the git fetcher AUTOREV fixRichard Purdie2011-04-201-12/+9
| | | | | | | | | The ordering constrains on the urldata_init functions are not straight forward. To avoid further problems, create a helper function to setup the source revisions which the init functions can all at the appropriate point. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2/git: Fix a bug where AUTOREV and the git fetcher interact badlyRichard Purdie2011-04-191-2/+5
| | | | | | | | | | | | Fix a bug where ud.branches were being referenced before it was set by the git fetcher when using AUTOREV. To do this some ordering needed to be changed. This fixes errors like: ERROR: Error parsing /recipes-kernel/linux/rt-tests_git.bb: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception AttributeError: 'FetchData' object has no attribute 'branches' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Allow local file:// urls to be found on mirrorsRichard Purdie2011-03-081-9/+6
| | | | | | | | | | | | With the current implementation, file:// urls as used by sstate don't access the mirror code, breaking sstate mirror support. This change enables the usual mirror handling. To do this, we remove the localfile special case, using the basename paramemter instead. We also ensure the downloads directory is checked for files. The drawback of this change is that file urls containing "*" globing require special casing in the core. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch, fetch2: Get rid of DeprecationWarning noticeKhem Raj2011-03-031-2/+2
| | | | | | | | | | | | | * This patch fixes a cosmetic issue currently we get with master WARNING: /home/kraj/work/bitbake/lib/bb/fetch2/__init__.py:733: DeprecationWarning: Call to deprecated function bb.mkdirhier: Please use bb.utils.mkdirhier instead. bb.mkdirhier("%s/%s" % (rootdir, destdir)) (Bitbake rev: 36fe59ce314c295d239b76de34c8714def2c32d5) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure SRCREV_pn-PN is checked for a revision when the ↵Richard Purdie2011-02-231-1/+3
| | | | | | SRC_URI is unnamed Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fecth2: Ensure BB_FETCH_PREMIRRORONLY being set as false is handled ↵Richard Purdie2011-02-161-1/+2
| | | | | | correctly Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Revert part of the unpack change until the issues with it ↵Richard Purdie2011-02-131-2/+2
| | | | | | are resolved Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: unpack revisionMark Hatle2011-02-121-39/+53
| | | | | | | | | | | | Revise the unpack function to have a way to disable the unpack. This is based on the work from "Andreas Oberritter <obi@opendreambox.org>", see http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=2bdfe8519eda8067845019a699acdf19a21ba380 In addition, the to_boolean function comes from the work of "Chris Larson <chris_larson@mentor.com>", see http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=900cc29b603691eb3a077cb660545ead3715ed54 Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
* bitbake/fetch2: Add explict network access exception and handling to give ↵Richard Purdie2011-02-111-2/+18
| | | | | | users usable error messages Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix message about md5 checksums when sha256 is incorrectRichard Purdie2011-02-111-1/+9
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Fix patch merge errorRichard Purdie2011-02-111-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Add some debug output so its clear when PREMIRRORS, upstream ↵Richard Purdie2011-02-111-0/+3
| | | | | | and MIRRORS are being used Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure network access status is reset for each loop ↵Richard Purdie2011-02-111-0/+4
| | | | | | iteration in download() Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Ensure failed fetch attempts are logged in the debug logsRichard Purdie2011-02-111-2/+4
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Correctly handle git mirror tarball fetchingRichard Purdie2011-02-111-4/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch2: Pass over malformatted (empty) mirror url linesRichard Purdie2011-02-111-1/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Correct the clean() mechanism for the fetcher2 codeSaul Wold2011-02-101-0/+34
| | | | | | | This create a clean() method in each of the fetcher modules and correctly cleans the .done stamp file and lock files Signed-off-by: Saul Wold <sgw@linux.intel.com>
* bitbake/fetch2: Fix negated if check for BB_FETCH_PREMIRRORONLYRichard Purdie2011-02-091-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Add SRPM knowledgeMark Hatle2011-02-081-24/+38
| | | | | | | | | | | | | | | | Enable the fetcher to be able to unpack and SRPM. By default the system will unpack the contents of the SRPM into the WORKDIR. A new syntax "unpack=file" was developed for the SRC_URI, to allow for a recipe to extract a specific file within an SRPM. An unpack operation will then be executed on the extracted file. In order to apply extracted patches (or unpack files not specified with unpack), you must specify the path using WORKDIR, i.e.: file://${WORKDIR}/mypatch.patch Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-19 19:22:57 +0000