summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch
Commit message (Collapse)AuthorAgeFilesLines
* Switch bitbake internals to use logging directly rather than bb.msgChris Larson2011-01-0412-76/+78
| | | | | | | | | | | | | | We use a custom Logger subclass for our loggers This logger provides: - 'debug' method which accepts a debug level - 'plain' method which bypasses log formatting - 'verbose' method which is more detail than info, but less than debug (Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Allow %20 in a file name in the SRC_URIC Michael Sundius2011-01-041-1/+2
| | | | | | | | (Bitbake rev: f7c181a0f6ab0b4d33bf80a0e24a788de441f82b) Signed-off-by: C Michael Sundius <msundius@sundius.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Fix fetch errorRichard Purdie2010-12-211-3/+3
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Only checksum downloads when they're first downloaded and a ↵Richard Purdie2010-12-211-5/+5
| | | | | | checksum is present Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Add missing return so if a checksum isn't present, it isn't ↵Richard Purdie2010-12-201-0/+1
| | | | | | checked Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Checksum validity fixesRichard Purdie2010-12-201-11/+7
| | | | | | | | | If the checksum check failed, the .md5 stamp file would still have been created meaning subsequent builds would proceed with the corrupt file. Reorder the calls to avoid this. Also raise a specific error for the checksum not specified error case. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* FetchData: add SRC_URI checksumYu Ke2010-12-201-0/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch add the per-recipe SRC_URI checksum verification. - SRC_URI format The format of SRC_URI checksum follow OE definition: 1. SRC_URI has single src SRC_URI = "http://some.domain/file.tar.gz" SRC_URI[md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" 2. SRC_URI has multiple src, every src need specify name SRC_URI = "http://some.domain/file1.tar.gz;name=name1 \ http://some.domain/file2.tar.gz;name=name2 " SRC_URI[name1.md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[name1.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" SRC_URI[name2.md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[name2.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" - SRC_URI checking invocation: the checksum checking is invoked in do_fetch phase, so it can be invoked manually by # bitbake -f -c fetch <recipe_name> if recipes has no SRC_URI checksum item, bitbake will show warning: " WARNING: Missing SRC_URI checksum for xxxx.tar.gz, consider to add SRC_URI[md5sum] = "5c69f16d452b0bb3d44bc3c10556c072" SRC_URI[sha256sum] = "f4e0ada8d4d516bbb8600a3ee7d9046c9c79e38cd781df9ffc46d8f16acd1768" " thus recipe author can add it to recpie file after SRC_URI - control variable BB_STRICT_CHECKSUM when SRC_URI checksum is missing, this variable decide pass or not if BB_STRICT_CHECKSUM = "1", bitbake should fatal in this case, otherwise bitbake just pass Signed-off-by: Yu Ke <ke.yu@intel.com>
* bitbake/fetch: Allow checking of a single url at a time (massive sstate ↵Richard Purdie2010-11-131-2/+5
| | | | | | speed improvement) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Fix handling of mirrors when checking for url validityRichard Purdie2010-11-131-2/+4
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Revert "bitbake/fetch/local: Also check DL_DIR for files since they could ↵Richard Purdie2010-11-131-4/+0
| | | | | | | | already exists there" These changes were incorrect. This reverts commit ae98f7eacb9e61fe086d88dc694b4c651af9fee3.
* Revert "bitbake/fetch/local: Fix os.exists reference"Richard Purdie2010-11-131-1/+1
| | | | | | These changes were incorrect. This reverts commit f8e33979352528bb7c289e7c839605a5880e1e43.
* bitbake/fetch/local: Fix os.exists referenceRichard Purdie2010-11-131-1/+1
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch/local: Also check DL_DIR for files since they could already ↵Richard Purdie2010-11-131-0/+4
| | | | | | | | exists there [BUGID #533] Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: When fetchers return errors, ensure any partial download is ↵Richard Purdie2010-10-231-0/+9
| | | | | | cleared Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Make URL checking slightly less verbose (distracting with the ↵Richard Purdie2010-10-221-1/+1
| | | | | | sstate code) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch/git: Ensure fullclone repositories are fully fetchedRichard Purdie2010-10-191-1/+4
| | | | | | | | | | | The git fetcher was failing to pull in new branches into a git repository mirror tarball as the git fetch command being used didn't add new remote branches. This patch uses "git fetch --all" for fullclones to ensure any new remote branches are cloned correctly. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetcher: Deal with a ton of different bugsRichard Purdie2010-10-192-36/+38
| | | | | | | | | | | | | | | | | | | | | | The more we try and patch up the fetcher code, the more things break. The code blocks in question are practically unreadable and are full of corner cases where fetching could fail. In summary the issues noticed included: a) Always fetching strange broken urls from the premirror for "noclone" git repositories b) Not creating or rewriting .md5 stamp files inconsistently c) Always fetching git source mirror tarballs from the premirror even if they already exist but the checkout directory does now d) Passing "None" values to os.access() and os.path.extsts() checks under certain circumstances e) Not using fetched git mirror tarballs if the preexist and always try and fetch them. This patch rewrites the sections of code in question to be simpler and more readable, fixing the above problems and most likely other odd corner cases. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch/git.py: Fix git fetcher to correctly use mirror tarballsRichard Purdie2010-10-191-1/+5
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Ensure SRCREV is still set correctly if the pn-X override has ↵Richard Purdie2010-10-091-0/+2
| | | | | | been expanded Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch/__init__.py: Abstract mirror variable handling and enhance to ↵Richard Purdie2010-10-071-5/+8
| | | | | | accept \n delimitation in variables Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Export ALL_PROXY for bitbake commandsDarren Hart2010-10-071-1/+4
| | | | | | | | We already export http,ftp,https proxy environment variables. Some environments, GNOME for instance, place the socks proxy in ALL_PROXY and all_proxy. Export it as well. Signed-off-by: Darren Hart <dvhart@linux.intel.com>
* bitbake/fetch: if mirror fetching fails, ensure exception is raisedJoshua Lock2010-10-051-1/+3
| | | | | | | | | | | | | | We catch any exception raised by the fetchers go() method and attempt to work around it by trying any (post) mirrors which are configured. However, should the mirrors fail the exception is lost and the fetch is assumed to have completed successfully. Instead, save the exception and if the local file does not exist after trying the mirrors re-raise the exception. Fixes [BUGID #362] Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/fetch: fix logic to prevent fetches when the file already existsJoshua Lock2010-09-281-0/+2
| | | | Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/fetch: ensure the go() method completes when not using premirrorsJoshua Lock2010-09-101-6/+10
| | | | Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake: Add proxy variables to standard export listZhai Edwin2010-09-101-1/+1
| | | | | | | With "no_proxy" exported to bitbake, both internal and externel file mirror can be used. "https_proxy" enable fetching "https://" file through proxy. Signed-off-by: Zhai Edwin <edwin.zhai@intel.com>
* bitbake/__init__.py: Fix named SRCREVS variable name ordering with backwards ↵Richard Purdie2010-09-091-1/+3
| | | | | | compatibility Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch/git: fix try_premirror() definition .Method takes 4 parameters.Joshua Lock2010-09-081-1/+1
| | | | Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/fetch: fix by moving try_premirror() to the right placeJoshua Lock2010-09-081-12/+11
| | | | | | The previous commit didn't define the method as part of the Fetch object Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/fetch: add try_premirror method and special case git fetcherJoshua Lock2010-09-082-2/+20
| | | | | | | | | | | Add a new method, try_premirror, to the Fetch object which checks to see whether the file needs to be fetched from a premirror. Override this in the Git fetcher to only require a pre-mirror fetch when the clone directory does not exist. Fixes [BUGID 290] Signed-off-by: Joshua Lock <josh@linux.intel.com>
* fetch: fix setting of localpath in SRC_URI parametersJoshua Lock2010-09-061-0/+1
| | | | | | | | | When setting localpath in the SRC_URI parameters the basename should be set explicitly as the localpath() method is not called. Fixes [BUGID #81] Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/fetch: ensure the mirrored repository is updated as requiredJoshua Lock2010-09-011-1/+3
| | | | | | | | | If we fetch a tarball from a mirror it's entirely possible that the mirror will have been from before the required tag/branch/etc was included in the repository. To that end use forcefetch() as a way of testing whether the repository is up to date and if not fetch updates. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/git: define a forcefetch methodJoshua Lock2010-09-011-1/+6
| | | | | | | The git fetcher should force a fetch if the required tag is not present in the local clone, or if the fullclone parameter is set. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/fetch: Respect forcefetch even when pulling from a mirrorJoshua Lock2010-09-011-3/+3
| | | | | | | | | When pulling from a premirror we would prefer a local tarball even when the caller had specified the forcefetch parameter. Add an extra parameter 'force' to try_mirrors, defaulting to False. If set the mirrors will be tested even if the file exists locally. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/git.py: Make sure different branches can have different revisions ↵Richard Purdie2010-08-131-1/+1
| | | | | | without triggering build count increases Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/git.py: Allow tracking of branches in SRC_URI without cloning for ↵Richard Purdie2010-08-132-4/+9
| | | | | | use with fullclone Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/git.py: Make sure a full clone checkout always updatesRichard Purdie2010-08-131-1/+1
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* *: cosmetic whitespace removalBernhard Reutner-Fischer2010-08-031-1/+1
| | | | | | | (Bitbake rev: 2ed36a3d57de25e1af31d657a7b0b14857e5056a) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: fetch: Only mark srcrev recipes as uncached when outside of SRCREV ↵Richard Purdie2010-07-251-1/+2
| | | | | | caching mode Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* 3.0 prepChris Larson2010-07-022-12/+14
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Apply some 2to3 refactoringsChris Larson2010-07-021-1/+1
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Start ditching fatal() calls in favor of raising appropriate exceptionsChris Larson2010-07-021-3/+3
| | | | | | | (Bitbake rev: 681b73fcc04e22e692ed61650ad53c800b64cace) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Change wget fetcher to use the runfetchcmdRyan Phillips2010-07-022-25/+6
| | | | | | | | | | * Fixes proxy support to honor standard proxy environment variables. * Quote environment variables (Bitbake rev: f84f382f340d6db15b9e5afb8c7c93969249a958) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* persist_data: cache connection and use cursorBernhard Reutner-Fischer2010-07-021-4/+5
| | | | | | | | | | Store database connection to persistent database in fetcher. (Bitbake rev: 8a6876752b90efd81d92f0947bfc9527d8260969) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: commentary typo fix in fetcher_init()Bernhard Reutner-Fischer2010-07-021-1/+1
| | | | | | | (Bitbake rev: b786f1f60fb50565295e7c8e979b471327d924f3) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: documentation typo fixBernhard Reutner-Fischer2010-07-021-1/+1
| | | | | | | | | s/initilize/initialize/ (Bitbake rev: 1497dd0d5bc0cca0de7d0ee9d314287000a0c34d) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: Check for existence of tar file along with md5 signatureKhem Raj2010-07-021-2/+2
| | | | | | | | (Bitbake rev: a5b1c0605328660de15bb8127b6f7d798c664d32) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Kill unnecessary usages of the types moduleChris Larson2010-07-021-2/+1
| | | | | | | | | | | | | types.IntType -> int types.StringType -> basestring ... Also moves our ImmutableTypes tuple into our own namespace. (Bitbake rev: 83674a3a5564ecb1f9d2c9b2d5b1eeb3c31272ab) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Apply some 2to3 transforms that don't cause issues in 2.6Chris Larson2010-07-023-16/+16
| | | | | | | (Bitbake rev: d39ab776e7ceaefc8361150151cf0892dcb70d9c) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Switch some references to moved functionsChris Larson2010-07-022-3/+5
| | | | | | | (Bitbake rev: ddea2978cb969cf1381cfee5c055de0c9e56a4e3) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Add a missing import of print_functionChris Larson2010-07-021-0/+1
| | | | | | | (Bitbake rev: 30ae747cc5c733512e417c61fca23c0cd15b0e44) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-07-01 11:19:39 +0000