summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-272-19/+19
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync fetch back with upstreamRichard Purdie2011-05-061-9/+6
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch.git: fix a remnant wrt persist + keyerrorChristopher Larson2011-05-061-4/+4
| | | | | | | (Bitbake rev: 7492233f5249d348024bc3daa113a96b765f94db) Signed-off-by: Christopher Larson <kergoth@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* persist_data: implement comparison, same as dictChris Larson2011-05-061-4/+2
| | | | | | | (Bitbake rev: 1190406c526c7bb7cf415867be83e0403812a7dd) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* persist_data: raise KeyError on missing elementsChris Larson2011-05-062-16/+17
| | | | | | | (Bitbake rev: a4f62433845c29f98c6a9746d5d2847bf9506ea5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Initial work on getting bitbake working under pypyChris Larson2011-05-061-4/+4
| | | | | | | | | | | - use os.chmod, not os.fchmod, as the latter is missing under pypy - rearrange our imports a bit - don't die if sqlite3 is missing shared cache support (Bitbake rev: f229824dc9c453adf6067500e2bf6761536e4f2f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix more incorrect usages of 'is'Chris Larson2011-03-311-3/+3
| | | | | | | (Bitbake rev: a26a2f548419af0e971ad21ec0a29e5245fe307f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch, fetch2: Get rid of DeprecationWarning noticeKhem Raj2011-03-039-10/+10
| | | | | | | | | | | | | * This patch fixes a cosmetic issue currently we get with master WARNING: /home/kraj/work/bitbake/lib/bb/fetch2/__init__.py:733: DeprecationWarning: Call to deprecated function bb.mkdirhier: Please use bb.utils.mkdirhier instead. bb.mkdirhier("%s/%s" % (rootdir, destdir)) (Bitbake rev: 36fe59ce314c295d239b76de34c8714def2c32d5) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Remove bad commit hunk from old cvs fetcherRichard Purdie2011-02-101-11/+0
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Correct the clean() mechanism for the fetcher2 codeSaul Wold2011-02-101-0/+11
| | | | | | | This create a clean() method in each of the fetcher modules and correctly cleans the .done stamp file and lock files Signed-off-by: Saul Wold <sgw@linux.intel.com>
* Export KRB5CCNAME variableJavier Martin2011-02-101-1/+1
| | | | | | | | | | This allows fetching git repositories using Kerberos authentication. (Bitbake rev: d761cf98284b02eb3d3a1f879782c501c284b698) Signed-off-by: Javier Martin <javier.martin@vista-silicon.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix comparison with SRCREVINACTION constantJavier Martin2011-02-101-1/+1
| | | | | | | | | | | Use '==' instead of 'is', otherwise it will always return true since 'rev' and "SRCREVINACTION" are not the same object. (Bitbake rev: f30b3af975a071d1584817054a2996f08a3aba4f) Signed-off-by: Javier Martin <javier.martin@vista-silicon.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2Yu Ke2011-01-251-0/+2
| | | | | | | | there is case that we need to distingush bb.fetch and bb.fetch2, and use different API for bb.fetch and bb.fetch2. so it is necessary to add version info for distinguish purpose Signed-off-by: Yu Ke <ke.yu@intel.com>
* *: use utils.remove() some moreBernhard Reutner-Fischer2011-01-122-10/+5
| | | | | | | (Bitbake rev: d3489b141cac1197324661680fe38b8a88bc49b4) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fetcher: break the "SRCREVINACTION" deadlockYu Ke2011-01-101-0/+3
| | | | | | | | | | | | | | | | | | | | | Current fetcher has annoying "SRCREVINACTION" deadlock, which occurs when SRCREV=${AUTOREV}=@bb.fetch.get_srcrev(): get_srcrev()->setup_localpath()->srcrev_internal_helper() ->evaluate SRCREV->get_srcrev() current fetcher resolve the deadlock by introducing a "SRCREVINACTION" condition check. Althoguh it works, it is indeed not clean. This patch use antoehr idea to break the deadlock: break the dependency among SRCREV and get_srcrev(), i.e. assign a specific keyword "AUTOINC" to AUTOREV. when Fetcher meet this keyword, it will check and set the latest revision to urldata.revision. get_srcrev later can use the urldata.revision for value evaluation(SRCPV etc). In this case, SRCREV no longer depends on get_srcrev, and there is not deadlock anymore. Signed-off-by: Yu Ke <ke.yu@intel.com>
* bitbake/fetch/git: Add backwards compatibility code for branch name handlingRichard Purdie2011-01-101-2/+80
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/fetch: Sync up logger/whitespace improvementsRichard Purdie2011-01-101-10/+13
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Update persist_data usage to the new APIChris Larson2011-01-051-18/+15
| | | | | | | (Bitbake rev: 9723a1c474b72b096c5a3136bf446ed69f3a749e) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Revert "persist_data: cache connection and use cursor"Chris Larson2011-01-051-5/+4
| | | | | | | | | | | | | Caching the database connection can cause serious issues if it results in multiple processes (e.g. multiple tasks) simultaneously using the same connection. This reverts commit 8a6876752b90efd81d92f0947bfc9527d8260969. (Bitbake rev: 60b9b18eafad5ac46c7cf1048d749d673c2ee0ad) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bzr: use utils.remove instead of os.systemBernhard Reutner-Fischer2011-01-041-1/+1
| | | | | | | (Bitbake rev: 0ef8a9b3780525a28567ebe42956e5cae79ada8c) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetchers: Add parameter scmdata=keep to include .git/ and others in ↵Andreas Oberritter2011-01-046-11/+48
| | | | | | | | | | | | | | generated tarballs. * Allows generating version information from SCMs during build. * Note that tar doesn't need to use --exclude '.git', because git checkout-index doesn't clone the repository. (Bitbake rev: 05cbc1d1a01c667c77688f36fbc5b61c5f452a3a) Signed-off-by: Andreas Oberritter <obi@opendreambox.org> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Rename command events, adjust compareRevisionsChris Larson2011-01-041-1/+1
| | | | | | | | | | | | - Moved the logic for comparing revisions from cooker into command - Removed 'Cooker' from the event names - Renamed the 'ExitCode' event into CommandExit, and changed CommandFailed to be a subclass of CommandExit (Bitbake rev: c51ed5d7a9971fad6019dac6c35a71b8a54ab16a) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Revert "svn fetcher: warn people to switch to SRCREV"Chris Larson2011-01-041-2/+1
| | | | | | | | | 1) too spammy 2) can be implemented in the metadata instead This reverts commit 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* svn fetcher: warn people to switch to SRCREVBernhard Reutner-Fischer2011-01-041-1/+2
| | | | | | | | | as noted by rp in ac00ca89a4e43cd4f38ba86455079d31be78e644 (Bitbake rev: 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: be more pythonicBernhard Reutner-Fischer2011-01-049-64/+21
| | | | | | | | | no functional changes (Bitbake rev: e88834fb7c6821cc29c12d296f2edd51f6eb3746) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: use os.path.joinBernhard Reutner-Fischer2011-01-041-1/+1
| | | | | | | (Bitbake rev: c360b01df18d90a513a3d61d395f905102e7568e) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: add common helper _strip_leading_slashes()Bernhard Reutner-Fischer2011-01-046-22/+13
| | | | | | | | | | Several fetcher need a way to strip leading slashes off a local path. This helper-function consolidates all such occurances. (Bitbake rev: 823a02185ed109054c6c1ae366221aaed0353f24) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetch: rename suppports_srcrev to supports_srcrevBernhard Reutner-Fischer2011-01-046-8/+8
| | | | | | | | | osc had it already spelled correctly?! (Bitbake rev: b8bb4433de7a981c6826173e926ca34705c4ac70) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Allow mercurial fetcher to follow tipPeter Chubb2011-01-041-1/+8
| | | | | | | | | | | | | | | | | There are occasions when developing when I want a package always to grab the latest copy of a package. Witht eh CVS fetcher you can do this by setting the `date' tag to `now'. This patch adds similar functionality to the mercurial fetcher: if the revision to fetch is `tip' then always grab from the server, and don't use the cached tarball. Oh, and I fixed a typo in the Class comment. (Bitbake rev: 01b85608d8a37f8af66dfd80133e950120679079) Signed-off-by: Peter Chubb <peter.chubb@nicta.com.au> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: lib/bb/fetch/hg: fix fetching from a mercurial repositoryEric BENARD2011-01-041-0/+1
| | | | | | | | | | | | | | * without this fix, we get : updating working directory 74 files updated, 0 files merged, 0 files removed, 0 files unresolved abort: There is no Mercurial repository here (.hg not found)! (Bitbake rev: 75ea005ac8fc05b2b3afca803d77a6b5f558efee) Signed-off-by: Eric Bénard <eric@eukrea.com> Tested-by: Frans Meulenbroeks <fransmeulenbroeks@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* fetchers: Use tar --exclude pattern to remove SCM filesKhem Raj2011-01-045-5/+5
| | | | | | | | | | | | This option will exclude the SCM metadata from tar files. Tested with gcc where svn tar which used to be 156M for gcc 4.5 is now 77M (Bitbake rev: f264cb6d43472525ad787b0887764ea696ec52ba) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Switch bitbake internals to use logging directly rather than bb.msgChris Larson2011-01-0412-76/+78
| | | | | | | | | | | | | | We use a custom Logger subclass for our loggers This logger provides: - 'debug' method which accepts a debug level - 'plain' method which bypasses log formatting - 'verbose' method which is more detail than info, but less than debug (Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Allow %20 in a file name in the SRC_URIC Michael Sundius2011-01-041-1/+2
| | | | | | | | (Bitbake rev: f7c181a0f6ab0b4d33bf80a0e24a788de441f82b) Signed-off-by: C Michael Sundius <msundius@sundius.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Fix fetch errorRichard Purdie2010-12-211-3/+3
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Only checksum downloads when they're first downloaded and a ↵Richard Purdie2010-12-211-5/+5
| | | | | | checksum is present Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Add missing return so if a checksum isn't present, it isn't ↵Richard Purdie2010-12-201-0/+1
| | | | | | checked Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Checksum validity fixesRichard Purdie2010-12-201-11/+7
| | | | | | | | | If the checksum check failed, the .md5 stamp file would still have been created meaning subsequent builds would proceed with the corrupt file. Reorder the calls to avoid this. Also raise a specific error for the checksum not specified error case. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* FetchData: add SRC_URI checksumYu Ke2010-12-201-0/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch add the per-recipe SRC_URI checksum verification. - SRC_URI format The format of SRC_URI checksum follow OE definition: 1. SRC_URI has single src SRC_URI = "http://some.domain/file.tar.gz" SRC_URI[md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" 2. SRC_URI has multiple src, every src need specify name SRC_URI = "http://some.domain/file1.tar.gz;name=name1 \ http://some.domain/file2.tar.gz;name=name2 " SRC_URI[name1.md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[name1.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" SRC_URI[name2.md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[name2.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" - SRC_URI checking invocation: the checksum checking is invoked in do_fetch phase, so it can be invoked manually by # bitbake -f -c fetch <recipe_name> if recipes has no SRC_URI checksum item, bitbake will show warning: " WARNING: Missing SRC_URI checksum for xxxx.tar.gz, consider to add SRC_URI[md5sum] = "5c69f16d452b0bb3d44bc3c10556c072" SRC_URI[sha256sum] = "f4e0ada8d4d516bbb8600a3ee7d9046c9c79e38cd781df9ffc46d8f16acd1768" " thus recipe author can add it to recpie file after SRC_URI - control variable BB_STRICT_CHECKSUM when SRC_URI checksum is missing, this variable decide pass or not if BB_STRICT_CHECKSUM = "1", bitbake should fatal in this case, otherwise bitbake just pass Signed-off-by: Yu Ke <ke.yu@intel.com>
* bitbake/fetch: Allow checking of a single url at a time (massive sstate ↵Richard Purdie2010-11-131-2/+5
| | | | | | speed improvement) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Fix handling of mirrors when checking for url validityRichard Purdie2010-11-131-2/+4
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Revert "bitbake/fetch/local: Also check DL_DIR for files since they could ↵Richard Purdie2010-11-131-4/+0
| | | | | | | | already exists there" These changes were incorrect. This reverts commit ae98f7eacb9e61fe086d88dc694b4c651af9fee3.
* Revert "bitbake/fetch/local: Fix os.exists reference"Richard Purdie2010-11-131-1/+1
| | | | | | These changes were incorrect. This reverts commit f8e33979352528bb7c289e7c839605a5880e1e43.
* bitbake/fetch/local: Fix os.exists referenceRichard Purdie2010-11-131-1/+1
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch/local: Also check DL_DIR for files since they could already ↵Richard Purdie2010-11-131-0/+4
| | | | | | | | exists there [BUGID #533] Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: When fetchers return errors, ensure any partial download is ↵Richard Purdie2010-10-231-0/+9
| | | | | | cleared Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Make URL checking slightly less verbose (distracting with the ↵Richard Purdie2010-10-221-1/+1
| | | | | | sstate code) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch/git: Ensure fullclone repositories are fully fetchedRichard Purdie2010-10-191-1/+4
| | | | | | | | | | | The git fetcher was failing to pull in new branches into a git repository mirror tarball as the git fetch command being used didn't add new remote branches. This patch uses "git fetch --all" for fullclones to ensure any new remote branches are cloned correctly. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetcher: Deal with a ton of different bugsRichard Purdie2010-10-192-36/+38
| | | | | | | | | | | | | | | | | | | | | | The more we try and patch up the fetcher code, the more things break. The code blocks in question are practically unreadable and are full of corner cases where fetching could fail. In summary the issues noticed included: a) Always fetching strange broken urls from the premirror for "noclone" git repositories b) Not creating or rewriting .md5 stamp files inconsistently c) Always fetching git source mirror tarballs from the premirror even if they already exist but the checkout directory does now d) Passing "None" values to os.access() and os.path.extsts() checks under certain circumstances e) Not using fetched git mirror tarballs if the preexist and always try and fetch them. This patch rewrites the sections of code in question to be simpler and more readable, fixing the above problems and most likely other odd corner cases. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch/git.py: Fix git fetcher to correctly use mirror tarballsRichard Purdie2010-10-191-1/+5
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/fetch: Ensure SRCREV is still set correctly if the pn-X override has ↵Richard Purdie2010-10-091-0/+2
| | | | | | been expanded Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-24 05:44:09 +0000