summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch2/__init__.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch2: improve handling of two classes of URL parameter mistakesPaul Eggleton2014-05-301-4/+10
| | | | | | | | | | | | | | Handle the following situations in a URL (e.g. in SRC_URI): * Trailing semicolon in a URL - this is now ignored. * Parameter specified with no value (no equals sign). This still produces an error, but at least it is MalformedUrl with a proper message rather than "ValueError: need more than 1 value to unpack". (Bitbake rev: bfd13dfbc4c9f1dd8315002271791b1d9e274989) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Fix refetch of local files when they are read onlyNick Lewis2014-05-081-2/+2
| | | | | | | (Bitbake rev: 42b7be1a6c5ff559f4a213eed04b370ca955460d) Signed-off-by: Nick Lewis <nick.lewis@usa.g4s.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Cleanup file checksum generation, v2Jacob Kroon2014-04-291-6/+2
| | | | | | | | | | | | | | | | | The initial version of this patch was reverted in 0a94e568152de550dedc8135a766beb18bf064ab However, it was later agreed upon that it was reverted due to a misunderstanding during code review; the patch is ok. This version of the patch also removes an unneeded initialization of the 'checksum' variable outside the scopes it is being used in. (Bitbake rev: 9ee19a3ca2f8e11a91f0289ea3486310c61d40f2) Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Revert "fetch2: Cleanup file checksum generation"Richard Purdie2014-04-281-2/+5
| | | | | | | | This reverts commit 6dfb33fd7204e09815bde7ba5bf21941a73fe965. It causes an unindeded change in behaviour. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Cleanup file checksum generationJacob Kroon2014-04-271-5/+2
| | | | | | | | | | | | Cleanup the fix done in f9416e76e272ba3249abff099f6f3a47fe82e03e. Instead of adding continue statements we can just move the last statements into the final else-clause. (Bitbake rev: 6dfb33fd7204e09815bde7ba5bf21941a73fe965) Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fix traceback when a wildcard matches a directoryPaul Eggleton2014-04-081-10/+18
| | | | | | | | | | If there is a directory matching a wildcard in SRC_URI when getting file checksums, we should recurse into that instead of producing an error. (Bitbake rev: ae87b7eb414e3d5eefd2effec7b30c22d2186b02) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: handle wildcards correctly when recording file checksumsPaul Eggleton2014-04-081-0/+3
| | | | | | | | | | | | | The Local fetcher's localpath is returning the parent directory for a wildcard match; we need to handle this and add the wildcard specification so that we checksum the correct files. Fixes [YOCTO #6127]. (Bitbake rev: 4a90edd62c16cbf41b5b93280e155077564c774a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: avoid cache ignoring missing filesPaul Eggleton2014-04-081-1/+0
| | | | | | | | | | | | | | | Previously, if a file listed in SRC_URI was missing at the time of parsing, and then was added and bitbake run again, it would not be picked up because the recipe was cached without it. If we allow the file to be added to the list of files to checksum, then it will be checked for and found on the second run. Fixes [YOCTO #4790]. (Bitbake rev: 71da822762cb298261cccdfa54b9c0fea02c3c5d) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix bug in file checksum generationRichard Purdie2014-04-061-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For a while its been puzzling me why connman-gnome rebuilds as often as it does. It turns out you can trigger this with a new checkout of the metadata. The SRC_URI that is causing the problems is: SRC_URI = "file://images/*" and rather oddly the results in checksums for a file "." being added to the tree, e.g.: ('.', 'ab48a68186f0e0f277c21ef4cb390b4b') The problem is that when iterating files lists, the checksum variable can become set yet we don't break the out from the for loop, which leads to odd (and non-deterministic) entries being added into the file checksum list. The exact item added probably depends on the order of items on the disk. Before this change, bitbake-diffsigs on connman-gnome:do_fetch would report: This task depends on the checksums of files: [ ('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'), ('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'), ('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'), ('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'), ('.', 'ab48a68186f0e0f277c21ef4cb390b4b')] Afterwards: This task depends on the checksums of files: [ ('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'), ('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'), ('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'), ('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85')] which is correct and deterministic without the "." entry. (Bitbake rev: f9416e76e272ba3249abff099f6f3a47fe82e03e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2.URI: Set username/password should not change the otherOlof Johansson2014-03-241-2/+3
| | | | | | | | | | | When setting the username after already having set the password, the password was unexpectedly reset. This change fixes this issue and introduces unit tests to make sure it doesn't happen again. (Bitbake rev: 25faef3a047f9c7564089463d7c96f6910b640cb) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2.URI: Support URIs with both query strings and paramsOlof Johansson2014-03-241-11/+5
| | | | | | | | | | | | | | | | | There is a case in meta-intel where a SRC_URI contains both a query string and URI parameter: https://edc.intel.com/Download.aspx?id=6190;downloadfilename=LIN_IEMGD_1_14_GOLD_2443.tgz Python's urlparse thought the URI parameters were part of the query parameter value, but in the bitbake context this is obviously not the case. As bitbake's usage isn't really RFC compliant, we have to extract and remove the URI parameters *before* urlparse sees the URI. (Bitbake rev: c2f27ae4271985b48f957747b6ea372c8699cb49) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2.URI: add support for query parametersOlof Johansson2014-03-241-15/+23
| | | | | | | | | | | | | This change introduces the .query property of the URI class. It is a read/write dict of the parameters supplied in the query string of the URI. E.g.: http://example.com/?foo=bar => .query = {'foo': 'bar'} (Bitbake rev: 1cb2b3c458c8c5521591d2c8f2e0058143fc77bb) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2.URI: Coerce urlparse to use netloc for all schemesOlof Johansson2014-03-241-6/+15
| | | | | | | (Bitbake rev: 4d502578f022bcf772780550c047b8c09ba01443) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Drop svk fetcherRichard Purdie2014-03-021-2/+0
| | | | | | | | | | The svk fetcher never appears to be used by anyone and the development on svk appears to have stopped in 2010. We might as well drop support for it. (Bitbake rev: 8239264753977bd06ad5b1b574245d3842af489b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix mirror repo tarball creationRichard Purdie2014-02-241-1/+1
| | | | | | | | | | | A typo was meaning that the mirror creation method wasn't being called when it should have been. Fix the type to fix mirror tarball creation. [YOCTO #5284] (Bitbake rev: 348971d410bfd5d8b1757468d73e1d24ae78a594) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Improve invalid SRCREV error messageRichard Purdie2014-01-211-10/+5
| | | | | | | | | The current message can be ambiguous, improve it (and also rename a variable to clean up the rest of the function). (Bitbake rev: 0c1bb7c0fce7b0f334311a2893ccb00385fa8d55) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Sanity check SRCREV matches rev/tag parameterRichard Purdie2014-01-211-6/+14
| | | | | | | | | | | | | | | Add a sanity check so that if some SRCREV is set and a rev parameter is given to the url, the revision given should match. Any tag parameter behaves the same as rev. If both are specified, error to tell the user we're confused rather than do something which may or may not be what they intended. Also add some unittests for this. (Bitbake rev: e82a4ab48991035866da9914c8b75a9bfbc9a7fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Clean up srcrev_internal_helperRichard Purdie2014-01-211-9/+16
| | | | | | | | | | Currently INVALID and None are checked as incorrect values under different circumstances. This code standardises those checks to be consistent. We should phase out the use of "INVALID". (Bitbake rev: 86ef4e65ce18b71dc69643586bd2aa8f48703171) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/__init__.py: let try_mirror_url return correct valueRobert Yang2014-01-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | The fetcher will try: 1) PREMIRROR 2) Upstream 3) MIRROR If it fails to download from the Upstream, but succeeds from the MIRROR, and ud.localpath != origud.localpath (for example, the git tarball), then we will get the error (e.g.: xf86-video-omapfb): ERROR: Function failed: Fetcher failure for URL: 'xxx'. Unable to fetch URL from any source. ERROR: Logfile of failure stored in: /path/to/log.do_fetch.28024 It should not show the error and let the build go on since it succeeds. (e.g.: xf86-video-omapfb) [YOCTO #5686] (Bitbake rev: c08ca1e4eeb04f78e1354780cf5a4c3855e49572) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Don't allow '/' in user:pass, fix branch containing '@'Martin Jansa2014-01-181-1/+1
| | | | | | | | | | | | | | * currently decode_url regexp parses branch=@foo as username so it ends like this: - ('git', '', 'foo', 'git.openembedded.org/bitbake;branch=', '', {}) + ('git', 'git.openembedded.org', '/bitbake', '', '', {'branch': '@foo'}) * http://hg.python.org/cpython/file/2.7/Lib/urlparse.py also assumes that there is at least one '/' as separator between netloc and path, params, so it looks reasonable to prevent including '/' in username (Bitbake rev: 2c82742114091cb55055328b54223686816582f2) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: gitannex.py: Add Git Annex supportOtavio Salvador2014-01-081-0/+2
| | | | | | | | | | | | | This add a Git Annex backend which reuses the Git fetcher code; it allows managing files with git, without checking the file contents into git, being useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, time, or disk space. (Bitbake rev: a61fc4db598e9d13c966712a6a0e4783e19448be) Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fetcher2: clean(): remove the .patch.doneRobert Yang2014-01-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | There was a problem: $ bitbake xf86-video-omapfb -cfetch && bitbake xf86-video-omapfb -ccleanall Everything should be removed, but the 0006-omapfb-port-to-new-xserver-video-API.patch.done still exists in the DL_DIR, this is because the clean() in the fetch2/__init__.py skips removing the local file, so that it will skip removing the .done. The local file (file://) isn't needed to be removed since it is not downloaded into DL_DIR, but the .done should be removed, this patch will remove the .done, and it doesn't remove anything else since the clean() in local.py does nothing. [YOCTO #5687] (Bitbake rev: 2bc99b9dfa532430a13c39fca4e5ef3a2206b3b8) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: replace 3 spaces with 4Robert Yang2014-01-021-4/+4
| | | | | | | | | | A part of fetch2/__init__.py uses 3 spaces as the indent, I think that they should be typos. (Bitbake rev: abafd85e2fcf23cee872e0e9e468898101430f1f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: avoid printing "no checksum" error message twicePaul Eggleton2013-12-221-5/+4
| | | | | | | | | | | | | | Because of the way we were handling this error, it was printed twice - once via logger.error() (to avoid the log being printed) and a second time when the exception gets wrapped in a FuncFailed at a higher level. Call logger.error() earlier and change the text we send in the exception to be more brief, so it more closely resembles the behaviour when there is an invalid checksum. (Bitbake rev: 46765369d7f76ec7f67b90430131a79eb6a66235) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fail checksum validation if SRC_URI checksums set to ""Paul Eggleton2013-12-221-3/+3
| | | | | | | | | | | | We were checking SRC_URI md5sum/sha256sum values against None here, so if they were set to "" then no error was produced. Since the value is still effectively unset in this case, this is not the right behaviour; just check if the value doesn't evaluate to False instead. (Bitbake rev: 040943a718795c64dc4e604abfcf08b26b7d00e6) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Stop passing around the pointless url parameterRichard Purdie2013-11-221-32/+32
| | | | | | | | | | | | | | | There is no good reason to keep passing around the url parameter when its contained within urldata (ud). This is left around due to legacy reasons, some functions take it, some don't and its time to cleanup. This is fetcher internal API, there are a tiny number of external users of the internal API (buildhistory and distrodata) which can be fixed up after this change. (Bitbake rev: 6a48474de9505a3700863f31839a7c53c5e18a8d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Clean up some url parameters to functionsRichard Purdie2013-11-221-19/+19
| | | | | | | | | | | | | urldata contains the url so we might as well stop passing around pointless function parameters. This was done for legacy reasons but its time to clean this mess up. This is a first step in cleanup and is a standalone patch but there is more to be done in a second patch. (Bitbake rev: 06590cfebbcf6565a17b80cc298e3ecdfaba4656) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix handling of SCM mirrors in MIRRORSRichard Purdie2013-11-191-0/+4
| | | | | | | | | | | | | | | If an SCM mirror is in PREMIRRORS, the tarball is downloaded and then found by the "upstream" check and handled correctly. If an SCM mirror is in MIRRORS, the tarball is downloaded but not used since there is no "upstream" run after MIRRORS completes. It therefore sits there useless and unused. This code change forces the upstream to run after a mirror tarball is found and fixes the usage of SCM mirrors in MIRRORS. (Bitbake rev: a66ee0994645aa5658b2f5ea134ed17d89f8751a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fix unpack of .xz filesAndré Draszik2013-07-311-1/+1
| | | | | | | | | | | | | | | | | | If a file ends with .xz, it currently gets overwritten during unpack: The decompress command for .xz files is: 'xz -dc %s > %s' % (file, efile) and as efile == file, we end up overwriting file (the source). Fix this by adding .xz to the list of suffixes that that need to be removed from a file name for an extract command, leaving the bare file name. Now, for a given file foo.xz, file == foo.xz and efile == foo, similar to how .gz .bz2 and .Z files are treated. (Bitbake rev: 2cd2d0a48e12ab4358fb967eaf7a56c17993f48d) Signed-off-by: André Draszik <andre.draszik@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Correct a typoPeter Kjellerstedt2013-06-201-1/+1
| | | | | | (Bitbake rev: 087ec3a1b2dde04a8e5ce2943802fccd424ab3f3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Allow checksums to be used for files retrieved using SFTPPeter Kjellerstedt2013-06-201-2/+2
| | | | | | (Bitbake rev: fd438a334d1f90ff07dded61c9648987da42c34f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Enhance error message for BB_NO_NETWORKRichard Purdie2013-06-201-1/+1
| | | | | | | | | | | | BB_NO_NETWORK can be set by bitbake internally by the use of BB_FETCH_PREMIRRORONLY so update the error message to give users a hint about this. [YOCTO #3222] (Bitbake rev: cac3060d0bf8c7deeacda18d06d92787911380d0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix AUTOINC handlingRichard Purdie2013-06-141-3/+3
| | | | | | | | | | | | | AUTOINC was meant to appear once at the start of the version string. The list of names may not be sorted meaning it could get inserted in the middle. This patch simplifies the code and ensures it appears at the start. Include cache version bump to ensure the cache picks up these changes. (Bitbake rev: ad8bf10d873abb94d987860a3f6d06b134fb8a99) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix missing function callRichard Purdie2013-05-301-14/+15
| | | | | | | | | | | | | | | | | | | | Fix the issue: File: '/srv/home/pokybuild/yocto-autobuilder-new/yocto-slave/build-appliance/build/bitbake/lib/bb/fetch2/__init__.py', lineno: 813, function: try_mirror_url 0809: except bb.fetch2.BBFetchException as e: 0810: if isinstance(e, ChecksumError): 0811: logger.warn("Mirror checksum failure for url %s (original url: %s)\nCleaning and trying again." % (newuri, origud.url)) 0812: logger.warn(str(e)) *** 0813: self.rename_bad_checksum(ud, e.checksum) 0814: elif isinstance(e, NoChecksumError): 0815: raise 0816: else: 0817: logger.debug(1, "Mirror fetch failure for url %s (original url: %s)" % (newuri, origud.url)) Exception: NameError: global name 'self' is not defined (Bitbake rev: 5f960017bb1d4274a5bd5b598971f38c12fc0f11) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix missing assignmentRichard Purdie2013-05-241-1/+1
| | | | | | | | The assignment to True was missing from the code, well spotted Saul! (Bitbake rev: e493fe8cb4953935f01361ffc0240e5818ebb283) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Shorten long srcrevsRichard Purdie2013-05-221-0/+4
| | | | | | | | | | | | | | | | | | | | | The long srcrevs are mainly used or the workdir construction as well as the package version. The long entries are hashes generated by the git fetcher and other scms using a similar revision mechanism. We need these to change when the package changes however collisions are unlikely to happen within the domains we care about. The long revisions have generated negative user feedback due to the use in path and file names. This patch therefore truncates the revisions to 10 characters maximum. This should be safe in the contexts where these revisions are used as the chances of spatially close collisions is very low (distant collisions are not a major issue in the way we use these). (Bitbake rev: 43a8319cda7fae37862dae323eeb24cb39ca21b7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Clean up sortable_revisionRichard Purdie2013-05-221-17/+10
| | | | | | | | | | | Now we no longer try and provide increasing values from the fetcher, we can simplify the function structure for the sortable_revision pieces and move the AUTOINC handling directly into the function which needs it, simplifying the code. (Bitbake rev: fb068bee47bb1a06f02447daf16c2b2a79c03288) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/utils: Clean up importsRichard Purdie2013-05-171-7/+4
| | | | | | | | | Move various random imports to the start of the modules as cleanup and avoid an import issue with bb.process on python 2.6. (Bitbake rev: aed4adfbe3a591ca4f8e41fb763c9f961bf2e6d5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: rename file with bad checksum instead of removing it completelyMartin Jansa2013-04-151-1/+18
| | | | | | | | | | * this can be useful when someone wan't to compare old file with bad checksum and new one (Bitbake rev: 33c6b93597dd43ab03ce7b62ba3eeb1893a68c38) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/fetch: Add git submodules fetcherRichard Purdie2013-03-251-0/+2
| | | | | | | | | | | | | | | | | This adds very basic git submodule support to the fetcher. It can be used by replacing a git:// url prefix with a gitsm:// prefix, otherwise behaviour is the same as the git fetcher. Whilst this code should be functional, its not as efficient as the usual git fetcher due to the need to checkout the tree to fetch/update the submodule information. git doesn't support submodule operations on the bare clones the standard git fetcher uses which is also problematic. This code does however give a starting point to people wanting to use submodules. (Bitbake rev: 25e0b0bc50114f1fbf955de23cc0c96f5f7a41e3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Revert "fetch2: Adapt encode/decode url to use URI class"1.4_M4.rc11.4_M4.final1.4_M4Richard Purdie2013-02-191-17/+49
| | | | | | | | This reverts commit 21fe2683aefde10e847e66c11c26d4f4c1e07cfd since bitbake-selftest doesn't pass when this is applied and we're seeing multiple build failures from this change. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/fetch2: workaround urlparse in older python not support gitMatthew McClintock2013-02-191-0/+3
| | | | | | | (Bitbake rev: 7e479dc6a574a8f3bd9f24d2ed1c3ceef91f3828) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Add SFTP fetcherOlof Johansson2013-02-171-0/+2
| | | | | | | | | | | This fetcher differs from the SSH fetcher in that it adheres more strictly to the SECSH URI internet draft --- it uses the sftp:// instead of the ssh:// scheme, and it uses sftp instead of scp. (Bitbake rev: d240baeb7a4107d2eba3f08c411c0f086674d8e2) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Adapt encode/decode url to use URI classOlof Johansson2013-02-171-49/+17
| | | | | | | (Bitbake rev: 21fe2683aefde10e847e66c11c26d4f4c1e07cfd) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Add a class representing a generic URIOlof Johansson2013-02-171-0/+194
| | | | | | | | | | | | | | A class representing a generic URI, with methods for accessing the URI components, and stringifies to the URI. This class should be a bit more flexible than the existing {encode,decode}_url functions in that it supports more components (e.g. port) and that it does not rely on a specific order on the return values. This makes it easy to add new properties without affecting the API. (Bitbake rev: bd824da8a7eafe27310e410807319628378caeca) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Ensure directory for stampfile exists before trying to ↵Richard Purdie2013-02-151-0/+1
| | | | | | | | create it (Bitbake rev: cf510e3da36e53f98ca86501747364bf4699ecc1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Improve lock/done stamp file paths for local filesRichard Purdie2013-02-151-2/+9
| | | | | | | | | | | Currently stamps end up at the top level of DL_DIR even if the files themselves are in a directory structure. This patch preserves path components allowing the top level directory to be less populated which is an advantage for large sstate caches. (Bitbake rev: 59921ce3ed7a4c0b7f8ef1a101ad9127469bf1fd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Ensure expansions happen in URL parameters in uri_replaceRichard Purdie2013-02-151-0/+2
| | | | | | | | | For example, this allows us to specify downloadfilename=PATH in sstate mirror urls to improve directory structures in DL_DIR when using sstate. (Bitbake rev: 7850a1364b6b37c58664d84f9c14806b4479b45c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Remove broken git variables from the environmentDarren Hart2013-02-061-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following variables perform no function outside of bitbake: GIT_CONFIG GIT_PROXY_HOST GIT_PROXY_PORT GIT_PROXY_IGNORE GIT_CONFIG only affects the git-config command which is not relevant to the fetcher. This was previously used with the OE GIT_CORE_CONFIG variable which would provide a basic git config to use instead of the user's config. This usage was deprecated by git for over a year now: http://git.661346.n2.nabble.com/Overriding-gitconfig-using-GIT-CONFIG-td6680977.html GIT_PROXY_HOST and GIT_PROXY_PORT are not used by git. GIT_PROXY_IGNORE was an OE construct used to create the custom git config and had no meaning outside of the OE environment. It is not used by git. Remove these variables from the fetcher environment. Users wishing to configure git to work with a proxy should define the GIT_PROXY_COMMAND environment variable to use an external script. NO_PROXY can be used within this script to skip the proxy for certain hosts. (Bitbake rev: e60270bdce6b8c2f8da1a4838aa374da9db3c86a) Signed-off-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Export upper and lower case environment variablesDarren Hart2013-02-061-2/+7
| | | | | | | | | | | | | | | | | Applications are inconsistent in their use of upper and lower case proxy variables. Curl, for example, specifies NO_PROXY (not no_proxy) in the man page (changed in 2009 [1]). Avoid proxy issues by ensuring both the upper and lower case versions of each proxy variable is available in the environment for the fetcher commands. Add FTPS_PROXY and ftps_proxy to the list as well. 1. http://curl.haxx.se/mail/tracker-2009-04/0012.html (Bitbake rev: c3e6b2c5ec81d5ad7dcf606fff16fd5552bd267c) Signed-off-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-27 11:54:25 +0000