summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch2
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch2/__init__: Disable pseudo in runfetchcmd()Peter Kjellerstedt2018-07-021-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a fetcher, e.g., git, is run when pseudo is active it will think it is running as root. If it in turn uses ssh (as git does), ssh too will think it is running as root. This will cause it to try to read root's ssh configuration from /root/.ssh which will fail. If ssh then needs to ask for credentials it will hang indefinitely as there is nowhere for it to ask the user for them (and even if there was it would not access the correct private keys). The solution to the above is to temporarily disable pseudo while executing any fetcher commands. There should be no reason for them to be executed under pseudo anyway so this should not be a problem. RP Ammendum: We finally did get more information about how to reproduce this problem, something needs to trigger bb.fetch2.get_srcrev() in a pseudo context, for example when AUTOREV is in use or the recipe doesn't have a defined SRCREV. That SRC_URI needs to be using protocol=ssh. This would trigger an ls-remote of the remote repo and if that happens under pseudo, the wrong ssh credentials may be attempted which can hang. [YOCTO #12464] (Bitbake rev: d0fb46eead1f23356d8c3bd53a85047521ef29a7) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fix import error for Python 3.6.5Tzu Hsiang Lin2018-07-022-3/+1
| | | | | | | | | | | | When running bitbake command with Python 3.6.5 always result in import error causing by the change of distutils module. This patch replaces the method to search executable in PATH by "/usr/bin/env <command>". (Bitbake rev: e4fcf4d9cef6816afac9eebb8d57c82cd1ceb293) Signed-off-by: Tzu Hsiang Lin <t9360341@ntut.org.tw> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix missing logger import in repo fetcherOleksandr Andrushchenko2017-12-041-0/+1
| | | | | | | | | | | | | | | | After cleaning deprecated API usage repo fetcher is missing logger as it was indirectly imported via deprecated bb.data. Fix this by importing logger directly. Fixes: 9752fd1c10b8 ("fetch2: don't use deprecated bb.data APIs") (Bitbake rev: 7ae321a9ede9fb0ee1a0794aa22815a593d1568d) Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f8e027d26603db2f1fe757dca767ea35d95174c7) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: add stub latest_versionstring to FetchMethodRoss Burton2017-09-111-0/+8
| | | | | | | (Bitbake rev: c45453d1f6bc7bcecd84f58e2f7d93d6bd1e8499) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Replace deprecated git branch parameter "--set-upstream"Andre Rosa2017-09-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | Since 2017-08-17 (git version 2.14.1.473.g3ec7d702a) using deprecated git branch parameter "--set-upstream" causes a fetcher error. Replace it by "--set-upstream-to". https://git.kernel.org/pub/scm/git/git.git/commit/?id=52668846ea2d41ffbd87cda7cb8e492dea9f2c4d says, it's deprecated since 2012-08-30 so hopefully all still supported host distributions have new enough git to support "--set-upstream-to". ERROR: PACKAGE do_unpack: Fetcher failure: ...; git -c core.fsyncobjectfiles=0 branch --set-upstream master origin/master failed with exit code 128, output: fatal: the '--set-upstream' option is no longer supported. Please use '--track' or '--set-upstream-to' instead. ERROR: PACKAGE do_unpack: Function failed: base_do_unpack (Bitbake rev: 2ab50074c1a6c56a8a178755de108447d7b7acaf) Signed-off-by: Andre Rosa <andre.rosa@lge.com> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/npm: add noverify parameter to skip lockdown/shrinkwrapPaul Eggleton2017-08-311-20/+21
| | | | | | | | | | | | | | | | When fetching source for the first time within scripts such as OpenEmbedded's recipetool, we don't want to be showing warnings about NPM_SHRINKWRAP or NPM_LOCKDOWN not being set since there's no way we could have set them in advance. Previously we were using ud.ignore_checksums to suppress these but since we are now using a more standard task-based path to fetch the source, we need to disable these through the metadata. Look for a "noverify" parameter set on the npm URL and skip the checks if it is set to "1". (Bitbake rev: 8c4b35d1e4d31bae9fddd129d5ba230acb72c3bb) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: don't mandate path element in encodeurl()Paul Eggleton2017-08-231-6/+5
| | | | | | | | | | | | | URLs do not have to have a path; currently our npm URLs don't, so encodeurl() needs to handle if the path element isn't specified. This fixes errors using OpenEmbedded's devtool add / recipetool create on an npm URL after OE-Core revision ecca596b75cfda2f798a0bdde75f4f774e23a95b that uses decodeurl() and encodeurl() to change URL parameter values. (Bitbake rev: d5cab2dbf5682d2fd08e58316a3bf39a10f63df2) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: wget: fix FusionForge workaroundRoss Burton2017-07-311-6/+3
| | | | | | | | | | | | | | | | | My previous assertion about FusionForge appears to have been wrong, or FusionForge has changed behaviour, or both. FusionForge now mandates that downloads have the Accept header set, despite that header being optional, and returns a 406 Not Acceptable error if it isn't set. As we were pretending that 406 was actually 405 (Moved) and tried to handle it as a redirect this results in an infinite loop until Python kills the recursion. Delete the handling of 406 as 405, and pass Accept: */* in the headers. (Bitbake rev: bb70ae0c9aac5ec688026d23a64ac0cac1947187) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fix checkstatus fallback to MIRRORSRoss Burton2017-07-311-3/+2
| | | | | | | | | | The checkstatus() code was expecting checkstatus to throw exceptions if it failed, but in general it should return False. (Bitbake rev: 57be5cc6228518e60f564570a39cebbeb6cf564e) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/fetch2/__init__.py: Fix unpack commentMark Hatle2017-07-271-1/+1
| | | | | | | (Bitbake rev: b8f477de204ab5d0680b2b7c42370d13395be46c) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/__init__.py: replace stray logger.warn() with logger.warning()Andre McCurdy2017-07-251-1/+1
| | | | | | | | | | | | | | Update stray usage of deprecated logger.warn(), which was introduced to fetch2/__init__.py after all other instances had been replaced by logger.warning(): http://git.openembedded.org/bitbake/commit/?id=5a53e7d7b017769a6eb0f0a6335735a1fe51a5ec http://git.openembedded.org/bitbake/commit/?id=676a5f592e8507e81b8f748d58acfea7572f8796 (Bitbake rev: 1b14f115a9e929e29e91e8ac70826a3fe7259961) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/wget: mitigate a wget race condition when listing FTP directoriesRoss Burton2017-07-251-12/+11
| | | | | | | | | | | | | | | | | | | | | | When wget is fetching a listing for a directory over FTP it writes to a temporary file called .listing in the current directory. If there are many such operations happening in parallel - for example during 'bitbake world -c checkpkg' - then up to BB_NUMBER_THREADS instances of wget will be racing to write to, read, and delete the same file. This results in various failures such as the file disappearing before wget has processed it or the file changing contents, which causes checkpkg to randomly fail. Mitigate the race condition by creating a temporary directory to run wget in when doing directory listings. [ YOCTO #11828 ] (Bitbake rev: 91d4ca93df092cf86ab84faaa94cc66ff9f43057) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget.py: improve error handling during sstate checkPatrick Ohly2017-07-211-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the sstate is accessed via HTTP, the existence check can fail due to network issues, in which case bitbake silently continues without sstate. One such network issue is an HTTP server like Python's own SimpleHTTP which closes the TCP connection despite an explicit "Keep-Alive" in the HTTP request header. The server does that without a "close" in the HTTP response header, so the socket remains in the connection cache, leading to "urlopen failed: <urlopen error [Errno 9] Bad file descriptor>" (only visible in "bitbake -D -D" output) when trying to use the cached connection again. The connection might also get closed for other reasons (proxy, timeouts, etc.), so this is something that the client should be able to handle. This is achieved by checking for the error, removing the bad connection, and letting the check_status() method try again with a new connection. It is necessary to let the second attempt fail permanently, because bad proxy setups have been observed to also lead to such broken connections. In that case, we need to abort for real after trying twice, otherwise a build would just hang forever. [YOCTO #11782] (Bitbake rev: 6fa07752bbd3ac345cd8617da49a70e0b2dd565f) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fire an event when there are missing checksumsPaul Eggleton2017-07-211-0/+10
| | | | | | | | | | | If BB_STRICT_CHECKSUMS is set to anything other than "1" i.e. we're not going to raise an error, then fire an event so that scripts can listen for it and get the checksums. (Bitbake rev: 8b2ccb4b865f2df118ef668847df682a83f9c500) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: allow hiding checksum warningPaul Eggleton2017-07-211-0/+6
| | | | | | | | | | | | | If BB_STRICT_CHECKSUMS is set to "ignore" then don't display a warning if no checksums are specified in the recipe. This is not intended to be used from recipes - it is needed when we move to using more standard code paths to fetch new files from scripts i.e. where we don't know what the checksums are in advance. (Bitbake rev: f15ca7339de8a448a93a14cf6130b3925178a920) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fix indentationPaul Eggleton2017-07-181-47/+47
| | | | | | | (Bitbake rev: 6683338598ce97278f188fbcd780c3e3754e5b9a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: npm fetcher: fix unknown variable name.Ismo Puustinen2017-06-291-1/+1
| | | | | | | | | 'mirrortarball' is supposed to be a local variable to the function. (Bitbake rev: a457cbfb1f20a47db3978290921d0708cd96bd70) Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: fix handling of files with incorrect checksums from a premirrorJoshua Lock2017-06-221-0/+9
| | | | | | | | | | | Ensure that when an item fetched from a premirror has an invalid checksum the fetcher falls back to the usual logic of trying the upstream and any configured mirrors. (Bitbake rev: 022adb30dbb0df764c9fb515918cb9a88e4f8d6f) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for removing arbitrary revs for shallowChristopher Larson2017-06-021-1/+17
| | | | | | | | | | | | | | | | | | | | In certain cases, it's valuable to be able to exert more control over what history is removed, beyond srcrev+depth. As one example, you can remove most of the upstream kernel history from a kernel repository, keeping predominently the non-publically-accessible content. If the repository is private, the history in that repo couldn't be restored via `git fetch --unshallow`, but upstream history could be. Example usage: # Remove only these revs, not at a particular depth BB_GIT_SHALLOW_DEPTH_pn-linux-foo = "0" BB_GIT_SHALLOW_REVS_pn-linux-foo = "v4.1" (Bitbake rev: 97f856f0455d014ea34c28b1c25f09e13cdc851b) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/gitannex: add support for shallow mirror tarballsChristopher Larson2017-06-021-3/+20
| | | | | | | | | | | | When we're building from a shallow mirror tarball, we don't want to do anything with ud.clonedir, as it's not being used when we unpack. As such, disable updating annex in that case. Also include annex files in the shallow tarball. (Bitbake rev: ca0dd3c95502b22c369fbf37f915f45e02c06887) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/gitsm: add support for shallow mirror tarballsChristopher Larson2017-06-021-6/+11
| | | | | | | | | | | | | When we're building from a shallow mirror tarball, we don't want to do anything with ud.clonedir, as it's not being used when we unpack. As such, disable updating the submodules in that case. Also include the repositories in .git/modules in the shallow tarball. It does not actually make the submodule repositories shallow at this time. (Bitbake rev: 6c0613f1f2f9d4f009545f82a9173e80396f9d34) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for keeping extra refs for shallowChristopher Larson2017-06-021-1/+21
| | | | | | | | | | | | | | | | | | | | By default, all unused refs (branches & tags) are removed from the repository, as shallow processing scales with the number of refs it has to process. Add the ability to explicitly specify additional refs to keep. This is particularly useful for recipes with custom checkout processes, or whose git-based versioning requires a tag be available (i.e. for `git describe --tags`). The new `BB_GIT_SHALLOW_EXTRA_REFS` variable is a space-separated list of refs, fully specified, and support wildcards. Example usages: BB_GIT_SHALLOW_EXTRA_REFS = "refs/tags/v1.0" BB_GIT_SHALLOW_EXTRA_REFS += "refs/heads/*" (Bitbake rev: 1771934cd9f8b5847c6fcae0a906fb99d6b0db16) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: support per-branch/per-url depths for shallowChristopher Larson2017-06-021-21/+40
| | | | | | | | | | | | | | | | Allow the user to explicitly adjust the depth for named urls/branches. The un-suffixed BB_GIT_SHALLOW_DEPTH is used as the default. Example usage: BB_GIT_SHALLOW_DEPTH = "1" BB_GIT_SHALLOW_DEPTH_doc = "0" BB_GIT_SHALLOW_DEPTH_meta = "0" (Bitbake rev: 9dfc517e5bcc6dd203a0ad685cc884676d2984c4) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for shallow mirror tarballsChristopher Larson2017-06-021-15/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds support to the git fetcher for fetching, using, and generating mirror tarballs of shallow git repositories. The external git-make-shallow script is used for shallow mirror tarball creation. This implements support for shallow mirror tarballs, not shallow clones. Supporting shallow clones directly is not really doable for us, as we'd need to hardcode the depth between branch HEAD and the SRCREV, and that depth would change as the branch is updated. When BB_GIT_SHALLOW is enabled, we will always attempt to fetch a shallow mirror tarball. If the shallow mirror tarball cannot be fetched, it will try to fetch the full mirror tarball and use that. If a shallow tarball is to be used, it will be unpacked directly at `do_unpack` time, rather than extracting it to DL_DIR at `do_fetch` time and cloning from there, to keep things simple. There's no value in keeping a shallow repository in DL_DIR, and dealing with the state for when to convert the clonedir to/from shallow is not worthwhile. To clarify when shallow is used vs a real repository, a current clone is preferred to either tarball, a shallow tarball is preferred to an out of date clone, and a missing clone will use either tarball (attempting the shallow one first). All referenced branches are truncated to SRCREV (that is, commits *after* SRCREV but before HEAD are removed) to further shrink the repository. By default, the shallow construction process removes all unused refs (branches/tags) from the repository, other than those referenced by the URL. Example usage: BB_GIT_SHALLOW ?= "1" # Keep only the top commit BB_GIT_SHALLOW_DEPTH ?= "1" # This defaults to enabled if both BB_GIT_SHALLOW and # BB_GENERATE_MIRROR_TARBALLS are enabled BB_GENERATE_SHALLOW_TARBALLS ?= "1" (Bitbake rev: 5ed7d85fda7c671be10ec24d7981b87a7d0d3366) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: support multiple mirror tarball filenamesChristopher Larson2017-06-024-42/+47
| | | | | | | | | | | | Remove ud.mirrortarball in favor of ud.mirrortarballs. Each tarball will be attempted, in order, and the first available will be used. This is needed for git shallow mirror tarball support, as we want to be able to use either a shallow or full mirror tarball. (Bitbake rev: 02eebee6709e57b523862257f75929e64f16d6b0) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/fetch2: fix typo in conflicting revision messagePaul Eggleton2017-05-151-1/+1
| | | | | | | | | spcify -> specify (Bitbake rev: c46f183b3ea94789dd83cb280a1bf5735189c9bc) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/local: Drop FILESDIRRichard Purdie2017-04-051-9/+0
| | | | | | | | | | This has long since been deprecated and is no longer used anywhere, FILESPATH is the commonly used varaible which offers much more flexibility. Remove the FILESDIR code and references from bitbake. (Bitbake rev: 751c9dc51fd01fa64a1ff37ba2638110335f71af) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Do not fail to create symbolic links if they already existPeter Kjellerstedt2017-04-011-2/+13
| | | | | | | | | | | | | | | When the fetcher retrieves file:// URLs, there is no lock file being used. This means that in case two separate tasks (typically from two concurrent invocations of bitbake) want to download the same file:// URL at the same time, there is a very small chance that they also end up wanting to create a symbolic link to the file at the same time. This would previously lead to one of the tasks failing as the other task would have created the link. (Bitbake rev: 58a03531c8183b165bb7dcad86d8559c92bc150d) Signed-off-by: Peter Kjellerstedt <pkj@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: wget: Fix handling of urls with user/passwordRichard Purdie2017-03-311-3/+2
| | | | | | | | | | | | | URL decoding was improved in the core a while ago and this looks like a leftover from those times which caused urls needing a user/password to fail. Use the parameters from the core instead of the broken split implementation. [YOCTO #11262] (Bitbake rev: 6a917ec99d659e684b15fa8af94c325172676062) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: s3.py: Remove hardcoded awsElizabeth 'pidge' Flanagan2017-03-301-2/+4
| | | | | | | | | | | | | This commit looks to see if FETCHCMD_s3 is set and if not, sets it. This is needed because I've use cases where I don't use aws, but s3cmd (due to license). (Bitbake rev: fdeaed70a7d1ff8be1a1de937cb864130b0c2c86) Signed-off-by: Elizabeth 'pidge' Flanagan <pidge@toganlabs.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Ensure we don't have file downloads overwriting each otherRichard Purdie2017-03-291-1/+1
| | | | | | | | | | | | | | | | | | Imagine you have an sstate mirror accessed over http and an SSTATE_MIRRORS which maps file:// urls to http:// urls. File urls set donestampneeded = False, http urls don't. This can result in races in the try_mirror_url() code since it will trigger new downloads after aquiring the lockfile as verify_donestamp() doesn't look at origud and there is no donestamp. verify_donestamp() already has code to look at origud, we're just missing some code at the start of the function to do this. Fix it to avoid these races. (Bitbake rev: b8b14d975a254444461ba857fc6fb8c725de8874) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: prevent recursion on getting latest revisionPaul Eggleton2017-03-271-8/+20
| | | | | | | | | | | | | | | | | | We call git ls-remote to get the latest revision from a git repository, however by calling runfetchcmd() we can end up recursively running git ls-remote a number of times with OE e.g. if ${SRCPV} is in PV, ${PV} is in WORKDIR, and ${WORKDIR} is in PATH (as a result of recipe-specific sysroots), our call to runfetchcmd() exports PATH so _lsremote() will get called again - with the end result that we run git ls-remote 30 times in quick succession (!). Prevent that from happening by using a guard variable and returning a dummy value if it's called recursively. Fixes [YOCTO #11185]. (Bitbake rev: ff1ccd1db5d70b3fc9ad0d3e8f3d7b804c22bf36) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: fix FetchError referenceChristopher Larson2017-03-221-1/+1
| | | | | | | | | FetchError isn't defined, use bb.fetch2.FetchError in this context. (Bitbake rev: 945fa980e027753df2c21d84eb63dcaddb2caaee) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: drop pointless os.path.join, workdir=Christopher Larson2017-03-221-2/+2
| | | | | | | | | | The touch of .done explicitly specifies the path, so there's no need for workdir=, and "os.path.join('.')" is identical to just '.'. (Bitbake rev: 955cbfdaa2400d15ec428b65848e6835c9f44860) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: kill pointless quotes around single % argsChristopher Larson2017-03-221-5/+5
| | | | | | | (Bitbake rev: 4aebf12153369364eae2e6e773e2a921e9c91f72) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: use enumerate for ud.namesChristopher Larson2017-03-221-2/+2
| | | | | | | | | | list.index() isn't a particularly efficient operation, so keep track of our position via enumerate() instead, which is more pythonic as well. (Bitbake rev: dec6e90a4d27ee335e9c78aeebd277098fec94d1) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: pass --no-same-owner when extracting tar files compressed ↵Andre McCurdy2017-03-041-4/+4
| | | | | | | | | | | | with 7-Zip Treat tar files compressed with 7-Zip in the same way as tar files compressed with other compression formats. (Bitbake rev: 363a0f54dc7d9930537f0df25173fa31ca1f98ac) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: remove last remaining usage of deprecated bb.data APIsAndre McCurdy2017-03-042-3/+1
| | | | | | | (Bitbake rev: 62dc3c3b40a95f9df31f5c917574c3822fb8bb64) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: add initial Amazon AWS S3 fetcherAndre McCurdy2017-03-032-2/+100
| | | | | | | | | | | | | | Class for fetching files from Amazon S3 using the AWS Command Line Interface. The aws tool must be correctly installed and configured prior to use. The class supports both download() and checkstatus(), which therefore allows S3 mirrors to be used for SSTATE_MIRRORS. (Bitbake rev: 6fe07ed25457dd7952b60f4b2153d56b15d5eea6) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Allow whitespace only mirror entriesRichard Purdie2017-03-031-1/+5
| | | | | | | | | Forcing the use of "\n" in mirror variables is pointless, we can just require that there are pairs of values. (Bitbake rev: 044fb04dbe69313ee6908bf4d3cee7f797d0c41c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: don't use deprecated bb.data APIsAndre McCurdy2017-03-0115-26/+11
| | | | | | | | | Cleanup some more usage of bb.data APIs in the fetchers. (Bitbake rev: 9752fd1c10b8fcc819822fa6eabc2c1050fcc03b) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: wget.py: match .lz tarballs as well when determining latest versionAlexander Kanavin2017-03-011-1/+1
| | | | | | | (Bitbake rev: c55e09fbd2b64a2b909d9992350b6b2e26e5e86d) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/fetch2: rewrite / to - in package namesAnders Darander2017-03-011-2/+3
| | | | | | | | | | Most angular2 packages have names of the form @angular/xxx. The / obviously can't be used in a file name, replace it with -. (Bitbake rev: d3bd41d0ec9621307c362b394872b18b8b7ed8d6) Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/__init__: handle @ in package namesAnders Darander2017-03-011-1/+1
| | | | | | | | | | A number of npm packages use @ as a leading chararacter. Examples are most of the angular2 packages. (Bitbake rev: 628c4bf6c89b3d62c9b864380b5c8e131a899bff) Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/svn: change 'rsh' parameter to 'ssh'Markus Lehtonen2017-02-231-5/+5
| | | | | | | | | | | | | | | | | | | The functionality around the 'rsh' parameter seemed to be broken. The 'rsh' parameter was only used when 'protocol' was set to 'svn+ssh' which is confusing. The 'rsh' parameter was used for setting the value of 'svn_RSH' environment variable, which however, is not supported by svn (not at least according to SVN documentation). This patch removes the 'rsh' parameter and replaces it with 'ssh'. This new (optional) parameter is used when svn+ssh protocol is used and it can be used to specify the ssh program used by svn. This is achieved by setting the SVN_SSH environment variable which is mentioned in SVN documentation. (Bitbake rev: 5b364b02270b0d7c2b7ca8d67fa2731bf93720ee) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Add NFS Stale file exception handlingKhem Raj2017-02-191-0/+12
| | | | | | | | | | | | | | | | | | | | - In some cases the file descriptor is held by nfs client and none of os.path.* is catching that, it could mean that error is not doled out because client has cached the stat info. In this case we are out of luck. Needed to catch IOError, which would be causing the Stale error. - In download method, update_stamp is invoked md5sum validation which is found to be throwing Stale errors. - Added error handling to fix the stale errors. (Bitbake rev: 5a53e7d7b017769a6eb0f0a6335735a1fe51a5ec) Signed-off-by: Balaji Punnuru <balaji_punnuru@cable.comcast.com> Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Rename "setup_revisons" to "setup_revisions"Robert P. J. Day2017-02-196-6/+6
| | | | | | | | | | For spelling's sake, rename Python routine "setup_revisons" to "setup_revisions." (Bitbake rev: 4df59b027c02ef39d72476251ccd3fd62fc20bf6) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Correct two examples of typo "revsion."Robert P. J. Day2017-02-192-2/+2
| | | | | | | (Bitbake rev: 05f5421b2e44cd58c5912848de43d5884d070150) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/npm: fix handling of os fieldPaul Eggleton2017-02-071-1/+6
| | | | | | | | | | | | | | | | | When I originally added this check I didn't quite understand how the values in this field should be expressed - it seems from reading the documentation if there is an entry starting with '!' then the list is a blacklist and we shouldn't expect "linux" to be in the list, or we'll end up skipping important dependencies. This fixes fetching the "statsd" npm package. Fixes [YOCTO #10760]. (Bitbake rev: 7aa6d1586417e0e7d9925917a82caee5884957db) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/npm: handle items only in optionalDependenciesPaul Eggleton2017-02-071-0/+1
| | | | | | | | | | | | | | | | An npm package.json file has two dependency fields: dependencies and optionalDependencies. An item in optionalDependencies *may* also be listed in dependencies, but this is not required (and not necessary since if it's in optionalDependencies it will be optional, adding it to dependencies won't do anything). The code here was assuming that an optional dependency would always be in both, that's probably because that was true of the examples I was looking at at the time. To fix it, just add the optional ones to the list we're iterating over. (Bitbake rev: c0c50d43266150a80be31ae2c6fcaf37f5ba231d) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-09 16:12:18 +0000