linux/poky.git/bitbake/lib/bb/fetch2, branch uninative-2.1

bitbake: fetch/gitsm: avoid live submodule fetching during unpack()

2018-06-07T07:53:19+00:00

Although the submodules' histories have been fetched during the do_fetch() phase, the mechanics used to clone the workdir copy of the repo haven't been transferring the actual .git/modules directory from the repo fetched into downloads/ during the fetch task. Fix that, and for good measure also explicitly tell Git to avoid hitting the network during do_unpack() of the submodules. [YOCTO #12739] (Bitbake rev: 11b6a5d5c1b1bb0ce0c5bb3983610d13a3e8f84a) Signed-off-by: Matt Hoosier Signed-off-by: Richard Purdie

bitbake: fetch2: fix import error for Python 3.6.5

2018-05-29T20:07:46+00:00

When running bitbake command with Python 3.6.5 always result in import error causing by the change of distutils module. This patch replaces the method to search executable in PATH by "/usr/bin/env ". (Bitbake rev: bd9a1b063633af2936ba1dd87b19202424900151) Signed-off-by: Tzu Hsiang Lin Signed-off-by: Richard Purdie

bitbake: fetch2/init.py: Fixed handling of uris with empty path

2018-05-22T12:13:19+00:00

For mirrors or premirrors defined like: "http://.*/.* http://somewhere.org" fetching ends with errors because function fetch2/__init__.py:encodeurl() creates url like "http://somewhere.orgsomefile.tar.gz". It happens because function fetch2/__init__.py:decodeurl() for url "http://somewhere.org" returns ['http', 'somewhere.org', '', '', '', {}] and then in function fetch2/__init__.py:uri_replace() variable result_decode will be ['http', 'somewhere.org', 'somefile.tar.gz', '', '', {}] (because of line: result_decoded[loc] = os.path.join(result_decoded[loc], basename)) for which encodeurl returns "http://somewhere.orgsomefile.tar.gz". In addition for mirror "http://.*/.* http://somewhere.org/" everything works fine. (Bitbake rev: d822ae24ef5485e550804cbd9130ebd73b2aa48e) Signed-off-by: Jakub Dębski Signed-off-by: Richard Purdie

bitbake: fetch/git: make fewer calls to _contains_ref() from download()

2018-05-22T12:13:19+00:00

Updating a local git repo clone currently results in multiple calls to self._contains_ref(), some of which appear to be redundant and can be eliminated by minor tweaks to the logic in download(). Also drop redundant calls to os.path.exists(ud.clonedir) before self.need_update(), since need_update() includes its own built-in check for the existance of ud.clonedir. (Bitbake rev: 61b0df5523afc8f805043f3adc9c106690e6f133) Signed-off-by: Andre McCurdy Signed-off-by: Richard Purdie

bitbake: fetch2/git: log exception if ls-remote fails

2018-03-28T11:51:52+00:00

(Bitbake rev: 8212446de11c0e370c55f88cde86334b760cd939) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: fetch2: Handle missing donestamp file when content is valid

2018-02-14T15:26:03+00:00

In order to allow users to manually populate the download directory with valid content change the assumption that missing the donestamp file means unfetched content. This allows users to populate the download dir without needing to create dummy .done files such that a user does not need a PREMIRROR when using BB_NO_NETWORK to provide valid content files in the download directory. To ensure the correct result this change also fails first if the localpath does not exist. This prevents further parts of the function attempting to calculating the checksum on non-existent files. This also fixes some edge conditions around where if the donestamp exists but the localpath does not it returns, and did not remove the donestamp. Also added test cases to cover this use case and additional use cases where for example the fetcher does not support checksums. (Bitbake rev: a335dbbb65d5b56e71d98cf3e4fa9bfbec1dcde6) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

bitbake: fetch2/npm.py: Fix inverted condition to prevent infinite loop

2018-02-14T15:26:03+00:00

At least the cli-color node module has dependencies that have cyclic dependency among themselves. npm.py is prepared to deal with such a case but the condition is handled only for downloading or not a dependency again, but then it goes checking the its dependency which causes an infinite loop in _getdependencies(). Make this function simply return when a dependency is already downloaded and only download and check its dependencies when not. (Bitbake rev: 545540420112992e53f4a83104af10452df168d0) Signed-off-by: Zoltán Böszörményi Signed-off-by: Richard Purdie

bitbake: fetch/git: Use 'git-make-shallow' from bin directory

2018-02-14T15:26:03+00:00

Move the code that existed in tests/fetch.py for determining the path to 'git-make-shallow' into the git module and reference it. This ensures that 'git-make-shallow' is always available and the desired version regardless of the path variable or whether git exposes the command. (Bitbake rev: 6b508ab8fd5aa796c1c00c970e81e5e93f84d35d) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

bitbake: fetch/git: Add pack-refs command to mirror handling code

2018-01-29T08:50:08+00:00

We've noticed issues on our infrastucture iterating over the many tag/branch/head reference files that some git repositories may contain. By issuing the pack-refs command, we move these all to a single file which speeds up operations with the mirror repos in the downloads directory in general. (Bitbake rev: f8126aaf774186a6eaf0bd4067b89c074594886c) Signed-off-by: Richard Purdie

bitbake: fetch2/init: Disable pseudo in runfetchcmd()

2018-01-08T08:48:54+00:00

If a fetcher, e.g., git, is run when pseudo is active it will think it is running as root. If it in turn uses ssh (as git does), ssh too will think it is running as root. This will cause it to try to read root's ssh configuration from /root/.ssh which will fail. If ssh then needs to ask for credentials it will hang indefinitely as there is nowhere for it to ask the user for them (and even if there was it would not access the correct private keys). The solution to the above is to temporarily disable pseudo while executing any fetcher commands. There should be no reason for them to be executed under pseudo anyway so this should not be a problem. RP Ammendum: We finally did get more information about how to reproduce this problem, something needs to trigger bb.fetch2.get_srcrev() in a pseudo context, for example when AUTOREV is in use or the recipe doesn't have a defined SRCREV. That SRC_URI needs to be using protocol=ssh. This would trigger an ls-remote of the remote repo and if that happens under pseudo, the wrong ssh credentials may be attempted which can hang. [YOCTO #12464] (Bitbake rev: ceaca281cafa662aa2385b95641bce309dce843d) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie

linux/poky.git/bitbake/lib/bb/fetch2, branch uninative-2.1

bitbake: fetch/gitsm: avoid live submodule fetching during unpack()

bitbake: fetch2: fix import error for Python 3.6.5

bitbake: fetch2/__init__.py: Fixed handling of uris with empty path

bitbake: fetch/git: make fewer calls to _contains_ref() from download()

bitbake: fetch2/git: log exception if ls-remote fails

bitbake: fetch2: Handle missing donestamp file when content is valid

bitbake: fetch2/npm.py: Fix inverted condition to prevent infinite loop

bitbake: fetch/git: Use 'git-make-shallow' from bin directory

bitbake: fetch/git: Add pack-refs command to mirror handling code

bitbake: fetch2/__init__: Disable pseudo in runfetchcmd()

bitbake: fetch2/init.py: Fixed handling of uris with empty path

bitbake: fetch2/init: Disable pseudo in runfetchcmd()