linux/poky.git/bitbake/lib/bb/fetch2/git.py, branch yocto-3.1.20

bitbake: fetch/git: Fix usehead for non-default names

2022-07-15T11:41:59+00:00

The usehead url parameter for git repositories causes bitbake to use whatever commit the repository HEAD is pointing to if the repository happens to have the name 'default'. This is the default name so in many cases it works just fine, but if a different name is specified with the url parameter 'name=newName' then it will fail to parse the recipe with an error along the lines of: ERROR: ExpansionError during parsing /path/to/my/recipe.bb Traceback (most recent call last): File "/path/to/poky/bitbake/lib/bb/fetch2/git.py", line 235, in Git.urldata_init: > ud.setup_revisions(d) File "/path/to/poky/bitbake/lib/bb/fetch2/__init__.py", line 1302, in FetchData.setup_revisions: for name in self.names: > self.revisions[name] = srcrev_internal_helper(self, d, name) File "/path/to/poky/bitbake/lib/bb/fetch2/__init__.py", line 1167, in srcrev_internal_helper(name='newName'): if srcrev == "AUTOINC": > srcrev = ud.method.latest_revision(ud, d, name) File "/path/to/poky/bitbake/lib/bb/fetch2/__init__.py", line 1562, in Git.latest_revision(name='newName'): except KeyError: > revs[key] = rev = self._latest_revision(ud, d, name) return rev File "/path/to/poky/bitbake/lib/bb/fetch2/git.py", line 650, in Git._latest_revision(name='newName'): raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git repository in git ls-remote output for %s" % \ > (ud.unresolvedrev[name], ud.host+ud.path)) bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve 'master' in upstream git repository in git ls-remote output for /path/to/local/git/repo Let's fix this by setting the unresolved rev of _all_ repository names to 'HEAD' when the usehead url parameter is specified. Update the currently failing test, test_local_gitfetch_usehead_withname, to now expect success. This change preserves existing behavior that allows usehead to be overridden by a valid looking revision if one happens to be specified instead of AUTOREV. (Bitbake rev: a247f56df680382d62910bb9a174e0fdd29e4ca8) Signed-off-by: Joey Degges Signed-off-by: Richard Purdie (cherry picked from commit 01e901c44ab0f496606b1d45c8953dc54970204c) Signed-off-by: Paulo Neves Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch/git: Handle github dropping git:// support

2021-11-03T11:31:43+00:00

github is dropping support for git protocol in Git urls. Add code to remap this to https in a way that could be used in older bitbake versions. (Bitbake rev: c222eddcebe892ae209aea7776cfc1147ac1df6e) Signed-off-by: Richard Purdie

bitbake: fetch2/git: Use os.rename instead of mv

2021-10-23T16:47:38+00:00

os.rename will overwrite the destination file if present so we can use this instead of the process call overhead. (Bitbake rev: c5b8a2fce98c362ea77d74a8bc472d01b739a98a) Signed-off-by: Richard Purdie (cherry picked from commit b3cccaa6a896c41d8c9be5eebc327f726542d16b) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch2/git: Avoid races over mirror tarball creation

2021-10-23T16:47:38+00:00

There is a potential race over the mirror tarballs where a partial git repo could be extracted causing fetcher failures if the tarball is being rewritten whilst another build accesses it. Create the mirror tarball atomically to avoid this. [YOCTO #14441] (Bitbake rev: e3da0ecbd282da060b52a4bcf3ed36497295fde0) Signed-off-by: Richard Purdie (cherry picked from commit 3250bc950c56bd7dd2114df26e5a8e13b04ceac8) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch/gitsm: Fix crash when using git LFS and submodules

2021-05-13T12:12:00+00:00

Gitsm fetcher crashes when cloning a repository that contains LFS files. This happens because the unpack method is called during download, but the submodules have not been downloaded yet at this point. This issue was introduced in this commit: 977b7268bf4fd425cb86d4a57500350c9b829162 [YOCTO #14283] (Bitbake rev: b4acaa314cd8bc86c50e14464ff01c2d110122c4) Signed-off-by: Niels Avonds Signed-off-by: Richard Purdie (cherry picked from commit 26caedc4d2e9b5a0f1d57f9291754a7f6c5e437e) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie (cherry picked from commit e05d79a6ed92c9ce17b90fd5fb6186898a7b3bf8) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch/git: download LFS content too during do_fetch

2021-02-05T12:18:38+00:00

Insert an explicit pass to fetch all blobs needed by Git LFS, during the fetch() function. This avoids the default behavior of Git LFS to wait until 'git checkout' to begin downloading the blobs pointed to by LFS records. Network access is not allowed at that point in the recipe's lifecycle. [YOCTO #14191] (Bitbake rev: 2351b496bb63b96920d4ae67bec816f00d510df2) Signed-off-by: Matt Hoosier Signed-off-by: Richard Purdie (cherry picked from commit 0efac66043662e7a2027192f50e92e982db2ba1c) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: git.py: Use the correct branch to check if the repository has LFS objects.

2021-02-05T12:18:38+00:00

Function "contains_lfs" was only looking at the master branch when searching for LFS content. LFS may be configured in specific branches only, so we need to use the correct branch. (Bitbake rev: 274f790ac9161187810df3242d3570261e407b11) Signed-off-by: Mauro Queiros Signed-off-by: Richard Purdie (cherry picked from commit 4fa67c2af830035a1ddedc14592ee25a15ebff22) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: git.py: LFS bitbake note should not be printed if need_lfs is not set.

2021-02-05T12:18:38+00:00

The message "Repository %s has LFS content but it is not being fetched" was being printed, even when Git-LFS was available and "lfs=1" was set. In those situations, we want to fetch LFS content, so that message would not make sense. (Bitbake rev: 2795c6150c0b3218716195c0d10ff2843b1ef695) Signed-off-by: Mauro Queiros Signed-off-by: Richard Purdie (cherry picked from commit 45028dfda5a29a34ab408cb3f11d72ae17963340) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: git.py: skip smudging if lfs=0 is set

2021-02-05T12:18:38+00:00

Git-LFS objects were being fetched even when lfs=0 was not set. This patch disables LFS smudging when lfs=0. That way, only the LFS pointers are downloaded during checkout. (Bitbake rev: 3d09608cabd404733b95ea92d5b00830ded77834) Signed-off-by: Mauro Queiros Signed-off-by: Richard Purdie (cherry picked from commit 646d86df7de774255246a3d7051c308e43eb257d) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: bitbake: fetch/git: add support for SRC_URI containing spaces in url

2020-10-13T08:40:10+00:00

Microsoft's TFS VCS system allows for spaces in a git repository url. An example of a valid url is: ssh://tfs-my-company.org:22/tfs/My Projects/FooBar This commit adds support for such urls by implementing two changes. Firstly, when bitbake makes a git command line call the url is surrounded by quotes so that the url, regardless of spaces, is treated as one argument. Secondly, additional parsing of various filepath variables, which are based off of the url, are now completed with any spaces in the url replaced with underscores. (Bitbake rev: c4a51b7f4fd8a3c7d63c184675c014ec955d2606) Signed-off-by: Charlie Davies Signed-off-by: Richard Purdie (cherry picked from commit eb38b6f0935763f7ba19e5618f376fcae1dac41a) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie