linux/poky.git/bitbake/lib/bb/fetch2/git.py, branch uninative-2.1

bitbake: fetch/git: make fewer calls to _contains_ref() from download()

2018-05-22T12:13:19+00:00

Updating a local git repo clone currently results in multiple calls to self._contains_ref(), some of which appear to be redundant and can be eliminated by minor tweaks to the logic in download(). Also drop redundant calls to os.path.exists(ud.clonedir) before self.need_update(), since need_update() includes its own built-in check for the existance of ud.clonedir. (Bitbake rev: 61b0df5523afc8f805043f3adc9c106690e6f133) Signed-off-by: Andre McCurdy Signed-off-by: Richard Purdie

bitbake: fetch2/git: log exception if ls-remote fails

2018-03-28T11:51:52+00:00

(Bitbake rev: 8212446de11c0e370c55f88cde86334b760cd939) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: fetch/git: Use 'git-make-shallow' from bin directory

2018-02-14T15:26:03+00:00

Move the code that existed in tests/fetch.py for determining the path to 'git-make-shallow' into the git module and reference it. This ensures that 'git-make-shallow' is always available and the desired version regardless of the path variable or whether git exposes the command. (Bitbake rev: 6b508ab8fd5aa796c1c00c970e81e5e93f84d35d) Signed-off-by: Nathan Rossi Signed-off-by: Richard Purdie

bitbake: fetch/git: Add pack-refs command to mirror handling code

2018-01-29T08:50:08+00:00

We've noticed issues on our infrastucture iterating over the many tag/branch/head reference files that some git repositories may contain. By issuing the pack-refs command, we move these all to a single file which speeds up operations with the mirror repos in the downloads directory in general. (Bitbake rev: f8126aaf774186a6eaf0bd4067b89c074594886c) Signed-off-by: Richard Purdie

bitbake: Replace deprecated git branch parameter "--set-upstream"

2017-09-11T16:19:37+00:00

Since 2017-08-17 (git version 2.14.1.473.g3ec7d702a) using deprecated git branch parameter "--set-upstream" causes a fetcher error. Replace it by "--set-upstream-to". https://git.kernel.org/pub/scm/git/git.git/commit/?id=52668846ea2d41ffbd87cda7cb8e492dea9f2c4d says, it's deprecated since 2012-08-30 so hopefully all still supported host distributions have new enough git to support "--set-upstream-to". ERROR: PACKAGE do_unpack: Fetcher failure: ...; git -c core.fsyncobjectfiles=0 branch --set-upstream master origin/master failed with exit code 128, output: fatal: the '--set-upstream' option is no longer supported. Please use '--track' or '--set-upstream-to' instead. ERROR: PACKAGE do_unpack: Function failed: base_do_unpack (Bitbake rev: 2ab50074c1a6c56a8a178755de108447d7b7acaf) Signed-off-by: Andre Rosa Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie

bitbake: fetch/git: add support for removing arbitrary revs for shallow

2017-06-02T12:36:57+00:00

In certain cases, it's valuable to be able to exert more control over what history is removed, beyond srcrev+depth. As one example, you can remove most of the upstream kernel history from a kernel repository, keeping predominently the non-publically-accessible content. If the repository is private, the history in that repo couldn't be restored via `git fetch --unshallow`, but upstream history could be. Example usage: # Remove only these revs, not at a particular depth BB_GIT_SHALLOW_DEPTH_pn-linux-foo = "0" BB_GIT_SHALLOW_REVS_pn-linux-foo = "v4.1" (Bitbake rev: 97f856f0455d014ea34c28b1c25f09e13cdc851b) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch/git: add support for keeping extra refs for shallow

2017-06-02T12:36:57+00:00

By default, all unused refs (branches & tags) are removed from the repository, as shallow processing scales with the number of refs it has to process. Add the ability to explicitly specify additional refs to keep. This is particularly useful for recipes with custom checkout processes, or whose git-based versioning requires a tag be available (i.e. for `git describe --tags`). The new `BB_GIT_SHALLOW_EXTRA_REFS` variable is a space-separated list of refs, fully specified, and support wildcards. Example usages: BB_GIT_SHALLOW_EXTRA_REFS = "refs/tags/v1.0" BB_GIT_SHALLOW_EXTRA_REFS += "refs/heads/*" (Bitbake rev: 1771934cd9f8b5847c6fcae0a906fb99d6b0db16) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch/git: support per-branch/per-url depths for shallow

2017-06-02T12:36:57+00:00

Allow the user to explicitly adjust the depth for named urls/branches. The un-suffixed BB_GIT_SHALLOW_DEPTH is used as the default. Example usage: BB_GIT_SHALLOW_DEPTH = "1" BB_GIT_SHALLOW_DEPTH_doc = "0" BB_GIT_SHALLOW_DEPTH_meta = "0" (Bitbake rev: 9dfc517e5bcc6dd203a0ad685cc884676d2984c4) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch/git: add support for shallow mirror tarballs

2017-06-02T12:36:57+00:00

This adds support to the git fetcher for fetching, using, and generating mirror tarballs of shallow git repositories. The external git-make-shallow script is used for shallow mirror tarball creation. This implements support for shallow mirror tarballs, not shallow clones. Supporting shallow clones directly is not really doable for us, as we'd need to hardcode the depth between branch HEAD and the SRCREV, and that depth would change as the branch is updated. When BB_GIT_SHALLOW is enabled, we will always attempt to fetch a shallow mirror tarball. If the shallow mirror tarball cannot be fetched, it will try to fetch the full mirror tarball and use that. If a shallow tarball is to be used, it will be unpacked directly at `do_unpack` time, rather than extracting it to DL_DIR at `do_fetch` time and cloning from there, to keep things simple. There's no value in keeping a shallow repository in DL_DIR, and dealing with the state for when to convert the clonedir to/from shallow is not worthwhile. To clarify when shallow is used vs a real repository, a current clone is preferred to either tarball, a shallow tarball is preferred to an out of date clone, and a missing clone will use either tarball (attempting the shallow one first). All referenced branches are truncated to SRCREV (that is, commits *after* SRCREV but before HEAD are removed) to further shrink the repository. By default, the shallow construction process removes all unused refs (branches/tags) from the repository, other than those referenced by the URL. Example usage: BB_GIT_SHALLOW ?= "1" # Keep only the top commit BB_GIT_SHALLOW_DEPTH ?= "1" # This defaults to enabled if both BB_GIT_SHALLOW and # BB_GENERATE_MIRROR_TARBALLS are enabled BB_GENERATE_SHALLOW_TARBALLS ?= "1" (Bitbake rev: 5ed7d85fda7c671be10ec24d7981b87a7d0d3366) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch: support multiple mirror tarball filenames

2017-06-02T12:36:57+00:00

Remove ud.mirrortarball in favor of ud.mirrortarballs. Each tarball will be attempted, in order, and the first available will be used. This is needed for git shallow mirror tarball support, as we want to be able to use either a shallow or full mirror tarball. (Bitbake rev: 02eebee6709e57b523862257f75929e64f16d6b0) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie