linux/poky.git/bitbake/lib/bb/fetch2/git.py, branch uninative-1.7

bitbake: fetch/git: add support for removing arbitrary revs for shallow

2017-06-02T12:36:57+00:00

In certain cases, it's valuable to be able to exert more control over what history is removed, beyond srcrev+depth. As one example, you can remove most of the upstream kernel history from a kernel repository, keeping predominently the non-publically-accessible content. If the repository is private, the history in that repo couldn't be restored via `git fetch --unshallow`, but upstream history could be. Example usage: # Remove only these revs, not at a particular depth BB_GIT_SHALLOW_DEPTH_pn-linux-foo = "0" BB_GIT_SHALLOW_REVS_pn-linux-foo = "v4.1" (Bitbake rev: 97f856f0455d014ea34c28b1c25f09e13cdc851b) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch/git: add support for keeping extra refs for shallow

2017-06-02T12:36:57+00:00

By default, all unused refs (branches & tags) are removed from the repository, as shallow processing scales with the number of refs it has to process. Add the ability to explicitly specify additional refs to keep. This is particularly useful for recipes with custom checkout processes, or whose git-based versioning requires a tag be available (i.e. for `git describe --tags`). The new `BB_GIT_SHALLOW_EXTRA_REFS` variable is a space-separated list of refs, fully specified, and support wildcards. Example usages: BB_GIT_SHALLOW_EXTRA_REFS = "refs/tags/v1.0" BB_GIT_SHALLOW_EXTRA_REFS += "refs/heads/*" (Bitbake rev: 1771934cd9f8b5847c6fcae0a906fb99d6b0db16) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch/git: support per-branch/per-url depths for shallow

2017-06-02T12:36:57+00:00

Allow the user to explicitly adjust the depth for named urls/branches. The un-suffixed BB_GIT_SHALLOW_DEPTH is used as the default. Example usage: BB_GIT_SHALLOW_DEPTH = "1" BB_GIT_SHALLOW_DEPTH_doc = "0" BB_GIT_SHALLOW_DEPTH_meta = "0" (Bitbake rev: 9dfc517e5bcc6dd203a0ad685cc884676d2984c4) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch/git: add support for shallow mirror tarballs

2017-06-02T12:36:57+00:00

This adds support to the git fetcher for fetching, using, and generating mirror tarballs of shallow git repositories. The external git-make-shallow script is used for shallow mirror tarball creation. This implements support for shallow mirror tarballs, not shallow clones. Supporting shallow clones directly is not really doable for us, as we'd need to hardcode the depth between branch HEAD and the SRCREV, and that depth would change as the branch is updated. When BB_GIT_SHALLOW is enabled, we will always attempt to fetch a shallow mirror tarball. If the shallow mirror tarball cannot be fetched, it will try to fetch the full mirror tarball and use that. If a shallow tarball is to be used, it will be unpacked directly at `do_unpack` time, rather than extracting it to DL_DIR at `do_fetch` time and cloning from there, to keep things simple. There's no value in keeping a shallow repository in DL_DIR, and dealing with the state for when to convert the clonedir to/from shallow is not worthwhile. To clarify when shallow is used vs a real repository, a current clone is preferred to either tarball, a shallow tarball is preferred to an out of date clone, and a missing clone will use either tarball (attempting the shallow one first). All referenced branches are truncated to SRCREV (that is, commits *after* SRCREV but before HEAD are removed) to further shrink the repository. By default, the shallow construction process removes all unused refs (branches/tags) from the repository, other than those referenced by the URL. Example usage: BB_GIT_SHALLOW ?= "1" # Keep only the top commit BB_GIT_SHALLOW_DEPTH ?= "1" # This defaults to enabled if both BB_GIT_SHALLOW and # BB_GENERATE_MIRROR_TARBALLS are enabled BB_GENERATE_SHALLOW_TARBALLS ?= "1" (Bitbake rev: 5ed7d85fda7c671be10ec24d7981b87a7d0d3366) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch: support multiple mirror tarball filenames

2017-06-02T12:36:57+00:00

Remove ud.mirrortarball in favor of ud.mirrortarballs. Each tarball will be attempted, in order, and the first available will be used. This is needed for git shallow mirror tarball support, as we want to be able to use either a shallow or full mirror tarball. (Bitbake rev: 02eebee6709e57b523862257f75929e64f16d6b0) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch2/git: prevent recursion on getting latest revision

2017-03-27T19:23:19+00:00

We call git ls-remote to get the latest revision from a git repository, however by calling runfetchcmd() we can end up recursively running git ls-remote a number of times with OE e.g. if ${SRCPV} is in PV, ${PV} is in WORKDIR, and ${WORKDIR} is in PATH (as a result of recipe-specific sysroots), our call to runfetchcmd() exports PATH so _lsremote() will get called again - with the end result that we run git ls-remote 30 times in quick succession (!). Prevent that from happening by using a guard variable and returning a dummy value if it's called recursively. Fixes [YOCTO #11185]. (Bitbake rev: ff1ccd1db5d70b3fc9ad0d3e8f3d7b804c22bf36) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

bitbake: fetch/git: fix FetchError reference

2017-03-22T11:35:23+00:00

FetchError isn't defined, use bb.fetch2.FetchError in this context. (Bitbake rev: 945fa980e027753df2c21d84eb63dcaddb2caaee) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch/git: drop pointless os.path.join, workdir=

2017-03-22T11:35:23+00:00

The touch of .done explicitly specifies the path, so there's no need for workdir=, and "os.path.join('.')" is identical to just '.'. (Bitbake rev: 955cbfdaa2400d15ec428b65848e6835c9f44860) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch/git: kill pointless quotes around single % args

2017-03-22T11:35:23+00:00

(Bitbake rev: 4aebf12153369364eae2e6e773e2a921e9c91f72) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: fetch/git: use enumerate for ud.names

2017-03-22T11:35:23+00:00

list.index() isn't a particularly efficient operation, so keep track of our position via enumerate() instead, which is more pythonic as well. (Bitbake rev: dec6e90a4d27ee335e9c78aeebd277098fec94d1) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie