linux/poky.git/bitbake/lib/bb/fetch2/git.py, branch kirkstone-4.0.28

bitbake: fetch2/git: Use quote from shlex, not pipes

2024-11-02T13:06:09+00:00

The pipes module is removed in python 3.13. It was already using the quote function from shlex so use that directly instead. The module already imports shlex too so it is an easy substitution. (Bitbake rev: 53264dc14554890b3a2afc83cb1749cf10d86854) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: fetch2: Ensure that git LFS objects are available

2024-03-01T18:00:58+00:00

The current implementation only performs a git lfs fetch alongside of a regular git fetch. This causes issues when the downloaded revision is already part of the fetched repository (e.g. because of moving back in history or the updated revision already being part of the repository at the time of the initial clone). Fix this by explicitly checking whether the required LFS objects are available in the downloade directory before confirming that a downloaded repository is up-to-date. This issue previously went unnoticed as git lfs would silently fetch the missing objects during the `unpack` task. With network isolation turned on, this no longer works, and unpacking fails. (cherry picked from commit cfae1556bf671acec119a6c8bbc4b667a856b9ae) (Bitbake rev: 40fd5f4eef7460ca67f32cfce8e229e67e1ff607) Signed-off-by: Philip Lorenz Signed-off-by: Richard Purdie Signed-off-by: Philip Lorenz Signed-off-by: Steve Sakoman

bitbake: fetch/git: Fix local clone url to make it work with repo

2023-04-11T10:31:52+00:00

The "git clone /path/to/git/objects_symlink" couldn't work after the following change: https://github.com/git/git/commit/6f054f9fb3a501c35b55c65e547a244f14c38d56 But repo command manages the git repo as symlinks, so check whether the objects is an symlink to fix the problem: * Nothing is changed if git/objects is not a symlink * Use "git clone file://" when git/objects is a symlink (Bitbake rev: 2802adb572eb73a3eb2725a74a9bbdaafc543fa7) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie (cherry picked from commit a0d8108eba8d542707740d00c66c1c5f5b963f18) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch2/git: show SRCREV and git repo in error message about fixed SRCREV

2023-02-23T22:23:07+00:00

* in recipe with 17 git repos in SRC_URI I've accidentally pasted one SRCREV to be one character shorter and because fetcher uses: if not ud.revisions[name] or len(ud.revisions[name]) != 40 or (False in [c in "abcdef0123456789" for c in ud.revisions[name]]): to decide which SRCREV values are fixed SRCREVs this one was considered as tag or branch name, because it was only 39 chars long The original error message wasn't very helpful as it doesn't show which repo or which SRCREV was considered missing: do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None) with SRCPV included in PV as error recomments it's a bit better: bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '0a92994d729ff76a58f692d3028ca1b64b145d9' in upstream git repository in git ls-remote output for github.com/Maratyszcza/FP16 The variable dependency chain for the failure is: SRCPV -> PV -> WORKDIR -> T with this change the first error will read: do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch '0a92994d729ff76a58f692d3028ca1b64b145d9' for repo 'github.com/Maratyszcza/FP16' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None) (Bitbake rev: 9bbdedc0ba7ca819b898e2a29a151d6a2014ca11) Signed-off-by: Martin Jansa Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch2/git: Clarify the meaning of namespace

2023-02-04T10:34:05+00:00

Namespace in this context means a branch, a tag, etc., clarify it in the description. Also, fix a typo "a any", replace with plain "any". This patch is based of feedback on new applied patch d32e5b0e ("fetch2/git: Prevent git fetcher from fetching gitlab repository metadata") (Bitbake rev: 86f2fa5261da959cda706c794a0047e5e89d4d6b) Signed-off-by: Marek Vasut Signed-off-by: Richard Purdie (cherry picked from commit b4999425c812b25cb359d5163d11e3c1b030dc28) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch2/git: Prevent git fetcher from fetching gitlab repository metadata

2023-02-04T10:34:05+00:00

The bitbake git fetcher currently fetches 'refs/*:refs/*', i.e. every single object in the remote repository. This works poorly with gitlab and github, which use the remote git repository to track its metadata like merge requests, CI pipelines and such. Specifically, gitlab generates refs/merge-requests/*, refs/pipelines/* and refs/keep-around/* and they all contain massive amount of data that are useless for the bitbake build purposes. The amount of useless data can in fact be so massive (e.g. with FDO mesa.git repository) that some proxies may outright terminate the 'git fetch' connection, and make it appear as if bitbake got stuck on 'git fetch' with no output. To avoid fetching all these useless metadata, tweak the git fetcher such that it only fetches refs/heads/* and refs/tags/* . Avoid using negative refspecs as those are only available in new git versions. Per feedback on the ML, Gerrit may push commits outsides of branches or tags during CI runs, which currently works with the 'nobranch=1' fetcher parameter. To retain this functionality, keep fetching everything in case the 'nobranch=1' is present. This still avoids fetching massive amount of data in the common case, since 'nobranch=1' is rare. Update 'nobranch' documentation. Reviewed-by: Peter Kjellerstedt (Bitbake rev: c17fc1468ab84663b919e2809606b1b8ea2bebd9) Signed-off-by: Marek Vasut Signed-off-by: Alexandre Belloni (cherry picked from commit d32e5b0ec2ab85ffad7e56ac5b3160860b732556) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch/git : Use cat as pager

2022-05-12T12:41:42+00:00

We don't have less in HOSTTOOLS in OE and this can confuse git. Force the pager to cat to be consistent and minimal everywhere. (Bitbake rev: 59c16ae6c55c607c56efd2287537a1b97ba2bf52) Signed-off-by: Richard Purdie (cherry picked from commit d3d406e8552fdd865dc58b419a84411736475ad2) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch2/git: canonicalize ids in generated tarballs

2022-04-14T08:48:33+00:00

Change the owner information in the mirror tarballs generated using BB_GENERATE_MIRROR_TARBALLS="1". This is an extension of commit 0178ab83, which used the original pokybuild:user information, but failed to clean up the numerical user and group ids. Now set the more canonical values of oe:oe and 0:0. (Bitbake rev: 37437115d3fb1a9f5d8ed7356a0fc01a408e4f8c) Signed-off-by: Olaf Mandel CC: Marek Vasut Signed-off-by: Richard Purdie

bitbake: fetch2/git: stop generated tarballs from leaking info

2022-03-26T09:27:43+00:00

When using BB_GENERATE_MIRROR_TARBALLS="1" to generate mirror tarballs of git repositories, they leaked local information: username, group and time of the last fetch. Remove all these by setting fixed information: * uname = pokybuild * gname = users * mtime = committer time of newest commit in repo The username and group value were taken from the archives available on the downloads.yoctoproject.org mirror. The modification time is chosen so it still retains some relationship to the contents of the archive. (Bitbake rev: 0178ab83e6312e97e528aa8c5e12105f5165d896) Signed-off-by: Olaf Mandel Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: fetch2/git: Remove duplicate "the"

2022-02-20T16:45:24+00:00

(Bitbake rev: 02e29e90eac56e46635e9ae44e5c5da684b70ed2) Signed-off-by: Zygmunt Krynicki Signed-off-by: Richard Purdie