linux/poky.git/bitbake/lib/bb/fetch2/gitsm.py, branch 4.2_M1

bitbake: gitsm.py: process_submodules(): Set nobranch=1 for url

2022-11-14T16:22:00+00:00

Just like download_submodule() does, fixed warings when run bb.fetch2.Fetch([url], d) in process_submodules' function: WARNING: grpc-native-1.50.0-r0 do_fetch: URL: gitsm://github.com/abseil/abseil-cpp.git;protocol=https;name=third_party/abseil-cpp;subpath=third_party/abseil-cpp does not set any branch parameter. The future default branch used by tools and repositories is uncertain and we will therefore soon require this is set in all git urls. (Bitbake rev: 0ed7c75eb0508a1f699f47d7f22d559501865f61) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: gitsm: Fix regression in gitsm submodule path parsing

2022-11-14T16:22:00+00:00

Commit 0361ecf7eb82c386a9842cf1f3cb706c0a112e77 introduced regression in submodules path parsing. As the result gitsm fetcher fails on each submodule which name begins from the name of the parent repo which is totally valid usecase [Yocto #14045] [1] Fix the code to error out only if submodule's name is equal to parent name but not if it's part of it. [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=14045#c4 (Bitbake rev: 3ad27272c18f2bb9edd441f840167a3dabd5407b) Signed-off-by: Pavel Zhukov Signed-off-by: Richard Purdie

bitbake: gitsm: Error out if submodule refers to parent repo

2022-09-01T09:07:00+00:00

If submodule refers to specific revision of the parent repository it causes deadlock in bitbake locking mechanism (lock is acquired to fetch the parent and cannot be released before all submodules are fetched). raise FetchError in such situation to prevent deadlocking. [Yocto 14045] (Bitbake rev: 0361ecf7eb82c386a9842cf1f3cb706c0a112e77) Signed-off-by: Pavel Zhukov Signed-off-by: Richard Purdie

bitbake: fetch2: gitsm: fix incorrect handling of git submodule relative urls

2022-07-01T11:20:18+00:00

As specified by git submodule manual relative urls can start either with '..' or './', second case was incorrectly managed leading to an interpretation of urls starting with './' as absoulte urls. (Bitbake rev: 4a0bd3bcd1f7fc25364df8bbf185ff64881c015b) Signed-off-by: Gennaro Iorio Signed-off-by: Richard Purdie

bitbake: lib/bb: Clean up use of len()

2021-11-03T10:12:42+00:00

(Bitbake rev: bbbc843e86639604d00d76b1949b94a78cf1d95d) Signed-off-by: Richard Purdie

bitbake: fetch2/gitsm: remove the 'nugget' SRCREV caching

2021-10-08T15:45:05+00:00

The cached revisions which are used to decide if a repository doesn't need to be updated are misleading when used in conjunction with mirror tarballs and can cause partial fetches to happen, resulting in unpack errors as repositories were not fetched. A concrete example: edk2-firmware in meta-arm is at version 202102 (ef91b0). This is built on the autobuilder so the source mirror contains the repository as a mirror tarball. If I build edk2-firmware 202102 the gitsm fetcher will initially download the top-level repository and then iterate into the submodules to also fetch those repositories, including cmocka from cryptomilk.org. edk2-firmware will then unpack and build successfully. I then update edk2-firmware to 202105 (e1999b) and build it. Gitsm.needs_update() starts by calling Git.needs_update() which returns False, as the mirror tarball contains this revision. It then looks at the "nuggets" which are SRCREVs it has fetched before. The mirror tarball itself contains the nugget for e1999b as this has been built on the autobuilder, so needs_update return False, no more fetching is done, and the build proceeds to unpack. However, as part of the 202105 upgrade the URL of the cmocka submodule changed, and this new repository was never fetched. This means that unpack fails as one of the required git repositories isn't available. The nugget codepaths appear to be an attempt at optimising the fetch process, but have demonstratable failure cases. Just removing them entirely solves the edk2-firmware example, and all of the fetcher test cases still pass. (Bitbake rev: 51212507ce3f670ace9efb691c92887d66f7aaf8) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: logging: Make bitbake logger compatible with python logger

2021-02-10T23:48:16+00:00

The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] (Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: fetch2/gitsm: Make need_update() process submodules

2020-06-15T13:55:25+00:00

If the bitbake.srcrev nugget is not present for the commit we're interested in we should not just bail out and say that an update is needed. Instead we can recursively walk through the submodules and check for the presence of the required commits. (Bitbake rev: cfc78316309556bec487ef0a5a9205e41f1be86f) Signed-off-by: Paul Barker Signed-off-by: Richard Purdie

bitbake: fetch2/gitsm: Mark srcrev as fetched once all submodules are processed

2020-06-15T13:55:25+00:00

This prevents multiple bitbake.srcrev entries being created when a repository has more than one submodule. It also ensures that the bitbake.srcrev entry is not added unless all submodules are correctly fetched. (Bitbake rev: 860ff1193fe53f04696d41635a720c2d1f29fa7f) Signed-off-by: Paul Barker Signed-off-by: Richard Purdie

bitbake: fetch2: Add the ability to list expanded URL data

2020-06-04T12:28:16+00:00

Some fetchers may download additional sources along with those explicitly listed in SRC_URI. These "implicit URLs" will be needed by the archiver to ensure that all sources can be archived. We can't just return a list of URL strings since each URL may need its own SRCREV data so we return a list of FetchData objects. Each fetcher can override the implicit_urldata() function to provide the additional FetchData objects. For now this is just needed in the gitsm fetcher to walk git submodules recursively. (Bitbake rev: 1350f241b7d991bd191ce9e44f6662e4376c6e24) Signed-off-by: Paul Barker Signed-off-by: Richard Purdie