linux/poky.git/bitbake/lib/bb/fetch2, branch yocto-3.4

bitbake: fetch2: npmsw: Add support for local tarball and link sources

2021-10-08T15:45:06+00:00

(Bitbake rev: 4f983dc419a1a6f635a5d333f253d49244cec374) Signed-off-by: Stefan Herbrechtsmeier Signed-off-by: Richard Purdie

bitbake: fetch2: npm: Create config npmrc in environment instantiation

2021-10-08T15:45:05+00:00

Create a configuration npmrc per npm environment to avoid repeated creation of the same configuration file. Create the file via python to avoid multiple npm config calls and add the ability to pass a file path instead of a temporary file. Deprecate the npm configs argument of the run function. The configs should be passed to npm environment or as command specific arguments. (Bitbake rev: 2c2df49b06a2bad7a5b8872a9998338a4660498f) Signed-off-by: Stefan Herbrechtsmeier Signed-off-by: Richard Purdie

bitbake: fetch2: npm: Support npm archives with missing search directory mode

2021-10-08T15:45:05+00:00

Delay directory restore and set execute/search directory mode bits in unpack to support npm archives with a missing search directory mode. (Bitbake rev: 60cbd34d3da8f0f523281aad7eec93eec9cd4db8) Signed-off-by: Stefan Herbrechtsmeier Signed-off-by: Richard Purdie

bitbake: fetch2/gitsm: remove the 'nugget' SRCREV caching

2021-10-08T15:45:05+00:00

The cached revisions which are used to decide if a repository doesn't need to be updated are misleading when used in conjunction with mirror tarballs and can cause partial fetches to happen, resulting in unpack errors as repositories were not fetched. A concrete example: edk2-firmware in meta-arm is at version 202102 (ef91b0). This is built on the autobuilder so the source mirror contains the repository as a mirror tarball. If I build edk2-firmware 202102 the gitsm fetcher will initially download the top-level repository and then iterate into the submodules to also fetch those repositories, including cmocka from cryptomilk.org. edk2-firmware will then unpack and build successfully. I then update edk2-firmware to 202105 (e1999b) and build it. Gitsm.needs_update() starts by calling Git.needs_update() which returns False, as the mirror tarball contains this revision. It then looks at the "nuggets" which are SRCREVs it has fetched before. The mirror tarball itself contains the nugget for e1999b as this has been built on the autobuilder, so needs_update return False, no more fetching is done, and the build proceeds to unpack. However, as part of the 202105 upgrade the URL of the cmocka submodule changed, and this new repository was never fetched. This means that unpack fails as one of the required git repositories isn't available. The nugget codepaths appear to be an attempt at optimising the fetch process, but have demonstratable failure cases. Just removing them entirely solves the edk2-firmware example, and all of the fetcher test cases still pass. (Bitbake rev: 51212507ce3f670ace9efb691c92887d66f7aaf8) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: fetch2: clarify the command-no-found error message

2021-10-08T15:45:05+00:00

If runfetchcmd() fails with bb.process.NotFoundError, the message output is simply "Fetch command" which doesn't really explain what the problem is. Add "not found" to clarify what happened. (Bitbake rev: 8de9dc02ed6a73b47f2ab10be30d1aed7954bc72) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: fetch2/svn: Allow peg-revision functionality to be disabled

2021-09-21T22:44:08+00:00

Sometimes the peg revision functionality we use in the svn fetcher is not the correct option. Add a parameter nopegrevision which can be added to disable this behaviour. [YOCTO #6258] (Bitbake rev: ea26682b706f655a8e418f56bebe742e2d130a1d) Signed-off-by: Richard Purdie

bitbake: fetch2/git: Use os.rename instead of mv

2021-09-21T22:44:08+00:00

os.rename will overwrite the destination file if present so we can use this instead of the process call overhead. (Bitbake rev: b3cccaa6a896c41d8c9be5eebc327f726542d16b) Signed-off-by: Richard Purdie

bitbake: fetch2/s3: allow to switch profile from environment variable

2021-09-17T06:26:24+00:00

Make usage of Bitbake's s3 fetcher more flexible with different AWS profiles and switch between profiles as export of the AWS_PROFILE environment variable (Bitbake rev: 0f35dc4dfc829fe9c51c239d15567f86c5c14e58) Signed-off-by: Valentin Danaila Signed-off-by: Richard Purdie

bitbake: fetch2/git: Avoid races over mirror tarball creation

2021-09-17T06:26:23+00:00

There is a potential race over the mirror tarballs where a partial git repo could be extracted causing fetcher failures if the tarball is being rewritten whilst another build accesses it. Create the mirror tarball atomically to avoid this. [YOCTO #14441] (Bitbake rev: 3250bc950c56bd7dd2114df26e5a8e13b04ceac8) Signed-off-by: Richard Purdie

bitbake: fetch2/wget: Enable ftps

2021-09-16T08:51:14+00:00

The fetcher would fail with: Could not find a fetcher which supports the URL: ftps://... (Bitbake rev: 9e56710c7203b1ec6cbefa758c81b69b697fe1a4) Signed-off-by: Daniel Ammann Signed-off-by: Richard Purdie