linux/poky.git/bitbake/lib/bb/fetch2, branch yocto-5.0.7

bitbake: fetch2: use persist_data context managers

2024-12-06T13:50:24+00:00

Python 3.13 emits a ResourceWarning for unclosed sqlite3 `Connection`s. See https://docs.python.org/3/whatsnew/3.13.html#sqlite3 The previous commit fixed persist_data's context manager to close the connection, but we were never actually using `with` in the first place. This change is not necessary on 'master' because persist_data was removed. (Bitbake rev: 6c2641f7a9e92c1b82e306f59ddd3c1249c52cbf) Signed-off-by: Chris Laplante Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: fetch/wget: Increase timeout to 100s from 30s

2024-12-06T13:50:24+00:00

Testing shows the worst case CDN response time can be up to 100s. The wget fetcher is used for accessing sstate from the CDN so increase our timeouts there to match our worst case repsonse times. (Bitbake rev: c7f282cd27edfd78830b61db586ed669808893a5) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: fetch2/git: Use quote from shlex, not pipes

2024-11-28T13:36:00+00:00

The pipes module is removed in python 3.13. It was already using the quote function from shlex so use that directly instead. The module already imports shlex too so it is an easy substitution. (Bitbake rev: 70bd343fb273ad174e56d08c8b80c5594501e030) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: gitsm: Remove downloads/tmpdir when failed

2024-11-02T13:12:00+00:00

The tmpdir such as downloads/tmplp3cnemv won't be removed without this fix. (Bitbake rev: 6894f68204130713d3651fceacadc7f8061174a8) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie (cherry picked from commit 2ba8d3214759142afc11f0a88d80eb30a8bcde3a) Signed-off-by: Steve Sakoman

bitbake: gitsm: Add call_process_submodules() to remove duplicated code

2024-11-02T13:12:00+00:00

There are 14 lines can be removed, and can make it easy to maintain. (Bitbake rev: 76de94d46ce270c4e485f8178725973adf0f42eb) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie (cherry picked from commit 0ea2c1ac079d63349407a69172ff80cd9acc7252) Signed-off-by: Steve Sakoman

bitbake: fetch2/wget: Fix failure path for files that are empty or don't exist

2024-06-24T13:46:52+00:00

When we intercepted the file download to a temp file, we broke the exist/size checks which need to happen before the rename. Correct the ordering. For some reason, python 3.12 exposes this problem in the selftests differently to previous versions. (Bitbake rev: 8714a02e13477a9d97858b3642e05f28247454b5) Signed-off-by: Richard Purdie (cherry picked from commit c56bd9a9280378bc64c6a7fe6d7b70847e0b9e6d) Signed-off-by: Steve Sakoman

bitbake: fetch2/wget: Canonicalize DL_DIR paths for wget2 compatibility

2024-06-24T13:46:52+00:00

Some distributions (namely Fedora Core 40) have started replacing wget with wget2. There are some changes to wget2 that make it incompatible with wget: 1. ftp/ftps is not supported anymore 2. progress 'dot' is not yet supported 3. Relative paths in -P and -O are not correctly dealt with Item 1: Is already dealt with since Scarthgap by only adding the option --passive-ftp when the URL specifies ftp/sftp. While that won't help if ftp/sftp is actually required it at least does not break http/https downloads. Item 2: While not supported it at least does not break the operation. Item 3: If there are relative path components in -P or -O then wget2 only deals with them correctly if there is one, and only one, relative path component at the beginning of the path: -P ./downloads works -P ../downloads works -P ../../downloads does not work -P ./../downloads does not work -P /home/user/downloads/../downloads does not work In cases where there are more than one relative path component at the beginning of the path and/or one or more reltaive path component somewhere in the middle or end of the path, wget2 aborts with the message Internal error: Unexpected relative path: '') Such can happen if DL_DIR includes relative path components e.g. DL_DIR = "${TOPDIR}/../../downloads". This patch canonicalizes DL_DIR before it is passed to wget. (Bitbake rev: 47678142e26bb76d1351886060deff5e75039bc9) Signed-off-by: Rudolf J Streif Signed-off-by: Richard Purdie (cherry picked from commit 3e4208952b086adc510e78c1c5f9cf4550d79dc9) Signed-off-by: Steve Sakoman

bitbake: fetch2/gcp: Add missing runfetchcmd import

2024-05-29T14:50:00+00:00

This adds the missing import. This bug was introduced with 1ab1d36c. (Bitbake rev: b5159c0373e2e7d403aed16e096ad655f38b1fa7) Signed-off-by: Sven Schwermer Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: wget: Make wget --passive-ftp option conditional on ftp/ftps

2024-04-11T07:47:07+00:00

Fedora 40 introduces wget2 as a drop-in replacement for wget. This rewrite does not currently have support for FTP. This causes the wget fetcher to fail complaining about an unrecognized option. Making --passive-ftp conditional based on the protocol used in the SRC_URI limits the scope of the problem. It also gives us an opportunity to build the older wget as a host tool. (Bitbake rev: f10e630fd7561746d835a4378e8777e78f56e44a) Signed-off-by: Rob Woolley Signed-off-by: Richard Purdie

bitbake: fetch2/git: Install Git LFS in local repository config

2024-03-22T16:26:46+00:00

Git uses a lock file to prevent concurrent modifications to the global config, so if unpack tasks for different recipes try to run "git lfs install" simultaneously the operation can fail: error: could not lock config file /home/build/.gitconfig: File exists exit status 255 Run `git lfs install --force` to reset Git configuration. Adding "--local" sets the smudge and clean filters in the local repository's config instead of modifying the user's global config. (Bitbake rev: 328ca4de8422be514fa0d0c9e3cfd36bb9d3e9a7) Signed-off-by: Derek Erdmann Signed-off-by: Richard Purdie