linux/poky.git/bitbake/lib/bb/fetch2/wget.py, branch 4.2_M1

bitbake: wget: Avoid bad checksum race issues

2022-09-05T11:54:25+00:00

If two recipes have conflicting checksums for a file, the code will currently remove the existing file when a mismatch is downloaded, even if another task successfully fetched it. This changes the code to verify the checksum (if possible) before replacing the file. This removes a potential race window and stops builds failing everywhere from one incorrect checksum. To make this work, we need to be able to override localpath and avoid NoChecksum errors being logged. (Bitbake rev: 4b8de2e7d12667d69d86ffe6e9f85a7932c4c9a5) Signed-off-by: Richard Purdie

bitbake: fetch/wget: Move files into place atomically

2022-06-08T20:53:15+00:00

(Bitbake rev: cd7cce4cf4be5c742d29671169354fe84220b47a) Signed-off-by: Richard Purdie

bitbake: fetch2: Abstract fetcher environment to a function

2022-02-20T16:45:25+00:00

The changing of the environment inside the wget fetcher can race if threading is used, such as with sstate in OE-Core. Abstract the function so the environment can be correct before the function is called, removing the race since the enviroment is then no longer changed. (Bitbake rev: c73bb6023c73f003a160bb02aa4da1b580b86c23) Signed-off-by: Richard Purdie

bitbake: wget: Fix grammar "can happen"

2022-02-20T16:45:25+00:00

(Bitbake rev: 52630eefb5174e4ca357ac57085093a7f5767bd8) Signed-off-by: Zygmunt Krynicki Signed-off-by: Richard Purdie

bitbake: fetch2/wget: move loop-invariant load of BB_ORIGENV

2022-02-20T16:45:24+00:00

BB_ORIGENV is used as a fallback environment block. It is repeatedly accessed inside the loop. Since it is a loop invariant, move it out of the loop. (Bitbake rev: 346a1a6c76d40458d7b4c116147ec4d371bee74a) Signed-off-by: Zygmunt Krynicki Signed-off-by: Richard Purdie

bitbake: fetch2/wget: do not hardcode tarball compressors in version check

2021-12-08T20:22:53+00:00

(Bitbake rev: ba3aa8591327d43935f000c6884637997438ecb2) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: fetch2/wget: add redirectauth parameter

2021-12-08T20:22:53+00:00

Add a parameter that limits sending Basic authentication in the Authorization header to only the first host and not any that we're redirected to. Ignoring potential security concerns, temporary AWS URLs will reject any request that includes authentication details in both the query parameters (from the redirect) and in the Authorization header. Temporary AWS URLs are now being used for release assets from private Github repositories. According to the previous discussion linked below, they're also in use by bitbucket. See also: https://lore.kernel.org/bitbake-devel/CAC9ffDEuZL-k8199bUyN+8frjw6bg-g=vrumxxtvt+RVParQ8Q@mail.gmail.com/ (Bitbake rev: a6ab32013a4381a1b694ed46caf2c9da932644d0) Signed-off-by: Justin Bronder Signed-off-by: Richard Purdie

bitbake: fetch/wget: Add timeout for checkstatus calls (30s)

2021-11-10T19:25:38+00:00

We had an issue where a webserver serving sstate had filesystem issues so would accept connections but effectively not do anything with them. This causes bitbake to hang whilst processing things like sstate objects inside the checkstatus() calls. It can be replicated by setting up a server like: socat -u TCP4-LISTEN:NNN,fork OPEN:/dev/null and pointing SSTATE_MIRRORS in OE at that address. Adding a timeout to the checkstatus calls of 15s means that whilst the system will pause, it will then continue and not hang entirely. Since there isn't a large transfer here, 30s should be a reasonable response time after which we should fall back to building things ourselves. [YOCTO #13716] (Bitbake rev: edc3b0c3953cab675e29fe295b58cfa84ba811c3) Signed-off-by: Richard Purdie

bitbake: fetch2/wget: Enable ftps

2021-09-16T08:51:14+00:00

The fetcher would fail with: Could not find a fetcher which supports the URL: ftps://... (Bitbake rev: 9e56710c7203b1ec6cbefa758c81b69b697fe1a4) Signed-off-by: Daniel Ammann Signed-off-by: Richard Purdie

bitbake: fetch2/wget: fix 'no_proxy' handling

2021-08-23T07:30:54+00:00

The urllib.request.ProxyHandler constructor only reads the $http_proxy + $https_proxy environment variables. $no_proxy is evaluated later when the url is opened. It is therefore not sufficient to just construct the proxy handler in the | with bb.utils.environment(**newenv): context, but the 'opener.open(r)' call must also be made there. (Bitbake rev: 076baf4fbd328d247508fd399866a397eb34f67e) Signed-off-by: Enrico Scholz Signed-off-by: Richard Purdie