linux/poky.git/bitbake/lib/bb/fetch2/wget.py, branch nanbield

bitbake: wget.py: Combine urlopener exceptions

2023-03-11T20:49:22+00:00

No reason to have three identical exception handles, refactor to catch any of the exceptions with the same block of code. (Bitbake rev: b29f6e04091b6bfe697dc41c76880de466736fc3) Signed-off-by: Mark Hatle Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: wget.py: Add catch TimeoutError exception

2023-03-11T20:49:22+00:00

We've observed TimeoutError exceptions during the sstate-cache mirror fetch, it appears that due to the number of (invalid) files requested the remote side is eventually dropping the connection (not closing it) which can result in a TimeoutError exception being sent, while rate it is different from the urllib.error.URLError or ConnectionResetError. (Bitbake rev: 6041b34740deee09ea65d705702555456a5e05d8) Signed-off-by: Mark Hatle Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: fetch2/wget: Drop unused import

2023-02-26T11:49:41+00:00

This import is no longer used anywhere so can be removed. (Bitbake rev: 956128e394581855bf0d03b32a975dc91c2a7e0c) Signed-off-by: Richard Purdie

bitbake: fetch2/wget: clean up netrc usage

2023-02-17T18:02:04+00:00

Assigning a return value which is potentially None to a tuple and catching TypeError is pretty ugly. Rewrite the code to explicitly check the value for clarity. (Bitbake rev: f4ebb27616ac2df27c29a6052b1526a4c48db607) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: fetch2/wget.py: correctly match versioned directories

2022-12-11T16:33:19+00:00

When obtaining latest upstream versions, the code needs to check if the existing tarball is in a versioned directory (e.g. component-name/x.y/component-name-x.y.z.tar.gz) and if it is, it needs to first obtain the list of all such versioned directories and then check all of them by going one step up in the directory hierarchy. Existing code was returning a correct match when the component name did not have numbers, e.g. a check on 'source/epiphany/43/' would return 43, but was stopping too soon when the component name itself had numbers ('source/libxml2/2.10/' would return libxml2). This change ensures the last match is taken instead of the first. Also, adjust the fetcher tests to check that versioned directories are correctly traversed in this case (e.g. the step to go one level up is taken and a new tarball is discovered in a different versioned directory). (Bitbake rev: b6601be22c6d776327acdcd1fa931400f41ac786) Signed-off-by: Alexander Kanavin Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: fetch2/wget: handle username/password in uri

2022-12-09T13:22:11+00:00

In checkstatus() opener.open() is used to check if an artifact is available. The check fails if the uri contains username password in the format: "username:password@hostname..". Moreover, the checkstatus function already uses the username from the "ud" object to craft a header, is username and password is provided. This fix ensure the uri in the Requests object used does not contain username as password. (Bitbake rev: 88350002d45e0aa85ecd5356da2c8d71e450641e) Signed-off-by: Kasper Revsbech Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: wget: Avoid bad checksum race issues

2022-09-05T11:54:25+00:00

If two recipes have conflicting checksums for a file, the code will currently remove the existing file when a mismatch is downloaded, even if another task successfully fetched it. This changes the code to verify the checksum (if possible) before replacing the file. This removes a potential race window and stops builds failing everywhere from one incorrect checksum. To make this work, we need to be able to override localpath and avoid NoChecksum errors being logged. (Bitbake rev: 4b8de2e7d12667d69d86ffe6e9f85a7932c4c9a5) Signed-off-by: Richard Purdie

bitbake: fetch/wget: Move files into place atomically

2022-06-08T20:53:15+00:00

(Bitbake rev: cd7cce4cf4be5c742d29671169354fe84220b47a) Signed-off-by: Richard Purdie

bitbake: fetch2: Abstract fetcher environment to a function

2022-02-20T16:45:25+00:00

The changing of the environment inside the wget fetcher can race if threading is used, such as with sstate in OE-Core. Abstract the function so the environment can be correct before the function is called, removing the race since the enviroment is then no longer changed. (Bitbake rev: c73bb6023c73f003a160bb02aa4da1b580b86c23) Signed-off-by: Richard Purdie

bitbake: wget: Fix grammar "can happen"

2022-02-20T16:45:25+00:00

(Bitbake rev: 52630eefb5174e4ca357ac57085093a7f5767bd8) Signed-off-by: Zygmunt Krynicki Signed-off-by: Richard Purdie