linux/poky.git/bitbake/lib/bb/fetch2/wget.py, branch 3.5_M2

bitbake: fetch2/wget: do not hardcode tarball compressors in version check

2021-12-08T20:22:53+00:00

(Bitbake rev: ba3aa8591327d43935f000c6884637997438ecb2) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: fetch2/wget: add redirectauth parameter

2021-12-08T20:22:53+00:00

Add a parameter that limits sending Basic authentication in the Authorization header to only the first host and not any that we're redirected to. Ignoring potential security concerns, temporary AWS URLs will reject any request that includes authentication details in both the query parameters (from the redirect) and in the Authorization header. Temporary AWS URLs are now being used for release assets from private Github repositories. According to the previous discussion linked below, they're also in use by bitbucket. See also: https://lore.kernel.org/bitbake-devel/CAC9ffDEuZL-k8199bUyN+8frjw6bg-g=vrumxxtvt+RVParQ8Q@mail.gmail.com/ (Bitbake rev: a6ab32013a4381a1b694ed46caf2c9da932644d0) Signed-off-by: Justin Bronder Signed-off-by: Richard Purdie

bitbake: fetch/wget: Add timeout for checkstatus calls (30s)

2021-11-10T19:25:38+00:00

We had an issue where a webserver serving sstate had filesystem issues so would accept connections but effectively not do anything with them. This causes bitbake to hang whilst processing things like sstate objects inside the checkstatus() calls. It can be replicated by setting up a server like: socat -u TCP4-LISTEN:NNN,fork OPEN:/dev/null and pointing SSTATE_MIRRORS in OE at that address. Adding a timeout to the checkstatus calls of 15s means that whilst the system will pause, it will then continue and not hang entirely. Since there isn't a large transfer here, 30s should be a reasonable response time after which we should fall back to building things ourselves. [YOCTO #13716] (Bitbake rev: edc3b0c3953cab675e29fe295b58cfa84ba811c3) Signed-off-by: Richard Purdie

bitbake: fetch2/wget: Enable ftps

2021-09-16T08:51:14+00:00

The fetcher would fail with: Could not find a fetcher which supports the URL: ftps://... (Bitbake rev: 9e56710c7203b1ec6cbefa758c81b69b697fe1a4) Signed-off-by: Daniel Ammann Signed-off-by: Richard Purdie

bitbake: fetch2/wget: fix 'no_proxy' handling

2021-08-23T07:30:54+00:00

The urllib.request.ProxyHandler constructor only reads the $http_proxy + $https_proxy environment variables. $no_proxy is evaluated later when the url is opened. It is therefore not sufficient to just construct the proxy handler in the | with bb.utils.environment(**newenv): context, but the 'opener.open(r)' call must also be made there. (Bitbake rev: 076baf4fbd328d247508fd399866a397eb34f67e) Signed-off-by: Enrico Scholz Signed-off-by: Richard Purdie

bitbake: fetch2/wget: fetch securely by default

2021-08-12T05:28:01+00:00

The days of broken certificates are behind us now, so instead of always passing --no-check-certificate to wget, don't pass it by default and instead only pass it BB_CHECK_SSL_CERTS = "0". [ YOCTO #14108 ] (Bitbake rev: 4104850dd36096a9ff01836c5fca9ac0e452bcf8) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: fetch2/wget: ensure all variables are set when calling urllib

2021-08-12T05:28:01+00:00

Instead of just exporting the proxy variables when calling into urllib, use bb.utils.environment() to export all of the known variables that are needed for proper connectivity. Specifically, this ensures that SSL_CERT_FILE is set, so that libssl can find the certificates in buildtools environments (Bitbake rev: 116637b0e9aabae7f680b102dbf3577b8a58f049) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: fetch2/wget: when checking latest versions, consider all numerical directories

2021-05-14T06:57:57+00:00

Previously the regex was maching x.y, but wasn't matching x, which is a problem e.g. here: https://download.gnome.org/sources/epiphany/ (the new gnome version scheme adds 40-series at the end). (Bitbake rev: c03101576f447263ea38e8464210d3a3a2c27226) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: fetch2/wget: Avoid crashing when connection drops mid checkstatus

2021-02-26T17:47:56+00:00

If an exception is raised when running host python code, the fetcher immediately crashes, this might be temporary depending on the servers reliability. Catch the exception when the connection was reset and try once again to fetch the data. File: '/usr/lib/python3.8/socket.py', lineno: 669, function: readinto 0665: if self._timeout_occurred: 0666: raise OSError("cannot read from timed out object") 0667: while True: 0668: try: *** 0669: return self._sock.recv_into(b) 0670: except timeout: 0671: self._timeout_occurred = True 0672: raise 0673: except error as e: Exception: ConnectionResetError: [Errno 104] Connection reset by peer (Bitbake rev: d0f5c5905bc664e415a05e3130dfe0ae541d8b3e) Signed-off-by: Alejandro Enedino Hernandez Samaniego Signed-off-by: Richard Purdie

bitbake: logging: Make bitbake logger compatible with python logger

2021-02-10T23:48:16+00:00

The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] (Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie