linux/poky.git/bitbake/lib/bb/fetch2, branch yocto-3.3.2

bitbake: fetch2: add check for empty SRC_URI hash string

2021-07-06T22:28:30+00:00

No error was being reported when the hash string was set to empty. For example: SRC_URI[md5sum] = "" On a related note (not a bug): Because whitespace in the string will result in a checksum mismatch, the error message was updated to make it a little clearer why the error was thrown. For example: SRC_URI[md5sum] = " " or SRC_URI[md5sum] = " 209f8326f5137d8817a6276d9577a2f1" Now creates a message like this: File: '/home/scott/yocto-cache/downloads/rsync-3.2.3.tar.gz' has md5 checksum '209f8326f5137d8817a6276d9577a2f1' when ' 209f8326f5137d8817a6276d9577a2f1' was expected [YOCTO #14232] (Bitbake rev: 731fb52eb03338c0bdb2a2256c22c64c22bcbace) Signed-off-by: Scott Weaver Signed-off-by: Richard Purdie (cherry picked from commit a13510d0028e234ea2f4744b0d0c38558395c70f) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: fetch/svn: Fix parsing revision of SVN repos with redirects

2021-07-06T22:28:30+00:00

svn was printing a message when encountering HTTP redirects. This confused the revision parser. (Bitbake rev: 40b4e2dcd24d3ed01d35d705928133aaa1c4a93b) Signed-off-by: Harald Brinkmann Signed-off-by: Richard Purdie (cherry picked from commit a944a335f8f4c4fe5df55f3d7d8e757bd2835146) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: fetch2/wget: when checking latest versions, consider all numerical directories

2021-05-20T20:40:07+00:00

Previously the regex was maching x.y, but wasn't matching x, which is a problem e.g. here: https://download.gnome.org/sources/epiphany/ (the new gnome version scheme adds 40-series at the end). (Bitbake rev: 8eddd1808387025a22d8ad1b009c2894d19bf79b) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: fetch/gitsm: Fix crash when using git LFS and submodules

2021-04-27T14:22:58+00:00

Gitsm fetcher crashes when cloning a repository that contains LFS files. This happens because the unpack method is called during download, but the submodules have not been downloaded yet at this point. This issue was introduced in this commit: 977b7268bf4fd425cb86d4a57500350c9b829162 [YOCTO #14283] (Bitbake rev: 88d1d2b65a70081389a1c8f9b590a013a1cb4452) Signed-off-by: Niels Avonds Signed-off-by: Richard Purdie (cherry picked from commit 26caedc4d2e9b5a0f1d57f9291754a7f6c5e437e) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: bitbake: Add Azure Storage fetcher implementation

2021-03-11T14:04:45+00:00

Allows bitbake to fetch from an Azure Storage account. The fetcher submodule is compatible with the az:// URI protocol, its functionality is based on bitbakes wget fetcher, superior in performance to using a propietary tool like azcopy which can handle cloud storage account operations with more functionality (that we dont need in a fetcher) but less compatibility. A sample URI uses can be defined in the following way: SRC_URI = "az://.blob.core.windows.net//foo.tar.xz" This fetcher can easily be used with PREMIRRORS and SSTATE_MIRRORS, e.g.: SSTATE_MIRRORS = "file://.* az://.blob.core.windows.net/sstate-cache/PATH;downloadfilename=PATH \n" PREMIRRORS_prepend = "\ git://.*/.* az://.blob.core.windows.net/downloads/ \n \ ftp://.*/.* az://.blob.core.windows.net/downloads/ \n \ http://.*/.* az://.blob.core.windows.net/downloads/ \n \ https://.*/.* az://.blob.core.windows.net/downloads/ \n \ " Can also be used with non-public access Azure Storage accounts/containers via a Shared Access Signature by declaring the AZ_SAS variable which will be automatically used by the fetcher: AZ_SAS="?sv=2000-01-01&ss=...&sig=somesignature" (Bitbake rev: b103b02f2ce2f8f5079f17ec1a854f904c2110a4) Signed-off-by: Alejandro Enedino Hernandez Samaniego Signed-off-by: Richard Purdie

bitbake: fetch2/wget: Avoid crashing when connection drops mid checkstatus

2021-02-26T17:47:56+00:00

If an exception is raised when running host python code, the fetcher immediately crashes, this might be temporary depending on the servers reliability. Catch the exception when the connection was reset and try once again to fetch the data. File: '/usr/lib/python3.8/socket.py', lineno: 669, function: readinto 0665: if self._timeout_occurred: 0666: raise OSError("cannot read from timed out object") 0667: while True: 0668: try: *** 0669: return self._sock.recv_into(b) 0670: except timeout: 0671: self._timeout_occurred = True 0672: raise 0673: except error as e: Exception: ConnectionResetError: [Errno 104] Connection reset by peer (Bitbake rev: d0f5c5905bc664e415a05e3130dfe0ae541d8b3e) Signed-off-by: Alejandro Enedino Hernandez Samaniego Signed-off-by: Richard Purdie

bitbake: logging: Make bitbake logger compatible with python logger

2021-02-10T23:48:16+00:00

The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] (Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: fetch/git: download LFS content too during do_fetch

2021-01-23T17:10:46+00:00

Insert an explicit pass to fetch all blobs needed by Git LFS, during the fetch() function. This avoids the default behavior of Git LFS to wait until 'git checkout' to begin downloading the blobs pointed to by LFS records. Network access is not allowed at that point in the recipe's lifecycle. [YOCTO #14191] (Bitbake rev: 0efac66043662e7a2027192f50e92e982db2ba1c) Signed-off-by: Matt Hoosier Signed-off-by: Richard Purdie

bitbake: fetch2: handle empty elements in _param_str_split

2021-01-23T17:09:26+00:00

_param_str_split is used to split ?foo=1;bar=2 into a dictionary, but throws an exception if a lone semicolon is used as the value doesn't split into two items. Fix by checking that the result of the first split has content. (Bitbake rev: 7662f8c8676d87cb318f811423cc02fe8cb146f6) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: fetch2/perforce: Fix localfile to include ud.module

2021-01-23T17:09:26+00:00

As reported by Katu Txakur, the output depends on the module path set so this needs to be accounted for in localfile. (Bitbake rev: 9861ed37bb1c5d09c3b4852d2a252e3f3c86ab14) Signed-off-by: Richard Purdie