linux/poky.git/bitbake/lib/bb/fetch2, branch yocto-5.0

bitbake: wget: Make wget --passive-ftp option conditional on ftp/ftps

2024-04-11T07:47:07+00:00

Fedora 40 introduces wget2 as a drop-in replacement for wget. This rewrite does not currently have support for FTP. This causes the wget fetcher to fail complaining about an unrecognized option. Making --passive-ftp conditional based on the protocol used in the SRC_URI limits the scope of the problem. It also gives us an opportunity to build the older wget as a host tool. (Bitbake rev: f10e630fd7561746d835a4378e8777e78f56e44a) Signed-off-by: Rob Woolley Signed-off-by: Richard Purdie

bitbake: fetch2/git: Install Git LFS in local repository config

2024-03-22T16:26:46+00:00

Git uses a lock file to prevent concurrent modifications to the global config, so if unpack tasks for different recipes try to run "git lfs install" simultaneously the operation can fail: error: could not lock config file /home/build/.gitconfig: File exists exit status 255 Run `git lfs install --force` to reset Git configuration. Adding "--local" sets the smudge and clean filters in the local repository's config instead of modifying the user's global config. (Bitbake rev: 328ca4de8422be514fa0d0c9e3cfd36bb9d3e9a7) Signed-off-by: Derek Erdmann Signed-off-by: Richard Purdie

bitbake: fetch2: Fix misleading "no output" msg

2024-03-22T16:26:45+00:00

When a command is run with a non-null log, errors are only output to the log and are not returned in the exception. In that case direct users to that logfile instead of telling the command had no output. (Bitbake rev: 944fe0a77932a5559e01ae6035c4bffa5185ea6a) Signed-off-by: Viswanath Kraleti Signed-off-by: Richard Purdie

bitbake: fetch2: handle URIs with single-valued query parameters

2024-03-22T16:26:45+00:00

Whilst typically the URI query is a list of key-value pairs, that's not actually required by the URI specification. For example: http://example.com/foo?bar is a valid query, but this will result in the fetcher raising an exception: File "bitbake/lib/bb/fetch2/__init__.py", line 265, in __init__ self.query = self._param_str_split(urlp.query, "&") File "bitbake/lib/bb/fetch2/__init__.py", line 293, in _param_str_split for k, v in [x.split(kvdelim, 1) for x in string.split(elmdelim) if x]: ValueError: not enough values to unpack (expected 2, got 1) In this case the query is just "bar", but the fetcher is trying to split it into a key-value pair. The URI object exposes the parsed query explicitly as a dictionary of key-value pairs, so we have to be a little creative here: if a value is None then it isn't a key-value pair, but a bare key. Fix this by handling elements without the deliminator in _param_str_split() (by assigning the value to None), and handle a None value when formatting the query in _param_str_join(). (Bitbake rev: eac583bd4c46f3bb9661852cb6a1448f16147ff1) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: bitbake: fetch2/git: Escape parentheses in git src name

2024-03-22T16:26:45+00:00

FIXES [YOCTO #15404] When using git fetcher on a repo with parentheses in its URL, the invocation of the git clone command will fail. The clone directory is not quoted thus the shell will return an error and won't execute the command. (Bitbake rev: b5624ee5643d881afa004571a096a189ab5389b5) Cc: Philippe Rivest (Bitbake rev: 12f9738577934ad7c99f0770f1392a9d6050e7d6) Signed-off-by: Philippe Rivest Signed-off-by: Richard Purdie

bitbake: fetch/git: Avoid clean upon failure

2024-03-03T16:26:17+00:00

Currently when git fetches fail, it destroys all the existing local clone data. For large repositories this can introduce long build delays when for example, you just typo'd the git revision hash. The git fetcher should be able to recover most directories so when the fetch is for a git repo, avoid removing things unless clean is explicitly called (e.g. a -c cleanall task). (Bitbake rev: 1b3cd039fe19b24bd4be9a0202a98cdcbb0e9443) Signed-off-by: Richard Purdie

bitbake: fetch2: Ensure that git LFS objects are available

2024-02-23T14:34:05+00:00

The current implementation only performs a git lfs fetch alongside of a regular git fetch. This causes issues when the downloaded revision is already part of the fetched repository (e.g. because of moving back in history or the updated revision already being part of the repository at the time of the initial clone). Fix this by explicitly checking whether the required LFS objects are available in the downloade directory before confirming that a downloaded repository is up-to-date. This issue previously went unnoticed as git lfs would silently fetch the missing objects during the `unpack` task. With network isolation turned on, this no longer works, and unpacking fails. (Bitbake rev: cfae1556bf671acec119a6c8bbc4b667a856b9ae) Signed-off-by: Philip Lorenz Signed-off-by: Richard Purdie

bitbake: Revert "bitbake: wget.py: always use the custom user agent"

2024-02-20T12:56:40+00:00

This reverts commit 987ab2a446aab235d6e86e97c05f25fb800d7acc. There's been a report that this breaks downloads from Jfrog Artifactory as self.user_agent is set to 'Mozilla Firefox', and when Artifactory sees that, it sends a response tailored for showing in an interactive browser (which in my opinion it has every right to). If we're using wget, we should say so via wget's default; handling uncooperative servers should be done on per-recipe basis, and ideally with tickets to admins of those servers. (Bitbake rev: feef5cd12e877f42ffcace168d44b0e6eb80a907) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: fetch2/git: Make latest_versionstring extract tags with slashes correctly

2024-02-19T15:08:30+00:00

Before, everything up to the last slash was removed when extracting the names of the tags. This would lead to that a tag such as "agent/11.0.0" would be incorrectly identified as "11.0.0", which would then be treated as a correct version matching "^(?P\d+(\.\d+)+)". (Bitbake rev: 8b21024b9966d5158ac4a77e87ffb935c2a57764) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie

bitbake: fetch2/git: A bit of clean-up of latest_versionstring()

2024-02-19T15:08:30+00:00

This is mostly preparations for the next commit. (Bitbake rev: dcd2abfde55cc59d9869a7c97620b6fc30a52047) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie