linux/poky.git/bitbake/lib/bb/tests, branch halstead/hashclient

bitbake: fetch2: handle URIs with single-valued query parameters

2024-03-22T16:26:45+00:00

Whilst typically the URI query is a list of key-value pairs, that's not actually required by the URI specification. For example: http://example.com/foo?bar is a valid query, but this will result in the fetcher raising an exception: File "bitbake/lib/bb/fetch2/__init__.py", line 265, in __init__ self.query = self._param_str_split(urlp.query, "&") File "bitbake/lib/bb/fetch2/__init__.py", line 293, in _param_str_split for k, v in [x.split(kvdelim, 1) for x in string.split(elmdelim) if x]: ValueError: not enough values to unpack (expected 2, got 1) In this case the query is just "bar", but the fetcher is trying to split it into a key-value pair. The URI object exposes the parsed query explicitly as a dictionary of key-value pairs, so we have to be a little creative here: if a value is None then it isn't a key-value pair, but a bare key. Fix this by handling elements without the deliminator in _param_str_split() (by assigning the value to None), and handle a None value when formatting the query in _param_str_join(). (Bitbake rev: eac583bd4c46f3bb9661852cb6a1448f16147ff1) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: fetch2: Ensure that git LFS objects are available

2024-02-23T14:34:05+00:00

The current implementation only performs a git lfs fetch alongside of a regular git fetch. This causes issues when the downloaded revision is already part of the fetched repository (e.g. because of moving back in history or the updated revision already being part of the repository at the time of the initial clone). Fix this by explicitly checking whether the required LFS objects are available in the downloade directory before confirming that a downloaded repository is up-to-date. This issue previously went unnoticed as git lfs would silently fetch the missing objects during the `unpack` task. With network isolation turned on, this no longer works, and unpacking fails. (Bitbake rev: cfae1556bf671acec119a6c8bbc4b667a856b9ae) Signed-off-by: Philip Lorenz Signed-off-by: Richard Purdie

bitbake: fetch2/git: Make latest_versionstring extract tags with slashes correctly

2024-02-19T15:08:30+00:00

Before, everything up to the last slash was removed when extracting the names of the tags. This would lead to that a tag such as "agent/11.0.0" would be incorrectly identified as "11.0.0", which would then be treated as a correct version matching "^(?P\d+(\.\d+)+)". (Bitbake rev: 8b21024b9966d5158ac4a77e87ffb935c2a57764) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie

bitbake: tests/fetch: Make test_git_latest_versionstring support a max version

2024-02-19T15:08:30+00:00

In some cases, the version found by latest_versionstring() may be higher than the real version. Make it possible to specify a maximum version so that this case can be detected. (Bitbake rev: 9134d4777109bc78410c3e641420d9a78b485e33) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie

bitbake: tests/fetch.py: add test case for using premirror in restricted network

2024-02-19T15:08:30+00:00

We had issue when BB_ALLOWED_NETWORKS is set and `bitbake grpc-native -c fetch' failed even with all contents available in PREMIRRORS. Add a test case to ensure no regression in the future. (Bitbake rev: 80c91ceb81b1cae203067af58d3f1fe9c619ae83) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie

bitbake: tests/fetch.py: add multiple fetches test

2024-02-19T12:52:30+00:00

Fetch from premirror few times to emulate multiple machines sharing same clonedir or few rebuilds of the package from (pre)mirror Regression test for [Yocto #15369] (Bitbake rev: 7fcbac574c68f16b95ab7abb2874931d168d3c9e) Signed-off-by: Pavel Zhukov Signed-off-by: Richard Purdie

bitbake: tests/fetch: support git's safe.bareRepository

2024-02-19T12:00:59+00:00

When git is configured with safe.bareRepository=explicit [1], the bitbake selftests fail miserably. LWN has an article about the problem that this configuration option addresses and why it is useful in [2]. It also seems that it is being rolled out in some environments as a default for users. In order to allow having this configuration turned on for a user's environment in general, the fetcher has to be tought to use --git-dir= for all relevent git operations. The alternative, implemented here, is to forcibly turn off that option for all git operations. In the future, we could look into converting these to using the --git-dir= command line argument instead. Link: https://git.kernel.org/pub/scm/git/git.git/tree/Documentation/config/safe.txt#n1 [1] Link: https://lwn.net/Articles/892755/ [2] (Bitbake rev: a45e14a7343e36101e45639931322e5649587f57) Signed-off-by: André Draszik Signed-off-by: Richard Purdie

bitbake: tests: fetch.py: use real subversion repository

2024-02-08T10:59:32+00:00

* github no longer provides support for subversion clients: https://docs.github.com/en/enterprise-server@3.11/get-started/working-with-subversion-on-github/support-for-subversion-clients it was shut down on 2024-01-08: https://github.blog/2023-01-20-sunsetting-subversion-support/ and this test was now failing with: ====================================================================== ERROR: test_external_svn (bb.tests.fetch.SVNTest.test_external_svn) ---------------------------------------------------------------------- Traceback (most recent call last): File "/OE/layers/bitbake/lib/bb/tests/fetch.py", line 1287, in test_external_svn fetcher.download() File "/OE/layers/bitbake/lib/bb/fetch2/__init__.py", line 1896, in download raise FetchError("Unable to fetch URL from any source.", u) bb.fetch2.FetchError: Fetcher failure for URL: 'svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2'. Unable to fetch URL from any source. Stdout: Fetch svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2 Failed to fetch URL svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2, attempting MIRRORS if available Fetcher failure: Fetch command export PSEUDO_DISABLED=1; /usr/bin/env svn --non-interactive --trust-server-cert co --no-auth-cache -r 2 file:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project/trunk@2 trunk failed with exit code 1, output: A trunk/README.md U trunk Checked out revision 2. svn: warning: W205011: Error handling externals definition for 'trunk/bitbake': svn: warning: W170013: Unable to connect to a repository at URL 'https://github.com/PhilipHazel/pcre2.git' svn: E205011: Failure occurred processing one or more externals definitions in the rare cases where subversion was still installed on the host running bitbake-selftest :). to avoid this use still alive repository from https://svn.apache.org/ and pick something rather small and only the trunk subdirectory which is fast to fetch: svn co https://svn.apache.org/repos/asf/serf/trunk takes just 2 sec here adjust expected dir/file to use "protocols/fcgi_buckets.h" instead of "trunk/README" (Bitbake rev: a735898abcf056f897c9350bb128a5637e6b4617) Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie

bitbake: ast: Fix EXPORT_FUNCTIONS bug

2024-01-10T13:55:33+00:00

If you have two classes, both of which set EXPORT_FUNCTIONS for the same funciton and a standard funciton definition for the function that is exported, the export function can sometimes overwrite the standard one. The issue is that the internal flag the code uses isn't ovweritten if the variable is giving a new value. Fix the issue by using a comment in the code that is injected so that we know if it is ours or not. Also add some testing for EXPORT_FUNCTIONS, not perfect but a start. (Bitbake rev: 66306d5151acb0a26a171c338d8f60eb9eb16c6b) Signed-off-by: Richard Purdie

bitbake: bitbake: tests/event: Add test_lineno_in_eventhandler

2024-01-10T13:55:33+00:00

Add test_lineno_in_eventhandler to test lineno in eventhandler. (Bitbake rev: 4e5de537bebb68180c5755858c81b095eb9ae2f6) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie