linux/poky.git/bitbake/lib/bb/tests, branch thud

bitbake: tests/fetch: Allow wget upgrade tests to run against a local server

2020-09-09T08:44:53+00:00

Currently these tests rely upon multiple uptream webservers which may change or be unavailable. Add local copies of the test data, copy the httpserver from OE-Core (used for testing there) and run these tests against a local server instead. (Bitbake rev: a21671e8a483ba8a6986d961987eda2d36ec61ca) Signed-off-by: Richard Purdie

bitbake: tests/fetch: Switch from git.infradead.org to a YP mirror

2020-07-10T20:10:54+00:00

Upstream is unavailable, breaking tests. Switch to a YP mirror since if we can't reach that there are bigger problems. This should remove a source of intermittent failures on the autobuilder. (Bitbake rev: f4e60b29df88393302957c5bbdbe24ca38c4633c) Signed-off-by: Richard Purdie

bitbake: tests/fetch: Resolve fetch error in bitbake-selftest

2019-09-09T08:58:09+00:00

FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0]) AssertionError: '' is not true : Could not find upstream version for db [YOCTO #13496] The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget. Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the recipe. Also change the version being checked. (Bitbake rev: 8a58c3c64240c6ab14858d18e6b89febdb315311) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: gitsm: Add need_update method to determine when we are going to a new SRCREV

2019-06-18T10:31:10+00:00

If the system had previously fetched a source repository for use by gitsm, and then the SRCREV was updated and the new commit already existed, the system would not re-evaluate the submodules and update them accordingly. The cause of this issue was that need_update was being used, unmodified, from the base git fetcher. It did not have any knowledge, nor did it care if we were moving commits and needed to re-evaluate what was happening due to this switch. To fix the issue, during the download process we add all processed (by gitsm) srcrevs to the git config file, as bitbake.srcrev. This allows us to use a new need_update function that not only checks if the git commit is present, but if we have previously processed this commit to ensure all of the submodule components are also present. This approach is used, instead of iterating over the submodules in need_update to avoid a potential race condition that has affected us in the past. The need_update is called only with the parent locking. Any time we need to dive into the submodules, we need to lock, and unlock them, at each stage. This opens the possibility of errors in either the code, or unintended race conditions with rm_work. This issue was discovered by William A. Kennington III . The included test case was also written by him, and included unmodified. (Bitbake rev: 4ce92f43eeac6a4bfd06e8567fa6891614b5b3b0) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie

bitbake: gitsm: Fix a bug where the wrong path was used for the submodule init

2019-06-18T10:31:10+00:00

Because we are trying to avoid network activity and use our own fetcher, the system emulates the behavior of 'git submodule init'. git submodule init uses the .gitmodules file, where typically the module name and path are the same. However, in this case the module name and path (in the tree) were different. i.e.: [submodule "edgelet/hsm-sys/azure-iot-hsm-c/deps/azure-c-shared-utility"] path = edgelet/hsm-sys/azure-iot-hsm-c/deps/c-shared url = https://github.com/Azure/azure-c-shared-utility.git Previously the code assumed the 'path' was both the checkout location under .git/modules, as well as the path to extract the components. This proved to be incorrect as the .git/modules path needs to match the submodule 'name'. This causes the components that were fetched to be initialized in the wrong location, which later caused the 'git submodule update' process to skip not properly initialized modules. A test case was added for this specific case to ensure a regression does not appear in the future. (Bitbake rev: ffd7ed530a17d22df576d986ac78428a6979e79c) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie

bitbake: bitbake: fix version comparison when one of the versions ends in .

2019-06-15T10:06:38+00:00

Previously, this would happen: ====================================================================== ERROR: test_vercmpstring (bb.tests.utils.VerCmpString) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/alexander/development/poky/bitbake/lib/bb/tests/utils.py", line 45, in test_vercmpstring result = bb.utils.vercmp_string('1.', '1.1') File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 143, in vercmp_string return vercmp(ta, tb) File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 135, in vercmp r = vercmp_part(va, vb) File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 124, in vercmp_part elif ca < cb: TypeError: '<' not supported between instances of 'NoneType' and 'int' ---------------------------------------------------------------------- (Bitbake rev: bd953d56d007a8bfa5ecb6e753da4abfb035f9f2) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: gitsm: The fetcher did not process some recursive submodules properly.

2019-03-24T16:42:22+00:00

Move the submodule processing outside of the if statement to avoid any optimizations that may happen. Update the test cases to include the additional case, and split the other test cases into individual tests to make it easier to figure out what the failure may be. (Bitbake rev: 7c1eb51d1e8a4c5f39bf9dddf05fb0b3598da72b) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit 0ec98c01ae50f95c9c74acf53013ac59e0e72b08) Signed-off-by: Scott Murray Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: gitsmy.py: Fix unpack of submodules of submodules

2019-03-24T16:42:22+00:00

If the submodule is in a subdirectory, it needs to have that structure preserved. This means the unpack path needs to be in the 'dirname' of the final path -- since the unpack directory name is specified in the URI. Additional specific test cases were added to ensure this is working properly based on two recent error reports. (Bitbake rev: acca06d060e49b2441562b4dc94416af9ab8187e) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit 8c8ecec2a722bc2885e2648d41ac8df07bdf660d) Signed-off-by: Scott Murray Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: gitsm.py: Fix relative URLs

2019-03-24T16:42:22+00:00

Prior code happened to work with relative URLs, when the code was recently restructured it caused all relative urls to no longer work. Restore the prior code flow for relative support and better comment why that code is there. (Bitbake rev: 39afa6ead1d72813b2a294f065d759bad08fb53d) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit 14bb383af0ca98e0e04ec217e537b6a899f3658e) Signed-off-by: Scott Murray Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: gitsm.py: Refactor the functions and simplify the class

2019-03-24T16:42:22+00:00

The update_submodules and unpack_submodules functions were nearly indentical, so we made a common function where the different behavior could be passed in by the download and unpack users. The new function is process_submodules. Moved the parse_gitmodules function under the new process_submodules, since there are no external callers. Refactor the file relative path processing to the URL translation code. We also add a warning to the translation if a relative ssh URL has been detected. Since this can cause a problem. In the case of a relative URL that does not work after being translated, it should be possible to use the MIRROR functions to manual translate the generated relative URL into one that works properly. Remove 'git config' processing on download contents. It turns out this is not necessary since all of the later components work using the git fetcher. Limit the 'git submodule update' call to only when unpacking a non-bare repository. Submodules are always loaded as bare, so this prevents intermediate unpacks from being attempted. Finally, the test cases were updated and the new commit ids in the test repository were updates as well. (Bitbake rev: 8b02eb8e5d8e3a09e19ed96a4ccdf5f755e97a74) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit 610dbee5634677f5055e2b36a3043cd197fb8c51) Signed-off-by: Scott Murray Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie