linux/poky.git/bitbake/lib/bb/tests/fetch.py, branch 4.2_M3

bitbake: fetch/npmsw: add more short forms for git operations

2023-02-26T11:49:41+00:00

>From the npm-install documentation [1] the CLI provides a set of short forms when the install fetches from git. These include "github:" example: npm install github:mygithubuser/myproject "gist:" example: npm install gist:101a11beef "gitlab:" example: npm install gitlab:mygitlabuser/myproject "bitbucket:" example: npm install bitbucket:mybitbucketuser/myproject Commit 1d8af6aed0a9 [fetch2: npmsw: Add support for github prefix in npm shrinkwrap version] by Stefan Herbrechtsmeier added support for the "github:" but the others would marked as 'Unsupported dependency'. The other prefixes are added in this commit, along with extending the tests to cover some of these. However, there is one more short form for github which npm-install allows which forgoes the prefix altogether. example: npm install mygithubuser/myproject Unfortunately this format is a bit problematic as it lacks any easily identifiable 'marker' to match against, and it could be either the github short form or install from folder format. Experimentation shows that the folder format requires a leading './' or '/', so we use this to rule out the ambiguity. If this approach to folder and github formats disambiguation is incorrect it won't matter anyways as the folder format is unrecognized by the code as-is and thus with this change or without, things would fail. Since we have to be less strict in the check for git operations we move it to be the last install format which we check, such that the less ambiguous formats can be sorted out first. [1] https://docs.npmjs.com/cli/v9/commands/npm-install [Yocto #14236] (Bitbake rev: 0ac6f6cb5d807919ed13a8b7bb3fb551b79c5a71) Signed-off-by: Mark Asselstine Signed-off-by: Richard Purdie

bitbake: tests/fetch: Add real git lfs tests and decorator

2023-02-22T12:03:39+00:00

Added tests that verify that git-lfs works with an actual real git-lfs server. This was not previously the case because the repo in the test was a simulation of git-lfs but not a real git lfs repo. The 2 added tests are almost the same but test that the git lfs file checkout is successfult with or without the lfs=1 flag. The lfs=1 URI parameter is a quirk that triggers 2 different code paths for git lfs. lfs=1, when used on git lfs repositories triggers the git lfs downloading at the fetch bare stage. lfs query parameter unset triggers the git lfs downloading only on checkout as an implicit behavior of git. This leads to possible network access on the unpack stage and outside the DL_DIR. lfs=0 actually disables git-lfs functionality even if supported. (Bitbake rev: d2be7f7f652360f13cd66d0850f3e19ffe2afb0a) Signed-off-by: Paulo Neves Signed-off-by: Richard Purdie

bitbake: tests/fetch: git-lfs restore _find_git_lfs

2023-02-22T12:03:39+00:00

Not restoring the mocked _find_git_lfs leads to other tests failing. (Bitbake rev: 70f848631450bd723c223227c21c60e815ee033d) Signed-off-by: Paulo Neves Signed-off-by: Richard Purdie

bitbake: fetch2/tests: Add parameter to recipe_uri

2023-01-20T17:49:26+00:00

While the parameter is not required it allows testing of possible regression in fetcher code when parameter specified and mirrors are used. (Bitbake rev: 1a21918049091a6d77426dbf8868ffdc14ba1003) Signed-off-by: Pavel Zhukov Signed-off-by: Luca Ceresoli Signed-off-by: Richard Purdie

bitbake: fetch2/tests: Add test for Mercurial

2023-01-18T16:42:27+00:00

Basic test for mercurial functionality including PREMIRRORONLY with disabled network (Bitbake rev: c0e349a5fc92e9c90afc53800c7e83495393ff79) Signed-off-by: Pavel Zhukov Signed-off-by: Luca Ceresoli Signed-off-by: Richard Purdie

bitbake: fetch2/wget.py: correctly match versioned directories

2022-12-11T16:33:19+00:00

When obtaining latest upstream versions, the code needs to check if the existing tarball is in a versioned directory (e.g. component-name/x.y/component-name-x.y.z.tar.gz) and if it is, it needs to first obtain the list of all such versioned directories and then check all of them by going one step up in the directory hierarchy. Existing code was returning a correct match when the component name did not have numbers, e.g. a check on 'source/epiphany/43/' would return 43, but was stopping too soon when the component name itself had numbers ('source/libxml2/2.10/' would return libxml2). This change ensures the last match is taken instead of the first. Also, adjust the fetcher tests to check that versioned directories are correctly traversed in this case (e.g. the step to go one level up is taken and a new tarball is discovered in a different versioned directory). (Bitbake rev: b6601be22c6d776327acdcd1fa931400f41ac786) Signed-off-by: Alexander Kanavin Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: tests/fetch: Allow handling of a file:// url within a submodule

2022-10-26T22:02:13+00:00

CVE-2022-39253 in git meant file:// urls within submodules were disabled. Add a parameter to the commands in the tests to allow this to continue to work. (Bitbake rev: 209f7ba352b60722830157054e3fc56cb9c693eb) Signed-off-by: Richard Purdie

bitbake: tests: bb.tests.fetch.URLHandle: add 2 new tests

2022-10-26T11:32:08+00:00

Add a test for special characters in user and password to qualify decodeurl() inspired by a bug report describing that '=' signs in a password was problematic. Add a second test to qualify decodeurl() as related to the change in commit 628c4bf6c89b [fetch2/__init__: handle @ in package names]. Relates to [YOCTO #14476] (Bitbake rev: ee04cf09c7022168c035affa654773652a49793e) Signed-off-by: Mark Asselstine Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: tests: Add test for possible gitsm deadlock

2022-09-01T09:07:00+00:00

(Bitbake rev: b0506480baa9bcf3ef645b0aed5a07ad9950245c) Signed-off-by: Pavel Zhukov Signed-off-by: Richard Purdie

bitbake: tests: Add Timeout class

2022-09-01T09:07:00+00:00

The class and exception aim to test rare cases there deadlocks are possible. Can be used in context managers: with Timeout(): do_deadlock() (Bitbake rev: c5fcdd804d422f959a189b270d72123a50e74da6) Signed-off-by: Pavel Zhukov Signed-off-by: Richard Purdie