summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/tests/fetch.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: tests/fetch: Resolve fetch error in bitbake-selftestrockoArmin Kuster2019-11-071-2/+2
| | | | | | | | | | | | | | | | | | | | | FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0]) AssertionError: '' is not true : Could not find upstream version for db [YOCTO #13496] The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget. Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the recipe. Also change the version being checked. (Bitbake rev: 1b989fb76e3e5f044c2e14c7b26a5562803fde9d) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Update gnome.org urls after upstream changesRichard Purdie2018-08-291-2/+2
| | | | | | (Bitbake rev: cdda992d4eedcf8e23b9a7222c16d49dee1586bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Add ftp test urlRichard Purdie2017-12-041-0/+1
| | | | | | | | | | | | Add in a tets ftp url so we ensure ftp urls contnue to work after the loss of the ftp.gnu.org ones. (Bitbake rev: 7016bd9c4b05df2e888ec98e37a8ae6f3ac398bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e1e8565b5e19dd3f7ef6e7e41932456adaa3df81) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: use subtests in the wget testsRoss Burton2017-12-041-8/+10
| | | | | | | | | | | | | | As we test multiple URLs in this these tests and one failing abandons the test, use subtests so all URLs are tested. This should help us identify patterns in the failing URLs. (Bitbake rev: 0eadcf94540c7e4a634c5c1e873658b65996f334) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c4c4465b32e82d4b6e46a44e776be5039aef6b18) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Switch gnu.org urls from ftp -> http/httpsRichard Purdie2017-12-041-3/+3
| | | | | | | | | | | | | | | The ftp server at ftp.gnu.org is likely to be retired at some point soon so siwtch over to the http/https services. This means bitbake-selftest doesn't have ftp test urls, however finding stable ftp test servers is proving increasingly hard. (Bitbake rev: 6497a030463cc7fd61bb8319d4c3ec824003c2fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 892a08245ddb21a464aeb37d3e32377e99dd7e2b) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: skip network tests the idiomatic wayRoss Burton2017-12-041-212/+222
| | | | | | | | | | | | | Instead of not even having the test functions if network tests are disabled, use a custom decorator to mark the network tests and skip them. (Bitbake rev: 618cf9693b9f4b48208603b2359f5717a6a35f8f) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit cc420f430b1dafd9ca944bea259a564aaab34595) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: handle network failures gracefullyRoss Burton2017-09-111-0/+2
| | | | | | | | | | | | | | If there is a network failure the return value from latest_versionstring() is ('','') which later causes an exception when comparing versions. Improve this by checking the return value and failing the test early. [ YOCTO #12053 ] (Bitbake rev: 3f034d2172bf64ecc43577b43e0cf032a54b1358) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: fix GitShallowTest.test_bitbakeChristopher Larson2017-07-301-1/+1
| | | | | | | | | | | | | | | `git fetch --tags` seems to interact badly with `mirror=fetch`, resulting in the regular branches not being fetched, so drop the unnecessary `--tags`. This fixes this unit test failure: `bb.fetch2.FetchError: Fetcher failure: Unable to resolve 'master' in upstream git repository in git ls-remote output for /tmp/tmp4ag_mgmn/gitsource` [YOCTO #11698] (Bitbake rev: 2d0203fae08c5ff8dc3e9afaa9a819abc4a1af6f) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for removing arbitrary revs for shallowChristopher Larson2017-06-021-1/+65
| | | | | | | | | | | | | | | | | | | | In certain cases, it's valuable to be able to exert more control over what history is removed, beyond srcrev+depth. As one example, you can remove most of the upstream kernel history from a kernel repository, keeping predominently the non-publically-accessible content. If the repository is private, the history in that repo couldn't be restored via `git fetch --unshallow`, but upstream history could be. Example usage: # Remove only these revs, not at a particular depth BB_GIT_SHALLOW_DEPTH_pn-linux-foo = "0" BB_GIT_SHALLOW_REVS_pn-linux-foo = "v4.1" (Bitbake rev: 97f856f0455d014ea34c28b1c25f09e13cdc851b) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/gitannex: add support for shallow mirror tarballsChristopher Larson2017-06-021-0/+17
| | | | | | | | | | | | When we're building from a shallow mirror tarball, we don't want to do anything with ud.clonedir, as it's not being used when we unpack. As such, disable updating annex in that case. Also include annex files in the shallow tarball. (Bitbake rev: ca0dd3c95502b22c369fbf37f915f45e02c06887) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/gitsm: add support for shallow mirror tarballsChristopher Larson2017-06-021-4/+27
| | | | | | | | | | | | | When we're building from a shallow mirror tarball, we don't want to do anything with ud.clonedir, as it's not being used when we unpack. As such, disable updating the submodules in that case. Also include the repositories in .git/modules in the shallow tarball. It does not actually make the submodule repositories shallow at this time. (Bitbake rev: 6c0613f1f2f9d4f009545f82a9173e80396f9d34) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for keeping extra refs for shallowChristopher Larson2017-06-021-0/+42
| | | | | | | | | | | | | | | | | | | | By default, all unused refs (branches & tags) are removed from the repository, as shallow processing scales with the number of refs it has to process. Add the ability to explicitly specify additional refs to keep. This is particularly useful for recipes with custom checkout processes, or whose git-based versioning requires a tag be available (i.e. for `git describe --tags`). The new `BB_GIT_SHALLOW_EXTRA_REFS` variable is a space-separated list of refs, fully specified, and support wildcards. Example usages: BB_GIT_SHALLOW_EXTRA_REFS = "refs/tags/v1.0" BB_GIT_SHALLOW_EXTRA_REFS += "refs/heads/*" (Bitbake rev: 1771934cd9f8b5847c6fcae0a906fb99d6b0db16) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: support per-branch/per-url depths for shallowChristopher Larson2017-06-021-2/+33
| | | | | | | | | | | | | | | | Allow the user to explicitly adjust the depth for named urls/branches. The un-suffixed BB_GIT_SHALLOW_DEPTH is used as the default. Example usage: BB_GIT_SHALLOW_DEPTH = "1" BB_GIT_SHALLOW_DEPTH_doc = "0" BB_GIT_SHALLOW_DEPTH_meta = "0" (Bitbake rev: 9dfc517e5bcc6dd203a0ad685cc884676d2984c4) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for shallow mirror tarballsChristopher Larson2017-06-021-0/+299
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds support to the git fetcher for fetching, using, and generating mirror tarballs of shallow git repositories. The external git-make-shallow script is used for shallow mirror tarball creation. This implements support for shallow mirror tarballs, not shallow clones. Supporting shallow clones directly is not really doable for us, as we'd need to hardcode the depth between branch HEAD and the SRCREV, and that depth would change as the branch is updated. When BB_GIT_SHALLOW is enabled, we will always attempt to fetch a shallow mirror tarball. If the shallow mirror tarball cannot be fetched, it will try to fetch the full mirror tarball and use that. If a shallow tarball is to be used, it will be unpacked directly at `do_unpack` time, rather than extracting it to DL_DIR at `do_fetch` time and cloning from there, to keep things simple. There's no value in keeping a shallow repository in DL_DIR, and dealing with the state for when to convert the clonedir to/from shallow is not worthwhile. To clarify when shallow is used vs a real repository, a current clone is preferred to either tarball, a shallow tarball is preferred to an out of date clone, and a missing clone will use either tarball (attempting the shallow one first). All referenced branches are truncated to SRCREV (that is, commits *after* SRCREV but before HEAD are removed) to further shrink the repository. By default, the shallow construction process removes all unused refs (branches/tags) from the repository, other than those referenced by the URL. Example usage: BB_GIT_SHALLOW ?= "1" # Keep only the top commit BB_GIT_SHALLOW_DEPTH ?= "1" # This defaults to enabled if both BB_GIT_SHALLOW and # BB_GENERATE_MIRROR_TARBALLS are enabled BB_GENERATE_SHALLOW_TARBALLS ?= "1" (Bitbake rev: 5ed7d85fda7c671be10ec24d7981b87a7d0d3366) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: git-make-shallow: add script to make a git repo shallowChristopher Larson2017-06-021-0/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This script will be used by the git fetcher to create shallow mirror tarballs. usage: git-make-shallow [-h] [--ref REF] [--shrink] REVISION [REVISION ...] Remove the history of the specified revisions, then optionally filter the available refs to those specified. positional arguments: REVISION a git revision/commit optional arguments: -h, --help show this help message and exit --ref REF, -r REF remove all but the specified refs (cumulative) --shrink, -s shrink the git repository by repacking and pruning While git does provide the ability to clone at a specific depth, and fetch all remote refs at a particular depth, the depth is across all branches/tags, and doesn't provide the flexibility we need, hence this script. Refs (branches+tags) can be filtered, as the process of history removal scales up rapidly with the number of refs. Even the existing `git fetch --depth=` is extremely slow on an upstream kernel repository with all the branches and tags kept. This uses the same underlying mechanism to implement the history removal which git itself uses (.git/shallow), and the results, when configured similarly, are in line with the results git itself produces with `fetch --depth`. (Bitbake rev: 0254020f0e1911c0eaf99111b91828d2a74a4ee1) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: handle absolute paths in subdirRoss Burton2016-09-221-0/+9
| | | | | | | | | | | | | | | | Currently if you use the subdir parameter in a SRC_URI and pass an absolute path then it gets appended to the unpack directory instead of being used directly. This is inconvenient as it may be useful to use ${S} when you want to unpack a file into the source tree. Change this behaviour so that absolute paths are used directly instead of being appended to the root directory. To ensure that recipes cannot write files to an arbitrary location enforce that the subdir starts with the unpack root. (Bitbake rev: c3873346c6fa1021a1d63bddd9b898a77c618432) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests: add unit tests for the usehead url parameterMarkus Lehtonen2016-08-201-0/+30
| | | | | | | | | [YOCTO #9351] (Bitbake rev: 63031c0236ace10a9d52b9db9bbb892c1b4bf7db) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-selftest: introduce BB_TMPDIR_NOCLEANMarkus Lehtonen2016-08-201-1/+4
| | | | | | | | | | Set this env variable to 'yes' to preserve temporary directories used by the fetcher tests. Useful for debugging tests. (Bitbake rev: 04132b261df9def3a0cff14c93c29b26ff906e8b) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/tests/fetch: remove URL that doesn't exist anymoreRoss Burton2016-08-171-1/+0
| | | | | | | | | | The CUPS ipptool URL we were checking now redirects to github where the tarball isn't present, so remove it from the test suite. (Bitbake rev: 4b50895fb3462b21e3874a2e99c363c8d05e89e6) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: Use OrderedDict for url parametersRichard Purdie2016-05-131-5/+6
| | | | | | | | Without this, the dict can reorder causing sanity test failures. (Bitbake rev: ca8c91acc9396385834b266d4e8b84d917e5e298) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: daemonize/prserv/tests/fetch: Convert file() -> open()Richard Purdie2016-05-131-1/+1
| | | | | | | | Use python3 compatible functions. (Bitbake rev: e6a0296ba29c3fbc8417d1df7a01d50562668a41) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: Improve unit tests for trusted network checkOlof Johansson2016-04-031-35/+42
| | | | | | | | | | | | The tests were skipped when running without network even though they didn't require network. This commit also adds a test case for URLs with ports in them (the ports should not be considered when doing trusted network checks). (Bitbake rev: 77747de6b20538063eef3b188489a35bef225359) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb/tests/fetch: Update cups urlRichard Purdie2016-03-211-1/+1
| | | | | | | | | Update the upstream url used for testing cups versions after upstream website changes. (Bitbake rev: 5f06041d4936fc22297945bbbad7020bfa9083c6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fixes copying of file://dir; subdir=foo, bug 6128 and bug 6129Alexander Shashkevich2016-03-021-8/+4
| | | | | | | | | | | | | | When in SRC_URI appears file://dir;subdir=foo unpacker copies 'dir' to ${WORKDIR}, not ${WORKDIR}/foo as it should be. These changes are fixing following bugs as well: Bug 6128 - Incorrect wildcard unpack behaviour in fetcher Bug 6129 - Local directories unpack to a different location than local files (Bitbake rev: e659a3b0c2771679057ee3e13cd42e6c62383ff2) Signed-off-by: Alexander Shashkevich <alex@stunpix.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/__init__: Fix decodeurl to better handle urls without pathsRichard Purdie2016-02-281-2/+40
| | | | | | | | | | | | | | | | | | | | | If we specify urls such as npm://somehost;someparams the fetcher currently does a poor job of handling mirrors of these urls due to deficiencies in the way decodeurl works. This is because "somehost" is returned as a path, not a host. This tweaks the code so that unless its a file url, the host is returned correctly. This patch also adds test cases for these urls to the exist set of test urls. We need to tweak the URI() class since this thinks this is a relative url which is clearly isn't. We also need to handle the fact that encodeurl will error if passed a url of this form (it would want the path to be '/'. (Bitbake rev: 83203cd2e677706e0111892a7843b83263cb8bd9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: fallback to GET if HEAD is rejected in checkstatus()Ross Burton2016-01-261-0/+2
| | | | | | | | | | | | | | | The core change here is to fall back to GET requests if HEAD is rejected in the checkstatus() method, as you can't do a HEAD on Amazon S3 (used by Github archives). This meant removing the monkey patch that the default method was GET and adding a fixed redirect handler that doesn't reset to GET. Also, change the way the opener is constructed from an if/elif cluster to a conditionally constructed list. (Bitbake rev: 6ec70d5d2e330b41b932b0a655b838a5f37df01e) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: rename REGEX, REGEX_URI, and GITTAGREGEX.Alexander Kanavin2015-12-081-5/+5
| | | | | | | | | | | | Rename REGEX to UPSTREAM_CHECK_REGEX, REGEX_URI to UPSTREAM_CHECK_URI, and GITTAGREGEX to UPSTREAM_CHECK_GITTAGREGEX to better reflect their purpose and to reflect a common namespace. (Bitbake rev: f0a9e783f9969573fd74edfa241ef14f18ac684e) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: Fix recursion failure in url mappingMark Hatle2015-09-181-2/+5
| | | | | | | | | | | | | | | Instead of reproducessing the same line over and over and over, we remove the current line from the mirror list. This permits us to re-evaluate the list while excluding all matches that have previousily occured. Without this fix, adding this test results in a failure: RuntimeError: maximum recursion depth exceeded in cmp (Bitbake rev: 24a8e9a5b0ba145ae589178d74365c986ebca325) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/__init__.py: uri_replace regex handlingMark Hatle2015-09-181-0/+7
| | | | | | | | | | | | | | | | | | | | | | We should only substitute one time. If we do it without a max count, we can end up matching over and over. Before this change: https://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz with a mirror of https://.*/[^/]* http://AAAA/A/A/A/ would end up either recursing indefinitely or result in: http://AAAA/A/A/A/A/A/A/A/A/A/bitbake-1.0.tar.gz (Bitbake rev: 4d254e02e2867dd9a6663508c8ca9f2733af71a8) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: Updated test name FetchMethodTest -> ↵Aníbal Limón2015-07-221-1/+1
| | | | | | | | | | | FetchLatestVersionTest Change the test name to be more specific on what is tested. (Bitbake rev: 3e39156bc330c4c726058a5b9c13d33e2daad89f) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget.py: latest_versionstring now returns (version, revision)Aníbal Limón2015-07-221-1/+2
| | | | | | | | | | | | | | | | Now latest_versionstring method returns (version, revision) for comply the new return convention needed by SCM's like git get the current revision. bb/tests/fetch.py: Updated wget latest_versionstring test for comply new convention. [YOCTO #7605] (Bitbake rev: 8d454646cbe1b04758ca178d8c6fcfd02b818b7b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git.py: latest_versionstring now returns (version, revision)Aníbal Limón2015-07-221-1/+2
| | | | | | | | | | | | | | | | We need to know the revision associated with the upstream version in SCM like git, this change broke return convention, oe-core recipeutils get_recipe_upstream_version need to be updated. tests/fetch.py: Updated git latest_versionstring test for comply new convention. [YOCTO #7605] (Bitbake rev: fd175dc90024c503134c11cbd83e77d88c406ac8) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: FetchCheckStatusTest add cases for ftp and https.Aníbal Limón2015-07-121-1/+7
| | | | | | | (Bitbake rev: 36f2577d075f87090766877473f9030e44a941a2) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: Add FetchCheckStatusTest tests.Aníbal Limón2015-07-121-0/+39
| | | | | | | | | | | | Add tests for checkstatus method using http with/without connection cache. [YOCTO #7796] (Bitbake rev: b1f2d3edefb7dd274174eb983666213b0f49c994) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Add mirror of mirror testsRichard Purdie2015-05-161-0/+25
| | | | | | | | Add some tests of mirrors or mirrors to the fetcher unittests. (Bitbake rev: e33d82bc10283d533f928836d56a6f0af80ea5c1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: ensure fetch tests preserve current dirPaul Eggleton2015-05-151-0/+2
| | | | | | | | | | | | | | | The fetcher calls os.chdir() in a number of places, which can affect other tests (since the directory it changes into gets deleted) - let's just put the current directory back to where it was when we're done. (This fixes bb.tests.Path.test_unsafe_delete_path failing if it was run as part of a full bitbake-selftest run, where the fetcher tests get to run before it.) (Bitbake rev: b1653855c74f86909c9f329ed6d2b10391c28395) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Add BB_ALLOWED_NETWORKS supportLiam R. Howlett2015-05-121-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | BB_ALLOWED_NETWORKS is a list of hosts that the fetcher will be allowed to use when BB_NO_NETWORK is not set. If BB_NO_NETWORK is set, then networking is still disabled. If BB_ALLOWED_NETWORKS is not set, the behaviour remains the same as today. If BB_NO_NETWORK is NOT set, and BB_ALLOWED_NETWORKS is configured, then only the hosts in the list are usable by the fetcher. eg: BB_ALLOWED_NETWORKS="yoctoproject.org git.gnu.org" The fetcher will be able to download from yoctoproject.org, git.gnu.org, but not ftp.gnu.org or any other hostname that is not in the list. There is also limited support for wildcards on the beginning of the hosts, so BB_ALLOWED_NETWORKS="*.gnu.org" with match git.gnu.org and ftp.gnu.org as well as foo.git.gnu.org (Bitbake rev: c7263096ba31ba45daeeb9de90c1cb9ebef24a28) Signed-off-by: Liam R. Howlett <Liam.Howlett@WindRiver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: latest_versionstring add set of PNAníbal Limón2015-02-161-0/+2
| | | | | | | | | | Add set of PN in data because now latest_versionstring use it for validate version directory searching. (Bitbake rev: 2e4a03db967ac1459b2764108fc54c4566a7e371) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Add versionstring tests to network tests list, fix print()Richard Purdie2014-12-251-20/+20
| | | | | | | | | | | | The versionstring tests hit the network so should only run when network tests are enabled. Also remove the print statement which confuses the test output and add it to the test failure message instead. (Bitbake rev: 6c046660cfc4fb3792a42aeafff91a13f68a2e89) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Update wget latest_versionstring cups caseAníbal Limón2014-11-281-1/+1
| | | | | | | | | | Update test case for cups is needed because match only 2.0.0 versions see VERSION=2\.0\.0 in the previous string. (Bitbake rev: 148aba30155f4de17f6c6fb9b4c37f08a2db202b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/wget: Add latest_versionstring methodAníbal Limón2014-11-121-0/+35
| | | | | | | | | | | | | | | | | | | | | | | Being able to query whether updated versions of a url are available is useful, not least for the package reporting system. Since such code is closely linked to the url type and the url itself, the fetcher makes a locical place to contain this code. For wget based urls this means taking upstream directory listings and searching those for later versions, returning those that are found. The patch also adds unittests for this function so that if improvements are made, the original test urls can be used to evaulate the those changes. This is based on code from Irina Patru <irina.patru@intel.com>. (Bitbake rev: a8272e22b7819e0e8afd8e291d276f5f28fc0007) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Add latest_versionstring methodAníbal Limón2014-11-121-1/+43
| | | | | | | | | | | | | | | | | | | | | Being able to generate a version string representing the most recent git commit given git is useful, not least for the package reporting system. This adds in a latest_versionstring method to the git fetcher which allows users to query the latest version using ls-remote and filtering the responses. The patch also adds unittests for this function so that if improvements are made, the original test urls can be used to evaulate the those changes. This is based on code from Irina Patru <irina.patru@intel.com>. (Bitbake rev: f71c8c0354e87fed80bc845db6728e6e18ce9c4d) Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: Extend testing of subdir unpack parameter and fixRichard Purdie2014-09-291-0/+7
| | | | | | | | | This fixes urls of the form file://some/path/file;subdir=b. It also adds in a couple of tests so we now tests these corner cases. (Bitbake rev: 46306912a96444790efa9418d934dfdd36773ba1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-selftest: add tests for local fetchingPaul Eggleton2014-04-091-0/+55
| | | | | | | | | | | | Add some explicit tests for unpacking local files to the appropriate location. Some of these tests are actually testing for broken behaviour; these have been called out in the comments, and associated bugs have been filed. (Bitbake rev: ca921c773c52392a5a338b2f493ad38c8132f708) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2.URI: Set username/password should not change the otherOlof Johansson2014-03-241-0/+14
| | | | | | | | | | | When setting the username after already having set the password, the password was unexpectedly reset. This change fixes this issue and introduces unit tests to make sure it doesn't happen again. (Bitbake rev: 25faef3a047f9c7564089463d7c96f6910b640cb) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2.URI: Support URIs with both query strings and paramsOlof Johansson2014-03-241-0/+18
| | | | | | | | | | | | | | | | | There is a case in meta-intel where a SRC_URI contains both a query string and URI parameter: https://edc.intel.com/Download.aspx?id=6190;downloadfilename=LIN_IEMGD_1_14_GOLD_2443.tgz Python's urlparse thought the URI parameters were part of the query parameter value, but in the bitbake context this is obviously not the case. As bitbake's usage isn't really RFC compliant, we have to extract and remove the URI parameters *before* urlparse sees the URI. (Bitbake rev: c2f27ae4271985b48f957747b6ea372c8699cb49) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2.URI: add support for query parametersOlof Johansson2014-03-241-0/+30
| | | | | | | | | | | | | This change introduces the .query property of the URI class. It is a read/write dict of the parameters supplied in the query string of the URI. E.g.: http://example.com/?foo=bar => .query = {'foo': 'bar'} (Bitbake rev: 1cb2b3c458c8c5521591d2c8f2e0058143fc77bb) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests.fetch: Remove debug assertOlof Johansson2014-03-241-2/+0
| | | | | | | (Bitbake rev: f112660bca0ed8be061055b1e388deeb2d1980a7) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests: add test for gitsm fetcherPaul Eggleton2014-02-281-0/+7
| | | | | | | | | | Use a newly created "git-submodule-test" repo on git.yoctoproject.org which currently contains one submodule (the bitbake repository). (Bitbake rev: a750c57242928c546a5aace632543e956ee908eb) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Sanity check SRCREV matches rev/tag parameterRichard Purdie2014-01-211-0/+15
| | | | | | | | | | | | | | | Add a sanity check so that if some SRCREV is set and a rev parameter is given to the url, the revision given should match. Any tag parameter behaves the same as rev. If both are specified, error to tell the user we're confused rather than do something which may or may not be what they intended. Also add some unittests for this. (Bitbake rev: e82a4ab48991035866da9914c8b75a9bfbc9a7fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-04-27 15:54:36 +0000