summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/tests
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: tests/fetch: Update upstream master->main branchname transitionRichard Purdie2020-10-261-1/+1
| | | | | | (Bitbake rev: 89fc9450abefd682266efcbfa031d1ef115ff1a7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: tests/fetch: add unit tests for SRC_URI with spaces in urlCharlie Davies2020-10-131-0/+47
| | | | | | | | | | (Bitbake rev: d8daad57bdbceec041c4c2d288ed5d487ad8e6dc) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e7dab75c8d1923abcbbc7c9ac7de215d720ccf26) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Move away from problematic freedesktop.org urlsRichard Purdie2020-09-181-2/+4
| | | | | | | | | | | | | We're either hitting rate limiting with freedesktop.org or the servers have intermittent network connections. Use our own mirror of these repositories instead. (Bitbake rev: 18e1957337fd9f06bc673d28dd4f8277321d07bc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit a1b7ab5c9d5e64969f5ca0e41c0ac13c723e3761) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Drop cups.org from wget status checksRichard Purdie2020-09-171-3/+1
| | | | | | | | | | | | Its becomming clear the upstream server doesn't like this, drop these two urls from the test, not sure we need them here anyway. (Bitbake rev: 16774d4d41f2011e8db7d2986a1a701df123dfc1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ab2ef942dc21f9639793c972f2e546edf9444783) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: test/fetch: change to better svn sourceakuster2020-07-021-1/+1
| | | | | | | | | | | | | | | | | | fixes: svn: warning: W175002: Unexpected HTTP status 504 'Gateway Timeout' on '/openembedded/bitbake/!svn/vcc/default' svn: E205011: Failure occurred processing one or more externals definitions picked pcre2 [Yocto #13948] (Bitbake rev: 767aa9316603a1c92c9c433a0c11ae98089bbc3a) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1483d17108da02f5d615e83403d5fd6288ca957c) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Switch from git.infradead.org to a YP mirrorRichard Purdie2020-06-221-1/+2
| | | | | | | | | | Upstream is unavailable, breaking tests. Switch to a YP mirror since if we can't reach that there are bigger problems. This should remove a source of intermittent failures on the autobuilder. (Bitbake rev: 83296870bede70e31bdf6e73683bcc30681023fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data/siggen: Don't expand ignored variablesRichard Purdie2020-04-061-1/+1
| | | | | | | | | | | | | | | | | | If a variable is in the signature whitelist, we'd currently expand it, then later ignore the data. This is problemtic for code which has effects when expanded, recently source date epoch in OE-Core for example. We don't actually need to do this, if we pass the whitelist into the earlier function it can avoid the expansion. This also also give a small performance boost since we avoid running code in some cases. [YOCTO #13581] (Bitbake rev: f483ee4a869fb1dafbe4bdf2da228cdaa40b38bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: Simplify remote datastore connectionsRichard Purdie2020-03-241-139/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current approach to remote datastores used in tinfoil is breaking. For example, adding a devupstream extension to a recipe with a git upstream, making it the preferred version and then running "devtool modify" on it causes get_srcrev() circular dependency issues. The problem is the override handling in the datastore is broken. This gets broken since remotedata:recieve_datastore() sets d.dict but doesn't update d.overridedata (or d.inchistory or d.varhistory). We could play whack-a-mole but the current implementation seems to be flawed to me. It also doesn't cover, or only partially covers some datastore operations and each needs new dedicated command API. Instead, step back and reimplement the way the datastore connector works. With this change, the datastore is either remote or local but the data is not spread on two sides of the connection. All the API is proxied over the connection by a single function for the datastore (and two to support variable history and include history). This code does not support using the datastore as a parameter to any data store functions. We did have one case of that but its just bad code and can be replaced. The result is something which is much simpler and less invasive to the datastore code itself, meaning its behaviour should be much more consistent. The existing tests for the remote data no longer make any sense and are removed. The one bug this code would have is if key/value pairs are returned over the IPC and those values contained a DataSmart object since we don't recurse into return values to find such things. Nothing appears to do that currently so lets worry about it if its ever an issue. This change should simplfy a ton of other issues and avoid a ton of other bugs so is a huge net gain. Tested with bitbake's and OE's selftests. (Bitbake rev: 85e03a64dd0a4ebe71009ec4bdf4192c04a9786e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests: Add test for gitsm fetcher with shallow mirror tarballsPaul Barker2020-03-111-0/+41
| | | | | | | (Bitbake rev: 2e26e97129d4c54bf86cdea8f9791696a06a36b4) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: add npmsw testsJean-Marie LEMETAYER2020-01-271-0/+251
| | | | | | | | | | | | | | | | | | | This commit adds some tests to validate the npmsw fetcher: - bb.tests.fetch.NPMTest.test_npmsw - bb.tests.fetch.NPMTest.test_npmsw_bad_checksum - bb.tests.fetch.NPMTest.test_npmsw_destsuffix - bb.tests.fetch.NPMTest.test_npmsw_dev - bb.tests.fetch.NPMTest.test_npmsw_mirrors - bb.tests.fetch.NPMTest.test_npmsw_no_network_no_tarball - bb.tests.fetch.NPMTest.test_npmsw_no_network_with_tarball - bb.tests.fetch.NPMTest.test_npmsw_npm_reusability - bb.tests.fetch.NPMTest.test_npmsw_premirrors (Bitbake rev: ba205df20b6a07a4b1125332601c6c54c7b019b5) Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: add npm testsJean-Marie LEMETAYER2020-01-271-0/+183
| | | | | | | | | | | | | | | | | | | | | | | | | This commit adds some tests to validate the npm fetcher: - bb.tests.fetch.NPMTest.test_npm - bb.tests.fetch.NPMTest.test_npm_bad_checksum - bb.tests.fetch.NPMTest.test_npm_destsuffix_downloadfilename - bb.tests.fetch.NPMTest.test_npm_mirrors - bb.tests.fetch.NPMTest.test_npm_no_network_no_tarball - bb.tests.fetch.NPMTest.test_npm_no_network_with_tarball - bb.tests.fetch.NPMTest.test_npm_package_invalid - bb.tests.fetch.NPMTest.test_npm_package_none - bb.tests.fetch.NPMTest.test_npm_premirrors - bb.tests.fetch.NPMTest.test_npm_registry_alternate - bb.tests.fetch.NPMTest.test_npm_registry_invalid - bb.tests.fetch.NPMTest.test_npm_registry_none - bb.tests.fetch.NPMTest.test_npm_version_invalid - bb.tests.fetch.NPMTest.test_npm_version_latest - bb.tests.fetch.NPMTest.test_npm_version_none (Bitbake rev: b166bd3cc6cc1ca63e885319091f17daaaaa2537) Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Allow wget upgrade tests to run against a local server3.1_M2Richard Purdie2020-01-2240-20/+7691
| | | | | | | | | | | Currently these tests rely upon multiple uptream webservers which may change or be unavailable. Add local copies of the test data, copy the httpserver from OE-Core (used for testing there) and run these tests against a local server instead. (Bitbake rev: d5a4a352723258b4d499d3a51f340109c4f36f60) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/utils: add tests for bb.utils.get_referenced_varsChris Laplante via bitbake-devel2020-01-191-0/+44
| | | | | | | (Bitbake rev: 346e74e5d751aadf7881de70b5ab6670dfc463ce) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: remove unused importsFrazer Clews2020-01-194-4/+1
| | | | | | | | | | removed unused imports which made the code harder to read, and slightly but less efficient (Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697) Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/runqueue: Fix to match recent task migration fixesRichard Purdie2019-11-271-1/+1
| | | | | | (Bitbake rev: 35784582fdbb2092eddec094deb6ab9c87666b5e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests: add test for the hashing functionsRoss Burton2019-11-211-0/+26
| | | | | | | | | Add a basic test for bb.utils.md5_file() etc. (Bitbake rev: e944d02fe678f7c6b05c62419f8bceb0709f3037) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: add test for fetching shallow revsChristopher Larson2019-10-281-0/+20
| | | | | | | | | [YOCTO #13586] (Bitbake rev: f7a973604fbfd1059875ff1fabb3f885df9c5b95) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/runqueue: Fix hashserve shutdown raceRichard Purdie2019-10-021-0/+12
| | | | | | | | | | | | | The hashserve can delete its socket whilst the cleanup us happening leading to backtraces and test failures. Add code to avoid this race condition. [YOCTO #13542] (Bitbake rev: efd7b025cee25d0ee668c09476395d08fcf5ae1a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Change task migration behaviour for rerunning setscene tasksRichard Purdie2019-09-301-4/+1
| | | | | | | | | | | | | | | | | Currently runqueue will rerun setscene tasks multiple times as hashes change. This has caused numerous problems since a setscene task may become "unavailable" for some future signature combination and the code then can't easily "unskip" tasks its already passed into the execution queue. At least for now, only run setscene once and assume they're equivalent at that point. In practise that has been much more stable in testing. Tweak the test to match the change in behaviour. (Bitbake rev: 4205a3ef23834f317642bba155d67cd772176fb6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix task migration problemsRichard Purdie2019-09-301-89/+0
| | | | | | | | | | | | | | | | | Tasks were not migrating consistently, particularly: * if a task was rehashed which had already run * if a task which was valid became invalid due to a rehash We need to always run the migration code for rehashed tasks and then reprocess them for hash validity. This means rearranging the code. It also means several tests are no longer correct and can't be written correctly to work on all possible workflows so those are removed. (Bitbake rev: 8443989ee41e9b162972935513e437b5c66ea74d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/data: Test combinations of _append together with overrideJacob Kroon2019-09-271-0/+13
| | | | | | | (Bitbake rev: f31f35e8527c60a95931a4a8311a4cd237770b42) Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: add test case for git-lfs handlingRoss Burton2019-09-271-0/+80
| | | | | | | | | | Add a test case to exercise the detection of git-lfs repositories and the behaviour of the lfs parameter. (Bitbake rev: a7cf4fc72cce357c425084dc2c5f35b5ed1a4b7b) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Rework hash equivalenceJoshua Watt2019-09-181-6/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reworks the hash equivalence server to address performance issues that were encountered with the REST mechanism used previously, particularly during the heavy request load encountered during signature generation. Notable changes are: 1) The server protocol is no longer HTTP based. Instead, it uses a simpler JSON over a streaming protocol link. This protocol has much lower overhead than HTTP since it eliminates the HTTP headers. 2) The hash equivalence server can either bind to a TCP port, or a Unix domain socket. Unix domain sockets are more efficient for local communication, and so are preferred if the user enables hash equivalence only for the local build. The arguments to the 'bitbake-hashserve' command have been updated accordingly. 3) The value to which BB_HASHSERVE should be set to enable a local hash equivalence server is changed to "auto" instead of "localhost:0". The latter didn't make sense when the local server was using a Unix domain socket. 4) Clients are expected to keep a persistent connection to the server instead of creating a new connection each time a request is made for optimal performance. 5) Most of the client logic has been moved to the hashserve module in bitbake. This makes it easier to share the client code. 6) A new bitbake command has been added called 'bitbake-hashclient'. This command can be used to query a hash equivalence server, including fetching the statistics and running a performance stress test. 7) The table indexes in the SQLite database have been updated to optimize hash lookups. This change is backward compatible, as the database will delete the old indexes first if they exist. 8) The server has been reworked to use python async to maximize performance with persistently connected clients. This requires Python 3.5 or later. (Bitbake rev: 2124eec3a5830afe8e07ffb6f2a0df6a417ac973) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Resolve fetch error in bitbake-selftestArmin Kuster2019-09-011-2/+2
| | | | | | | | | | | | | | | | | | | | | FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0]) AssertionError: '' is not true : Could not find upstream version for db [YOCTO #13496] The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget. Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the recipe. Also change the version being checked. (Bitbake rev: 4cf5bb761c561ddea86f2875be35d05abc8486e1) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/runqueue: Fix testsRichard Purdie2019-08-152-5/+5
| | | | | | | | | There were paths being accidentally included in some of the hashserv tests. Remove that and update the hashes so the tests work independently of paths. (Bitbake rev: 6ddb9f09cb60c2354fa6a67cce412c4dc1e7dc2d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix event timing raceRichard Purdie2019-08-141-2/+2
| | | | | | | | | | | | The event from the task notifiing of hash equivalency should only be processed when the task completes. This can otherwise result in a race where a dependent task may run before the original task completes causing various failures. To make this work reliably, the code had to be restructured quite a bit. (Bitbake rev: 1bf5be46f92f125193638cf41ff207d68f592259) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/runqueue: Add further hash equivalence testsRichard Purdie2019-08-144-12/+166
| | | | | | | | | Add some extra hash equivalence runqueue tests based on recent scenarios that caused problems during testing. (Bitbake rev: 373b085ead992a725b2230ededd992b4c61a1a05) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/runqueue: Add hashserv+runqueue testRichard Purdie2019-08-063-3/+40
| | | | | | | | Add a test which tests the runqueue adaptations for hash equivalency. (Bitbake rev: 477321d0780df177c1582db119c2bb6795912fc6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Clean up task reference formatsRichard Purdie2019-08-061-1/+1
| | | | | | | | | | | | | | | | | | Currently siggen uses the format "<filename>.<taskname>" for referencing tasks whilst runqueue uses "<filename>:<taskname>". This converts to use ":" as the separator everywhere. This is an API breaking change since the cache is affected, as are siginfo files and any custom signature handlers such as those in OE-Core. Ultimately this will let us clean up and the accessor functions from runqueue, removing all the ".rsplit(".", 1)[0]" type code currently all over the place. Once a standard is used everwhere we can update the code over time to be more optimal. (Bitbake rev: 07e539e1c566ca3434901e1a00335cb76c69d496) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Clean up BB_HASHCHECK_FUNCTION APIRichard Purdie2019-08-061-12/+10
| | | | | | | | | | | | | This function uses an old API which uses offsets into lists as a communication mechanism. Update the API to use "tid" which is used universally in runqueue now. We can also add kwargs support to the funciton definition to drop some of the backwards compaiblility hoops we had to jump though with different function argument combinations. (Bitbake rev: dc23550047e5078da491ce9a6f30989cb5260df6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix non setscene tasks targets being lostRichard Purdie2019-07-181-0/+9
| | | | | | | | | | | | If you specify both setscene and non-setscene tasks on the commandline, the non-setscene tasks could be missed, e.g. "bitbake X:do_patch X:do_populate_sysroot" and do_patch would fail to run. Fix the problem in runqueue and add a testcase. (Bitbake rev: 75292fdec5d9c0b5b3c554c4b7474a63656f7e12) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/runqueue: Allow common sstate tasks to become validRichard Purdie2019-07-161-0/+6
| | | | | | | | | | | As the logic in bitbake improves, the logic in the tests needs to as well. Afer we built a task for the first time, allow its setscene hash verification status to change, mirroring what would happen in a multiconfig build. (Bitbake rev: 27ec2e69ab3e32972caf8b072b2945736696d83d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Optimise multiconfig with overlapping setsceneRichard Purdie2019-07-165-3/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently if a multiconfig build contains different configurations which have overlapping sstate artefacts, it will build them multiple times. This is clearly suboptimal and not what users want/expect. This adds code to detect this and stall all but one of the setscne tasks so that once its built, it can be found by the other tasks. We take care to iterate the multiconfigs in order so try and avoid dependency loops. We also match on PN+taskname+taskhash since this is what we know sstate in OE-Core would use. There are some tasks even within a multiconfig which match hashes (mostly do_populate_lic tasks) but those have a much higher chance of circular dependency so aren't work attempting to optimise. If a deadlock does occur the build will be slower but there is code to unbreak such a deadlock so it hopefully doens't break anything. Comments are injected into the test tasks so they have different task hashes and a new test for this optimisation is added. (Bitbake rev: a75c5fd6d4ec56836de0be2fe679c81297a080ad) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests: Add initial scenario based test for runqueueRichard Purdie2019-07-159-0/+441
| | | | | | | | | | We need some tests for runqueue, its been something which has been hard to test for a long time. Add some dummy metadata to allow this, mirroring the OE structure in spirit. (Bitbake rev: 37564d7440c5d7aa05ec537f3b79026b1c83bb68) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event/runqueue: Drop StampUpdate event, its pointless/unusedRichard Purdie2019-07-151-8/+0
| | | | | | | | | | Whilst this class has existed for years, it doesn't have any users and has a questionable interface. Drop it to allow for further simplification and changes. (Bitbake rev: 3ab51764f7965d696bb2c5a872bf161473df4289) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: add missing skipIfNoNetwork tags to tests that try ↵Luca Boccassi2019-06-281-0/+5
| | | | | | | | | | | | | to git clone test_git_submodule_dbus_broker, test_git_submodule_CLI11, test_git_submodule_update_CLI11, test_git_submodule_aktualizr and test_git_submodule_iotedge try to access the network via git clone, which fails when there is no network available. Add the relevant skip tag. (Bitbake rev: 9b0538753da0514e6518723dac537007abf7a649) Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/utils.py: add one more test cases for bb.utils.vercmp_stringMartin Jansa2019-06-211-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * this is just another test case for issue already fixed in: commit fef56d28c3efec4876c379898cbc4d4c65303aee Author: Alexander Kanavin <alex.kanavin@gmail.com> Date: Sun Feb 24 21:07:28 2019 +0100 Subject: bitbake: fix version comparison when one of the versions ends in . * The TypeError is triggered not by '.' at the end, but from the extra numberic component in one of the versions. * When one version has fewer elements, it's extended by another (0, None) element where 0 means numeric component. Then the result cannot be decided by comparing the types (oa < ob, ob > oa) and it continues to compare values (ca < cb) which fails when one of them is the None from (0, None) appended before. ====================================================================== ERROR: test_vercmpstring (bb.tests.utils.VerCmpString) ---------------------------------------------------------------------- Traceback (most recent call last): File "lib/bb/tests/utils.py", line 32, in test_vercmpstring result = bb.utils.vercmp_string('1a', '1a1') File "lib/bb/utils.py", line 131, in vercmp_string return vercmp(ta, tb) File "lib/bb/utils.py", line 123, in vercmp r = vercmp_part(va, vb) File "lib/bb/utils.py", line 112, in vercmp_part elif ca < cb: TypeError: '<' not supported between instances of 'NoneType' and 'int' ---------------------------------------------------------------------- Ran 3 tests in 0.002s (Bitbake rev: 9767fffe3115a1f1afa3c6a2b39720fefb8dc4d5) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: svn.py: Stop SVN from directly pulling from an external layer w/o ↵Mark Hatle2019-05-211-0/+80
| | | | | | | | | | | | | | | | | | | | | | | | | | | fetcher Add a new option to the svn fetcher url "externals=allowed". This will allow a user to enable svn co w/ externals. However, this does avoid the fetcher, network access and mirror systems. By default we no longer allow externals in the checkout. This ensures a deterministic download. The system does attempt to identify SVN repos that have externals enabled, and will warn the user. It is up to the user to determine if these are necessary for the recipe. They may disable the warning by adding "externals=nowarn" to the url. In the future we would like to parse this list and see if the items are already in the SRC_URI for that recipe, but with SVN being in limited use these days that extra work is likely not worth the trouble. Add test cases that generated a local SVN tree, with an external source set to github bitbake in svn format. One test case checks that externals are ignored, and one checks that they in downloaded. (Bitbake rev: bf53f07c3647e57d8452a7743a2b04bcb72c80d6) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: gitsm: Fix a bug where the wrong path was used for the submodule initMark Hatle2019-05-161-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Because we are trying to avoid network activity and use our own fetcher, the system emulates the behavior of 'git submodule init'. git submodule init uses the .gitmodules file, where typically the module name and path are the same. However, in this case the module name and path (in the tree) were different. i.e.: [submodule "edgelet/hsm-sys/azure-iot-hsm-c/deps/azure-c-shared-utility"] path = edgelet/hsm-sys/azure-iot-hsm-c/deps/c-shared url = https://github.com/Azure/azure-c-shared-utility.git Previously the code assumed the 'path' was both the checkout location under .git/modules, as well as the path to extract the components. This proved to be incorrect as the .git/modules path needs to match the submodule 'name'. This causes the components that were fetched to be initialized in the wrong location, which later caused the 'git submodule update' process to skip not properly initialized modules. A test case was added for this specific case to ensure a regression does not appear in the future. (Bitbake rev: fd27ab60d33553dba13de39394edaaac04e446b3) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Strip old editor directives from file headersRichard Purdie2019-05-049-18/+0
| | | | | | | | | | There are much better ways to handle this and most editors shouldn't need this in modern times, drop the noise from the files. Its not consitently applied anyway. (Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Drop duplicate license boilerplace textRichard Purdie2019-05-049-117/+0
| | | | | | | | | | With the introduction of SPDX-License-Identifier headers, we don't need a ton of header boilerplate in every file. Simplify the files and rely on the top level for the full licence text. (Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie2019-05-049-0/+18
| | | | | | | | | | | | | | | | | This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Disable warning about dependent tasks for nowRichard Purdie2019-05-011-1/+1
| | | | | | | | | This breaks with rm_work so disable the warning until we find a better solution (and change the test accordingly too). (Bitbake rev: 93e94c06baf013e3d072465a55bddd1fe61c0772) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Ensure warning for invalid task dependencies is usefulRichard Purdie2019-05-011-1/+1
| | | | | | | | | | | | WARNING: elfutils: dependent task do_rm_work does not exist is much less useful than WARNING: elfutils: dependent task do_rm_work for do_deploy does not exist (Bitbake rev: e034c6f75e3d7730ff16a8d1bd0cba03beda0af8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/parse.py: Add testcase for addtask and deltaskRobert Yang2019-04-301-0/+18
| | | | | | | (Bitbake rev: 4ac388646624e08bef848b560fa52deacf2ff4fb) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: gitsm: Add need_update method to determine when we are going to a ↵Mark Hatle2019-04-011-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | new SRCREV If the system had previously fetched a source repository for use by gitsm, and then the SRCREV was updated and the new commit already existed, the system would not re-evaluate the submodules and update them accordingly. The cause of this issue was that need_update was being used, unmodified, from the base git fetcher. It did not have any knowledge, nor did it care if we were moving commits and needed to re-evaluate what was happening due to this switch. To fix the issue, during the download process we add all processed (by gitsm) srcrevs to the git config file, as bitbake.srcrev. This allows us to use a new need_update function that not only checks if the git commit is present, but if we have previously processed this commit to ensure all of the submodule components are also present. This approach is used, instead of iterating over the submodules in need_update to avoid a potential race condition that has affected us in the past. The need_update is called only with the parent locking. Any time we need to dive into the submodules, we need to lock, and unlock them, at each stage. This opens the possibility of errors in either the code, or unintended race conditions with rm_work. This issue was discovered by William A. Kennington III <wak@google.com>. The included test case was also written by him, and included unmodified. (Bitbake rev: 30fe86d22c239afa75168cc5eb262b880886ef8a) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fix version comparison when one of the versions ends in .Alexander Kanavin2019-02-251-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | Previously, this would happen: ====================================================================== ERROR: test_vercmpstring (bb.tests.utils.VerCmpString) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/alexander/development/poky/bitbake/lib/bb/tests/utils.py", line 45, in test_vercmpstring result = bb.utils.vercmp_string('1.', '1.1') File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 143, in vercmp_string return vercmp(ta, tb) File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 135, in vercmp r = vercmp_part(va, vb) File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 124, in vercmp_part elif ca < cb: TypeError: '<' not supported between instances of 'NoneType' and 'int' ---------------------------------------------------------------------- (Bitbake rev: fef56d28c3efec4876c379898cbc4d4c65303aee) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar callsAndré Draszik2019-02-121-4/+4
| | | | | | | | | | | | | | | | | getVar() has been defaulting to expanding by default for a long time (2016), thus remove the True option from getVar() calls with a regex search and replace. Search & replace made using the following command: sed -e 's|\(d\.getVar \?\)( \?\([^,()]*\), \?True)|\1(\2)|g' \ -i $(git grep -E 'getVar ?\( ?([^,()]*), ?True\)' \ | cut -d':' -f1 \ | sort -u) (Bitbake rev: 3bba0dbd524cf72176a765957adff544ae5c255a) Signed-off-by: André Draszik <andre.draszik@jci.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: gitsm: The fetcher did not process some recursive submodules properly.Mark Hatle2019-01-281-0/+20
| | | | | | | | | | | | | | Move the submodule processing outside of the if statement to avoid any optimizations that may happen. Update the test cases to include the additional case, and split the other test cases into individual tests to make it easier to figure out what the failure may be. (Bitbake rev: 0ec98c01ae50f95c9c74acf53013ac59e0e72b08) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: gitsmy.py: Fix unpack of submodules of submodulesMark Hatle2019-01-241-0/+26
| | | | | | | | | | | | | | If the submodule is in a subdirectory, it needs to have that structure preserved. This means the unpack path needs to be in the 'dirname' of the final path -- since the unpack directory name is specified in the URI. Additional specific test cases were added to ensure this is working properly based on two recent error reports. (Bitbake rev: 8c8ecec2a722bc2885e2648d41ac8df07bdf660d) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-13 17:39:10 +0000