linux/poky.git/bitbake/lib, branch thud

bitbake: fetch/git: Handle github dropping git:// support

2021-11-03T11:31:58+00:00

github is dropping support for git protocol in Git urls. Add code to remap this to https in a way that could be used in older bitbake versions. (Bitbake rev: 964958b8b11dc69fb289fc6c97c1dbc8d76ad0f8) Signed-off-by: Richard Purdie

bitbake: fetch2: Fix os.errno references

2021-01-31T11:28:39+00:00

os.errno used to happen to work but is invalid. Correct to use errno. [YOCTO #13068] (Bitbake rev: b3fc65289d33274cd5dace4d4ffe55be11c991f4) Signed-off-by: Richard Purdie

bitbake: tests/fetch: Allow wget upgrade tests to run against a local server

2020-09-09T08:44:53+00:00

Currently these tests rely upon multiple uptream webservers which may change or be unavailable. Add local copies of the test data, copy the httpserver from OE-Core (used for testing there) and run these tests against a local server instead. (Bitbake rev: a21671e8a483ba8a6986d961987eda2d36ec61ca) Signed-off-by: Richard Purdie

bitbake: fetch2: Change git fetcher not to destroy old references

2020-09-08T13:46:13+00:00

It looks like we're about to see a lot of changes in branch names in repos. If we have the prune option here, those old names are lost, the changes propagate to our source mirrors and our old releases break. We have the force option so any replaced references should be replaced, its only orphaned branches which will now be preserved. I believe this behaviour will cause us fewer problems given the changes that look likely to happen. (Bitbake rev: 12d8cc3fecd550c4aadf0519e80711d755ee75ba) Signed-off-by: Richard Purdie

bitbake: tests/fetch: Switch from git.infradead.org to a YP mirror

2020-07-10T20:10:54+00:00

Upstream is unavailable, breaking tests. Switch to a YP mirror since if we can't reach that there are bigger problems. This should remove a source of intermittent failures on the autobuilder. (Bitbake rev: f4e60b29df88393302957c5bbdbe24ca38c4633c) Signed-off-by: Richard Purdie

bitbake: runqueue: fix multiconfig task dependency filtering

2019-12-12T20:02:51+00:00

multiconfig dependencies should be excluded from BB_TASKDEPDATA. However in thud, multiconfig filtering on task dependencies doesn't happen until after deps has already been added to taskdepdata. One manifestation of this results in multiconfig dependencies leaking into staging processing. File: 'exec_python_func() autogenerated', lineno: 2, function: 0001: *** 0002:extend_recipe_sysroot(d) 0003: File: '/home/user/thud/meta/classes/staging.bbclass', lineno: 344, function: extend_recipe_sysroot 0340: #bb.note(" start is %s" % str(start)) 0341: 0342: # Direct dependencies should be present and can be depended upon 0343: for dep in set(start): *** 0344: if setscenedeps[dep][1] == "do_populate_sysroot": 0345: if dep not in configuredeps: 0346: configuredeps.append(dep) 0347: bb.note("Direct dependencies are %s" % str(configuredeps)) 0348: #bb.note(" or %s" % str(start)) Exception: KeyError: 'multiconfig:musl:/home/user/thud/meta/recipes-kernel/linux/linux-yocto_4.18.bb:do_deploy' This can be reproduced on thud by backporting the multiconfig.MultiConfig.test_multiconfig test and mcextend bbclass from warrior. d22b6e03a5504145abed7c2ca44cf12854df85da mcextend: Add helper class useful for multiconfig d9018a3d9c828551c465b68b27920ec4681524ae selftest: Add multiconfig test Flipping the ordering to match warrior's behavior fixes the test case. (Bitbake rev: b690030efc87850951e8e3ecf4ae3c1dd1dc9b63) Signed-off-by: Kyle Russell Signed-off-by: Richard Purdie

bitbake: fetch2: Ensure cached url data is matched to a datastore

2019-11-13T22:00:33+00:00

There was a weird error in OE-Core where "devtool modify virtual/kernel" was showing basehash mismatch errors. This was due to SRCPV sometimes being: AUTOINC+b867b78b50_47b80ef7bd and sometimes AUTOINC+b867b78b50_255a750d28. The latter hash comes from KBRANCH and meant sometimes the correct branch was seen, sometimes it was not. The issue was complicated by the execution using a remote datastore over tinfoil. The problem turns out to be a fetcher caching error. If the datastore changes, the cached url data may not be valid. We therefore ensure we match cached url data against the datastore that generated it, which appears to fix this issue. (Bitbake rev: 97067634b1f149b56844b10e3a5e8d0d980b6e34) Signed-off-by: Richard Purdie

bitbake: cookerdata: Add mc conffiles hashes to cache hash

2019-10-10T15:53:26+00:00

The variable values that result from parsing multiconfig should be included in the cooker data hash, otherwise changes to these files won't be detected, which will allow the parsing cache to be loaded with the old values for the multiconfigs. This can either manifest as the variable values simply not updating, or getting basehash changed errors when building. This bug was previously undetected because all of the multiconfig base files were a direct file dependency in all parsed recipes. This was fixed in 34137a00f60 ("bitbake: bitbake: cooker: Rename __depends in all multiconfigs"), exposing this bug. [YOCTO #13541] (Bitbake rev: 6b045e074c6fea97d4e305a5a3c8bf82135d95eb) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: tests/fetch: Resolve fetch error in bitbake-selftest

2019-09-09T08:58:09+00:00

FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0]) AssertionError: '' is not true : Could not find upstream version for db [YOCTO #13496] The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget. Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the recipe. Also change the version being checked. (Bitbake rev: 8a58c3c64240c6ab14858d18e6b89febdb315311) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: fetch2: show warning when renaming the archive with bad checksum failed

2019-09-06T10:58:56+00:00

* noticed on read-only sshfs premirror * it was showing the warning about renaming the file: WARNING: laser-geometry-1.6.4-r0 do_fetch: Renaming /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz to /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441 and then failed because of movefile() issue with python3 (fixed in previous commit): ERROR: laser-geometry-1.6.4-r0 do_fetch: Error executing a python function in exec_python_func() autogenerated: with movefile() fixed, it let do_fetch continue and re-fetch locally with the right checksum, but still the renamed file didn't exist, because of movefile failure - add another warning when the movefile fails - for whatever reason - unfortunately movefile prints error messages with just print() so the real error is hidden only in log.do_fetch in this case: movefile: Failed to move /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz to /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441 [Errno 30] Read-only file system: '/jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz' -> '/jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441' (Bitbake rev: d36438759344caa447d9a0bf30749a0aa31d1fba) Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie