linux/poky.git/bitbake, branch yocto-3.1.8

bitbake: providers: selected version not available should be a warning

2021-05-22T11:11:06+00:00

If the selected version if not available, bitbake will happily attempt to build something else. This should be a loud warning not a small note. (Bitbake rev: 078f3164dcb1de7a141bec3a8fd52631d0362631) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie (cherry picked from commit 78cd63285713fde59506eb2e71a7b7ee59a594ff) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: fetch/gitsm: Fix crash when using git LFS and submodules

2021-05-13T12:12:00+00:00

Gitsm fetcher crashes when cloning a repository that contains LFS files. This happens because the unpack method is called during download, but the submodules have not been downloaded yet at this point. This issue was introduced in this commit: 977b7268bf4fd425cb86d4a57500350c9b829162 [YOCTO #14283] (Bitbake rev: b4acaa314cd8bc86c50e14464ff01c2d110122c4) Signed-off-by: Niels Avonds Signed-off-by: Richard Purdie (cherry picked from commit 26caedc4d2e9b5a0f1d57f9291754a7f6c5e437e) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie (cherry picked from commit e05d79a6ed92c9ce17b90fd5fb6186898a7b3bf8) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: runqueue: Fix deferred task issues

2021-05-13T12:12:00+00:00

In a multiconfig situation there are circumstances where firstly, tasks are deferred when they shouldn't be, then later, tasks can end up as both covered and not covered. This patch fixes two related issues. Firstly, the stamp validity checking is done up front in the build and not reevaulated. When rebuilding the deferred task list after scenequeue hash change updates, we need therefore need to check if a task was in notcovered *or* covered when deciding to defer it. This avoids strange logs like: NOTE: Running setscene task X of Y (mc:initrfs_guest:/A/alsa-state.bb:do_deploy_source_date_epoch_setscene) NOTE: Deferring mc:initrfs_guest:/A/alsa-state.bb:do_deploy_source_date_epoch after mc:host:/A/alsa-state.bb:do_deploy_source_date_epoch where tasks have run but are then deferred. Since we're recalculating the whole list, we also need to clear it before iterating to rebuild it. By ensuring covered tasks aren't added to the deferred queue, the covered + notcovered issue should also be avoided. in the task deadlock forcing code. [YOCTO #14342] (Bitbake rev: fa068b5a3430b1b580cacfaf9011cdc3324d5844) Signed-off-by: Richard Purdie (cherry picked from commit 3c8717fb9ee1114dd80fc1ad22ee6c9e312bdac7) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: bitbake: tests/fetch: remove write protected files too

2021-05-13T12:12:00+00:00

For some reason several git-annex files in Debian 10 buster are read-only and removing them with "rm -rf" fails. Fixes test failures like: $ bitbake-selftest ... rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied EE..................................ssss.sssssssssssssss.sssss....................................................................................................... ====================================================================== ERROR: test_shallow_annex (bb.tests.fetch.GitShallowTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1773, in test_shallow_annex fetcher, ud = self.fetch_shallow(uri) File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1541, in fetch_shallow bb.utils.remove(ud.clonedir, recurse=True) File "/home/builder/src/base/poky/bitbake/lib/bb/utils.py", line 700, in remove subprocess.check_call(cmd + ['rm', '-rf'] + glob.glob(path)) File "/usr/lib/python3.7/subprocess.py", line 347, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['rm', '-rf', '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource']' returned non-zero exit status 1. Also, one "chmod" call was failing since the .git/annex subdirectory doesn't exist so just chmod the whole temporary directory which should cover any directory name differences between different git-annex versions. Fixes tests failing after chmod call: Running 'export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; chmod u+w -R /tmp/tmpwmfn4w64/git//.git/annex' in /tmp/tmpwmfn4w64/git/ (Bitbake rev: 1fae1c812138f35c35ea4c0586e21f022524c5f1) Signed-off-by: Mikko Rapeli Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: bitbake: tests/fetch: fix test execution without .gitconfig

2021-05-13T12:12:00+00:00

A CI user validating changes does not have any git push rights or even a .gitconfig file so fix tests so that they run by setting the user.name and user.email for the repo before committing changes. Fixes errors like: ERROR: test_that_unpack_throws_an_error_when_the_git_clone_nor_shallow_tarball_exist (bb.tests.fetch.GitShallowTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 2055, in test_that_unpack_throws_an_error_when_the_git_clone_no r_shallow_tarball_exist self.add_empty_file('a') File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1562, in add_empty_file self.git(['commit', '-m', msg, path], cwd) File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1553, in git return bb.process.run(cmd, cwd=cwd)[0] File "/home/builder/src/base/poky/bitbake/lib/bb/process.py", line 184, in run raise ExecutionError(cmd, pipe.returncode, stdout, stderr) bb.process.ExecutionError: Execution of 'git commit -m a a' failed with exit code 128: *** Please tell me who you are. Run git config --global user.email "you@example.com" git config --global user.name "Your Name" to set your account's default identity. Omit --global to set the identity only in this repository. (Bitbake rev: e341afcce40edf8078661630af1e0a780e8c1910) Signed-off-by: Mikko Rapeli Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: Force parser shutdown after catching an exception

2021-04-06T21:45:36+00:00

Commit bebef58b21bdff7a3ee1fa2449b7df19144f26fd introduced forcing parser shutdown as default in case of build abort. In this case bitbake sometimes hangs after facing error during parsing, waiting for child processes to finish. Killing it then will spawn zombie processes. Thus we force the shutdown after catching an exception. (Bitbake rev: 017a39ed05d065bf28fd38f91bcde8a098300551) Signed-off-by: Jan Brzezanski Signed-off-by: Richard Purdie (cherry picked from commit 915330e1dbae1ee8fd9a0358decf2c294f771961) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: runqueue: Add setscene task overlap sanity check

2021-03-24T14:36:21+00:00

We've seen hard to debug issues where a task ends up in both the covered and notcovered list. Add a sanity check to ensure if this happens in future, we see it in the logs. (Bitbake rev: da306cd4430a6fcf6b62ed0e6e5310432aa4c4c4) Signed-off-by: Richard Purdie (cherry picked from commit 6e001410854792f9bb66a0409a2ac176171b0507) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: runqueue: Fix task execution corruption issue

2021-03-24T14:36:21+00:00

We've seen occasional issues where linux-yocto:do_compile_kernelmodules would run without do_shared_workdir running before it. do_shared_workdir is an setscene task but never has an sstate object generated so it will always rerun. This should not happen since compile_kernemodules should only execute if a setscene that depends on it didn't run and that should trigger do_shared_workdir not to be marked as covered. The issue is that build-appliance-image:do_package is one of the tasks which covers linux-yocto:do_compile_kernelmodules but it is also a noexec task and has a dependecy on pseudo-native:do_populate_sysroot. In the problem case, pseudo-native:do_populate_sysroot is unavailable but marked as covered since it is noexec. The "harddeps" code then also marks it as notcovered. No task should ever be both covered and notcovered and this is where the problems come from. The solution is for the harddeps code only to to fail tasks if they've not already been handled in some way. The code is assuming code couldn't have handled revdeps at this point but we now have clear evidence they can. (Bitbake rev: 8a13955d8da86b976bb136ba1e724a02f7953bca) Signed-off-by: Richard Purdie (cherry picked from commit f66556bbb38449789ceea2fd105e9f68df7fb660) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: init.py: Fix bitbake debug log handling

2021-03-09T00:03:55+00:00

For a while I've been puzzled as to why debug logging from runqueue wouldn't appear on the console with -DD. The logic in the bbdebug handling is inverted so fix it and now we see the expected messages from runqueue with -D and -DD. This should then let us debug other issues using those log messages. (Bitbake rev: e0dd20a006ce6fdc656e6be6ac581e93e2ed4a95) Signed-off-by: Richard Purdie

bitbake: lib/bb/fetch2/init.py: drop _PYTHON_SYSCONFIGDATA_NAME unsetting

2021-02-18T16:48:28+00:00

With introduction of python3targetconfig class in core this is no longer needed. (Bitbake rev: fa94374baea75a94e3a488126ca7d8e241a77acd) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie (cherry picked from commit 47b64cfacd7c498ef9ed5486d117f2d69a39f225) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

linux/poky.git/bitbake, branch yocto-3.1.8

bitbake: providers: selected version not available should be a warning

bitbake: fetch/gitsm: Fix crash when using git LFS and submodules

bitbake: runqueue: Fix deferred task issues

bitbake: bitbake: tests/fetch: remove write protected files too

bitbake: bitbake: tests/fetch: fix test execution without .gitconfig

bitbake: Force parser shutdown after catching an exception

bitbake: runqueue: Add setscene task overlap sanity check

bitbake: runqueue: Fix task execution corruption issue

bitbake: __init__.py: Fix bitbake debug log handling

bitbake: lib/bb/fetch2/__init__.py: drop _PYTHON_SYSCONFIGDATA_NAME unsetting

bitbake: init.py: Fix bitbake debug log handling

bitbake: lib/bb/fetch2/init.py: drop _PYTHON_SYSCONFIGDATA_NAME unsetting