linux/poky.git/bitbake/lib/bb/fetch2, branch yocto-2.7

bitbake: gitsm: Add need_update method to determine when we are going to a new SRCREV

2019-04-01T13:48:44+00:00

If the system had previously fetched a source repository for use by gitsm, and then the SRCREV was updated and the new commit already existed, the system would not re-evaluate the submodules and update them accordingly. The cause of this issue was that need_update was being used, unmodified, from the base git fetcher. It did not have any knowledge, nor did it care if we were moving commits and needed to re-evaluate what was happening due to this switch. To fix the issue, during the download process we add all processed (by gitsm) srcrevs to the git config file, as bitbake.srcrev. This allows us to use a new need_update function that not only checks if the git commit is present, but if we have previously processed this commit to ensure all of the submodule components are also present. This approach is used, instead of iterating over the submodules in need_update to avoid a potential race condition that has affected us in the past. The need_update is called only with the parent locking. Any time we need to dive into the submodules, we need to lock, and unlock them, at each stage. This opens the possibility of errors in either the code, or unintended race conditions with rm_work. This issue was discovered by William A. Kennington III . The included test case was also written by him, and included unmodified. (Bitbake rev: 30fe86d22c239afa75168cc5eb262b880886ef8a) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie

bitbake: fetch2: Use bb.utils.to_boolean() for BB_NO_NETWORK

2019-03-26T14:02:28+00:00

Make it consistent with BB_FETCH_PREMIRRORONLY. (Bitbake rev: 4c6013f47ef36b03f590c909d7c9a2f50b698620) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: fetch2: Unify BB_FETCH_PREMIRRORONLY

2019-03-26T14:02:28+00:00

The fetch2/__init__.py checks whether "BB_FETCH_PREMIRRORONLY" == "1", but fetch2/git.py and hg.py checks whether it is None, this makes it discontinuous, and BB_FETCH_PREMIRRORONLY = "0" doens't work as expected in the later case, so unify it to the previous one. (As BB_NO_NETWORK does). And also use bb.utils.to_boolean() to make them consistent. (Bitbake rev: 85a0d22835588e9ad8ec29d88a8115227e88420c) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: fetch2: Print SCMs list when SRCREV_FORMAT is not set

2019-03-24T17:01:34+00:00

This makes it easier to debug, especially when multipe SCMs like gitsm, otherwise we don't know why there are multiple SCMs. (Bitbake rev: 313fe5e86b254eadfdead706be4bd7b274d5e3c0) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: fetch2/git: Fix clean to remove clonedir

2019-03-24T17:01:34+00:00

The localpath is a symlink to clonedir when it is cloned from a mirror, for example: $ bitbake systemtap-native -cfetch $ ls downloads/git2 sourceware.org.git.systemtap.git -> /path/to/downloads/git2/mirror.path.git.sourceware.org.git.systemtap.git mirror.path.git.sourceware.org.git.systemtap.git There are both sourceware.org.git.systemtap.git and mirror.path.git.sourceware.org.git.systemtap.git in DL_DIR/git2, the symlink sourceware.org.git.systemtap.git is created by try_mirror_url(), but do_cleanall" only removed the symlink, didn't remove the real dir mirror.path.git.sourceware.org.git.systemtap.git, this may cause confusions, for example, I assumed that do_cleanall removed everything, but it didn't, and it would the re-used next time when do_fetch. This patch fixes the problem. (Bitbake rev: 452e2200ad2c29dec3753f5f7a8cbc9183ec7dd8) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: fetch2: runfetchcmd(): Print workdir in debug message

2019-03-24T17:01:34+00:00

The workdir is an important message when run git command, for exmample, before the patch: Running git -c core.fsyncobjectfiles=0 branch --contains [snip] We don't know where it is running, now it is: Running 'git -c core.fsyncobjectfiles=0 branch --contains [snip]' in /path/to/git.opensvc.com.multipath-tools..git Which is easier to debug. (Bitbake rev: f25c05b73218d0e8edf2dd63b36bc09e547c5455) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: fetch2: Fix undefined variable issues

2019-03-19T23:54:41+00:00

Various refactors have left hanging variables, often in debug messages, hence why they haven't been spotted. Fix them (pylint spotted the issues). (Bitbake rev: e1f252408982a62485bbf874a01fd30a02aeed32) Signed-off-by: Richard Purdie

bitbake: fetch2/wget: Remove pointless lambda function

2019-03-19T23:54:41+00:00

(Bitbake rev: 329986c3f2d98c4cd1a43b725194003575e718d5) Signed-off-by: Richard Purdie

bitbake: fetch2/wget: Clean up whitespace/comments

2019-03-19T23:54:41+00:00

Clean up the whitespace and comments style, keep pylint happy. (Bitbake rev: ee59fdaae68543ade03cacfdbbf14fdc7e469412) Signed-off-by: Richard Purdie

bitbake: fetch2/wget: Fix typo in proxy handling

2019-03-19T23:54:41+00:00

Fix a typo in a variable which would affect proxy handling spotted by pylint. (Bitbake rev: 19f4ca0119e33df64da629253d5002fcdb385f4f) Signed-off-by: Richard Purdie