linux/poky.git/bitbake, branch hardknott-3.3.4

bitbake: fetch/git: Handle github dropping git:// support

2021-11-03T11:31:26+00:00

github is dropping support for git protocol in Git urls. Add code to remap this to https in a way that could be used in older bitbake versions. (Bitbake rev: 0fe1a9e2d2e33f80d807cefc7a23df4a5f760c74) Signed-off-by: Richard Purdie

bitbake: fetch2: clarify the command-no-found error message

2021-10-25T19:54:25+00:00

If runfetchcmd() fails with bb.process.NotFoundError, the message output is simply "Fetch command" which doesn't really explain what the problem is. Add "not found" to clarify what happened. (Bitbake rev: 7f3fd1f47f7972756b9b39b6c5ab25084fb4212c) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie (cherry picked from commit 8de9dc02ed6a73b47f2ab10be30d1aed7954bc72) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: build: Make exception printing clearer

2021-10-25T19:54:25+00:00

Shows: ERROR: SystemExit(1) instead of: ERROR: 1 (Bitbake rev: 16bb44e00fcad8daefd3b79c5564d9c8a0d03a6c) Signed-off-by: Richard Purdie (cherry picked from commit df89e37c33e4398a5f8ece9a8b973be3fe2ff361) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Handle pseudo shutdown in Ctrl+C case

2021-10-25T19:54:25+00:00

If the build is interrupted, handle the shutdown of pseudo even in this case to avoid data corruption inside docker containers. [YOCTO #14555] (Bitbake rev: 10d61b276d4ffc65640a862c7e5b5852a5716813) Signed-off-by: Richard Purdie (cherry picked from commit a2a04c6fe94bc56efcff299c669a151746e35916) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: build: Ensure python stdout/stderr is logged correctly

2021-10-25T19:54:25+00:00

Currently we see things like: Log data follows: | DEBUG: Executing python function do_pythontest_exit | DEBUG: Python function do_pythontest_exit finished | ERROR: 1 | This is python stdout Whilst after the change we see things like: Log data follows: | DEBUG: Executing python function do_pythontest_exit | This is python stdout | DEBUG: Python function do_pythontest_exit finished | ERROR: 1 since the output is now correctly mixed with the log messages. In some cases the logging tests indicate the output is being lost entirely which is bad for debugging and makes things rather confusing. (Bitbake rev: 3fbc46735ee3679ef1b7466810e4345d78ed05ea) Signed-off-by: Richard Purdie (cherry picked from commit 8966b43761500e0505333d8c9a3f0f2c3dbe7559) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Allow shutdown/database flush of pseudo server at task exit

2021-10-25T19:54:25+00:00

We have a problem where pseudo server processes exist after bitbake exits and hold the pseudo database in memory. In a docker container, the processes will be killed as the container is destroyed with no warning and no opportunity to write the data to disk. This leads to permissions/inode corruptions and data loss. Send a shutdown message to pseudo which in new versions of pseudo will flush the database, thereby fixing some of the issues people using docker containers see. (Bitbake rev: 7752c2c00245e91aeb17e22de484535190b18e5d) Signed-off-by: Richard Purdie (cherry picked from commit a07a971b40acd3eee12e203d2cfa3e49f56109f6) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: npmsw: Avoid race condition with multiple npm fetchers

2021-10-25T19:54:25+00:00

If multiple npmsw fetchers are trying to download the same npm file, one of them can try to download the file while other is calling verify. npmsw methods gets called without holding the lock, which causes race conditions in fetching and verification etc. Lock the lockfile before calling proxy fetcher methods. (Bitbake rev: 591a05d6126f29b501b9fa284c0618de8c903c69) Signed-off-by: Caner Altinbasak Signed-off-by: Richard Purdie (cherry picked from commit fa39e6689d0f0fff772e1c81682698f4b1587b8a) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: tests/runqueue: Ensure hashserv exits before deleting files

2021-10-25T19:54:25+00:00

We've seen races where the socket may be gone but the server is still writing out it's database. Handle that case too to avoid cleanup tracebacks. [YOCTO #14440] (Bitbake rev: ca25bc22c87b340cf5c430bd4a025586e0e9e30b) Signed-off-by: Richard Purdie (cherry picked from commit b9e4fb843cb9d3a4d4404af093a781fab5520465) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: fetch2/git: Use os.rename instead of mv

2021-10-25T19:54:25+00:00

os.rename will overwrite the destination file if present so we can use this instead of the process call overhead. (Bitbake rev: 7be6d18cd74291371f5327dcab2412f508c70189) Signed-off-by: Richard Purdie (cherry picked from commit b3cccaa6a896c41d8c9be5eebc327f726542d16b) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

bitbake: fetch2/git: Avoid races over mirror tarball creation

2021-10-25T19:54:25+00:00

There is a potential race over the mirror tarballs where a partial git repo could be extracted causing fetcher failures if the tarball is being rewritten whilst another build accesses it. Create the mirror tarball atomically to avoid this. [YOCTO #14441] (Bitbake rev: 7c8f344b81b8f8936214f87f695e24dc4e546659) Signed-off-by: Richard Purdie (cherry picked from commit 3250bc950c56bd7dd2114df26e5a8e13b04ceac8) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie