summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorMartin Jansa <Martin.Jansa@gmail.com>2022-10-21 01:03:51 +0200
committerRichard Purdie <richard.purdie@linuxfoundation.org>2022-10-26 12:28:39 +0100
commit9ba58ee035fa23957492dc93863f0e163cf66bfe (patch)
tree21b0ee0a36bdad2102c514dd66ba7b8e41f2505d
parent2f7093d9baccf73c16d9b947270ce797784c4f3a (diff)
downloadpoky-9ba58ee035fa23957492dc93863f0e163cf66bfe.tar.gz
externalsrc.bbclass: fix git repo detection
* fix issue introduced in: https://git.openembedded.org/openembedded-core/commit/?id=95fbac8dcad6c93f4c9737e9fe13e92ab6befa09 * it added check for s_dir + git-dir (typically '.git') isn't the same as ${TOPDIR} + git-dir, but due to copy-paste issue it was just comparing it with s_dir + git-dir again, resulting in most external repos (where git-dir is '.git') to be processed as regular directory (not taking advantage of git write-tree). * normally this wouldn't be an issue, but for big repo with a lot of files this added a lot of checksums in: d.setVarFlag('do_compile', 'file-checksums', '${@srctree_hash_files(d)}') and I mean *a lot, e.g. in chromium build it was 380227 paths which still wouldn't that bad, but the checksum processing in siggen.py isn't trivial and just looping through all these checksums takes very long time (over 1000sec on fast NVME drive with warm cache) and then https://git.openembedded.org/bitbake/commit/?id=b4975d2ecf615ac4c240808fbc5a3f879a93846b made the processing a bit more complicated and the loop in get_taskhash() function took 6448sec and to make things worse there was no output from bitbake during that time, so even with -DDD it looks like this: DEBUG: virtual/libgles2 resolved to: mesa (langdale/oe-core/meta/recipes-graphics/mesa/mesa_22.2.0.bb) Bitbake still alive (no events for 600s). Active tasks: Bitbake still alive (no events for 1200s). Active tasks: Bitbake still alive (no events for 1800s). Active tasks: Bitbake still alive (no events for 2400s). Active tasks: Bitbake still alive (no events for 3000s). Active tasks: Bitbake still alive (no events for 3600s). Active tasks: Bitbake still alive (no events for 4200s). Active tasks: Bitbake still alive (no events for 4800s). Active tasks: Bitbake still alive (no events for 5400s). Active tasks: Bitbake still alive (no events for 6000s). Active tasks: DEBUG: Starting bitbake-worker without -DDD it will get stuck for almost 2 hours in: "Initialising tasks..." before it finally writes sstate summary like: "Sstate summary: Wanted 3102 Local 0 Mirrors 0 Missed 3102 Current 1483 (0% match, 32% complete)" * fix the copy&paste typo to use git work-tree in most cases, but be aware that this issue still exists for huge local source trees not in git [YOCTO #14942] (From OE-Core rev: 9102e5a94b8146cb1da27afbe41d3db999a914ff) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-rw-r--r--meta/classes/externalsrc.bbclass4
1 files changed, 2 insertions, 2 deletions
diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index 06a9548a20..5905342589 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -212,8 +212,8 @@ def srctree_hash_files(d, srcdir=None):
212 try: 212 try:
213 git_dir = os.path.join(s_dir, 213 git_dir = os.path.join(s_dir,
214 subprocess.check_output(['git', '-C', s_dir, 'rev-parse', '--git-dir'], stderr=subprocess.DEVNULL).decode("utf-8").rstrip()) 214 subprocess.check_output(['git', '-C', s_dir, 'rev-parse', '--git-dir'], stderr=subprocess.DEVNULL).decode("utf-8").rstrip())
215 top_git_dir = os.path.join(s_dir, subprocess.check_output(['git', '-C', d.getVar("TOPDIR"), 'rev-parse', '--git-dir'], 215 top_git_dir = os.path.join(d.getVar("TOPDIR"),
216 stderr=subprocess.DEVNULL).decode("utf-8").rstrip()) 216 subprocess.check_output(['git', '-C', d.getVar("TOPDIR"), 'rev-parse', '--git-dir'], stderr=subprocess.DEVNULL).decode("utf-8").rstrip())
217 if git_dir == top_git_dir: 217 if git_dir == top_git_dir:
218 git_dir = None 218 git_dir = None
219 except subprocess.CalledProcessError: 219 except subprocess.CalledProcessError: