linux/poky.git/meta/classes/externalsrc.bbclass, branch yocto-4.0.28

externalsrc: Ensure SRCREV is processed before accessing SRC_URI

2023-12-23T02:36:55+00:00

When SRCREV is used, call bb.fetch.get_srcrev() before accessing SRC_URI. Without this new bb.fetch.get_srcrev() call, SRC_URI might be accessed before SRCREV had a chance to be processed. In master, this is fixed by https://git.yoctoproject.org/poky/commit/?id=62afa02d01794376efab75623f42e7e08af08526 However, this commit is not suited for backport since it is quite invasive. The part of the commit that fix the bug is: --- a/meta/classes/externalsrc.bbclass +++ b/meta/classes/externalsrc.bbclass @@ -63,6 +63,7 @@ python () { else: d.setVar('B', '${WORKDIR}/${BPN}-${PV}') + bb.fetch.get_hashvalue(d) local_srcuri = [] fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d) for url in fetch.urls: NB: bb.fetch.get_hashvalue() does not exist in kirkstone but is equivalent to bb.fetch.get_srcrev(). Fixes [YOCTO #14918] (From OE-Core rev: f6563cca6c4bf627e904d81fbe5b0b0f2b16a107) Signed-off-by: Yoann Congal Suggested-by: Chris Wyse Signed-off-by: Steve Sakoman

externalsrc: fix dependency chain issues

2023-09-23T15:26:15+00:00

Instead of deleting setscene tasks, now SSTATE_SKIP_CREATION is set instead. This seems to fix the compile issues where the populate_sysroot task was not run when an externalsrc recipe was built as a dependency. [YOCTO #15164] [RP addition: The deltask was added by me in 2012 when the class was created. The trouble is bitbake assumes 'sstate' tasks have a setscene task and by deleting the setscene task, bitbake stops thinking the task can be accelerated. There is other code in the sysroot code which assumes some tasks are always sstate tasks. We cannot delete the task without changes to the way bitbake learns about 'setscene' tasks so the patch is correct, avoiding creating files is the better approach given the way the world works now. There would be concerns about exisitng sstate reuse however this shouldn't occur since SRC_URI changes and that will change the underlying hashes. Hash equivalency could potentially cause issues by joining hashes together again however if the output matches, that shouldn't in theory cause any issue.] (From OE-Core rev: f6bb8438a18dfa2a520ad6fa65662d908f4ef0ed) Signed-off-by: Peter Suti Signed-off-by: Richard Purdie (cherry picked from commit ee4667a24ccdd8c9d547e73aecf661e6a1283890) Signed-off-by: Steve Sakoman

devtool: fix devtool finish when gitmodules file is empty

2023-02-04T23:32:20+00:00

When a .gitmodules file exists but is empty then devtool finish fails. Add an additional check for this. [YOCTO #14999] (From OE-Core rev: 3c3b16ed1c2e74389d570fba8800ffdec62fdd48) Signed-off-by: Thomas Roos Signed-off-by: Luca Ceresoli Signed-off-by: Richard Purdie (cherry picked from commit b4f0f7c4934bade9e4d4a1086f9d8b29d8e9ad45) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

externalsrc: fix lookup for .gitmodules

2022-12-23T23:05:50+00:00

Commit 0533edac277080e1bd130c14df0cbac61ba01a0c broke bitbake parsing when bitbake is executed from directory with existing .gitmodules and the recipe in externalsrc does not have .gitmodules The check needs to search for .gitmodules in sources path, not cwd. iParsing recipes...ERROR: ExpansionError during parsing ... bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception CalledProcessError: Command '['git', 'config', '--file', '.gitmodules', '--get-regexp', 'path']' returned non-zero exit status 1. (From OE-Core rev: 1f3b39c4d6828940934a341e9f8b73214dc67fdf) Signed-off-by: Peter Marko Signed-off-by: Alexandre Belloni (cherry picked from commit 66ff3d1f65cd2e7f5319e98fa41f47a59b714c72) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

externalsrc: git submodule--helper list unsupported

2022-11-10T14:39:24+00:00

Git has removed support for "git submodule--helper list". https://github.com/git/git/commit/31955475d1c283120d5d84247eb3fd55d9f5fdd9 This change provides an alternate method for gathering the submodules information. Tested: Build recipes with and without submodules (From OE-Core rev: 0750fa73ea9ff08e844480865cc51db0c924ad0b) Signed-off-by: Carson Labrado Signed-off-by: John Edward Broadbent Signed-off-by: Richard Purdie (cherry picked from commit 6d9364e5f3535954f65cbbc694ee7933ac1d664f) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

externalsrc.bbclass: Remove a trailing slash from ${B}

2022-11-10T14:39:24+00:00

The trailing slash in ${B} caused -fdebug-prefix-map=${B}=... to not match as intended, resulting in ${TMPDIR} ending up in files in ${PN}-dbg when externalsrc was in use, which in turn triggered buildpath QA warnings. (From OE-Core rev: e29509d855374995034d89b381e6ad9ffed90c23) Signed-off-by: Peter Kjellerstedt Signed-off-by: Alexandre Belloni (cherry picked from commit 9b5031ed5a0d102905fa75acc418246c23df6eef) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

externalsrc.bbclass: fix git repo detection

2022-11-10T14:39:24+00:00

* fix issue introduced in: https://git.openembedded.org/openembedded-core/commit/?id=95fbac8dcad6c93f4c9737e9fe13e92ab6befa09 * it added check for s_dir + git-dir (typically '.git') isn't the same as ${TOPDIR} + git-dir, but due to copy-paste issue it was just comparing it with s_dir + git-dir again, resulting in most external repos (where git-dir is '.git') to be processed as regular directory (not taking advantage of git write-tree). * normally this wouldn't be an issue, but for big repo with a lot of files this added a lot of checksums in: d.setVarFlag('do_compile', 'file-checksums', '${@srctree_hash_files(d)}') and I mean *a lot, e.g. in chromium build it was 380227 paths which still wouldn't that bad, but the checksum processing in siggen.py isn't trivial and just looping through all these checksums takes very long time (over 1000sec on fast NVME drive with warm cache) and then https://git.openembedded.org/bitbake/commit/?id=b4975d2ecf615ac4c240808fbc5a3f879a93846b made the processing a bit more complicated and the loop in get_taskhash() function took 6448sec and to make things worse there was no output from bitbake during that time, so even with -DDD it looks like this: DEBUG: virtual/libgles2 resolved to: mesa (langdale/oe-core/meta/recipes-graphics/mesa/mesa_22.2.0.bb) Bitbake still alive (no events for 600s). Active tasks: Bitbake still alive (no events for 1200s). Active tasks: Bitbake still alive (no events for 1800s). Active tasks: Bitbake still alive (no events for 2400s). Active tasks: Bitbake still alive (no events for 3000s). Active tasks: Bitbake still alive (no events for 3600s). Active tasks: Bitbake still alive (no events for 4200s). Active tasks: Bitbake still alive (no events for 4800s). Active tasks: Bitbake still alive (no events for 5400s). Active tasks: Bitbake still alive (no events for 6000s). Active tasks: DEBUG: Starting bitbake-worker without -DDD it will get stuck for almost 2 hours in: "Initialising tasks..." before it finally writes sstate summary like: "Sstate summary: Wanted 3102 Local 0 Mirrors 0 Missed 3102 Current 1483 (0% match, 32% complete)" * fix the copy&paste typo to use git work-tree in most cases, but be aware that this issue still exists for huge local source trees not in git [YOCTO #14942] (From OE-Core rev: 1b1b9756c5e50a9b195f774b902881a9b0052f69) Signed-off-by: Martin Jansa Signed-off-by: Alexandre Belloni (cherry picked from commit 9102e5a94b8146cb1da27afbe41d3db999a914ff) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

externalsrc: Don't wipe out src dir when EXPORT_FUNCTIONS is used.

2022-09-16T16:53:23+00:00

When 73fa855f6af5ef9c3 was introduced, the "cleandirs" variable flag started applying to functions exported using EXPORT_FUNCTIONS. The externalsrc class is supposed to remove cleandirs in order to prevent wiping out an external src folder (home directory?), but doesn't take the previous point into account. The result is that cleandirs is still in effect. To fix this, apply the cleandirs manipulation to all variables, not just predefined ones. This is expensive, but since it executes inside an `if externalsrc` clause, and EXTERNALSRC is usually only set for specific single recipes, it won't affect most recipes. Richard Purdie is the original author of this patch. I just submitted it under my name for blame purposes, and also we have been testing it a lot in my company. This is the original discussion: https://lists.openembedded.org/g/openembedded-core/topic/91374926 (From OE-Core rev: 70f7575bfe7e3b136125b1db7ad5549074e7fd1c) Signed-off-by: Kristian Amlie Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie (cherry picked from commit 74ad497a55007960a4869905878e3ccbd11e4369) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

externalsrc.bbclass: support crate fetcher on externalsrc

2022-07-25T14:11:46+00:00

To support crate:// fetcher on externalsrc, we need to make pass-through the URIs in SRC_URI. (From OE-Core rev: 73cc5a1cc62df616c2705280c8de27c38bb2ff0f) Signed-off-by: Chanho Park Signed-off-by: Luca Ceresoli Signed-off-by: Richard Purdie Signed-off-by: Richard Pastrick Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

externalsrc/devtool: Fix to work with fixed export funcition flags handling

2022-04-14T08:46:59+00:00

If we fix bitbake to correctly handle the cleandirs and fakeroot flags for tasks handled by EXPORT_FUNCTIONS, we break devtool since it only considers these for top level functions. Add in extra code to intercept the cleandirs flags for commonly used sub functions too. [YOCTO #8621] (From OE-Core rev: 67fad829f37ed0a8e20c599d2b65635158591d06) Signed-off-by: Richard Purdie