linux/poky.git/meta/classes/externalsrc.bbclass, branch hardknott-next

externalsrc: Detect code changes in submodules

2021-04-30T13:37:25+00:00

Further to 50ff9afb39, only detect code changes in submodules that are subdirectories of the EXTERNALSRC directory. The (undocumented) git submodule--helper returns a path for each submodule relative to the top of the repo. Don't add submodules that are not within our source subtree. [YOCTO #14333] (From OE-Core rev: d233735891872b73e66cb3ce9f73b9af4d32a186) Signed-off-by: Douglas Royds Signed-off-by: Richard Purdie (cherry picked from commit 1c18225d3ef94a41fc073ae87c163b68e6d46571) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

Revert "externalsrc: Detect code changes in submodules"

2021-04-30T13:37:25+00:00

This reverts commit 4525310d49d115a37705f04ac5c03d639e5e8f8c. Further to 50ff9afb39, only detect code changes in submodules that are subdirectories of the EXTERNALSRC directory. The (undocumented) git submodule--helper returns a path for each submodule relative to the top of the repo. Don't add submodules that are not within our EXTERNALSRC subtree. If we unpack one git repo inside another, like this: SRC_URI = "git://${GIT_SERVER}/repo1;name=repo1;destsuffix=repo1 \ git://${GIT_SERVER}/repo2;name=repo2;destsuffix=repo1/repo2 \ " Git status reports, for repo1: Untracked files: (use "git add ..." to include in what will be committed) repo2/ If we run `devtool modify` on this recipe, do_patch runs with: PATCHTOOL = "git" PATCH_COMMIT_FUNCTIONS = "1" The `patch_task_postfunc` (patch.bbclass, line 82) runs a `git add .` on the top-level repo1, leaving the checkout in an invalid state. The following git warning does not appear in the log: $ git add . warning: adding embedded git repository: repo2 hint: You've added another git repository inside your current repository. hint: Clones of the outer repository will not contain the contents of hint: the embedded repository and will not know how to obtain it. hint: If you meant to add a submodule, use: hint: hint: git submodule add repo2 hint: hint: If you added this path by mistake, you can remove it from the hint: index with: hint: hint: git rm --cached repo2 hint: hint: See "git help submodule" for more information. $ git submodule status fatal: no submodule mapping found in .gitmodules for path 'repo2' No further git submodule commands can be run on the checkout. We could enhance the `patch_task_postfunc` to look for any embedded git checkouts and add them as submodules, but this seems unnecessary complexity for an obscure edge-case. Although the git repo is left in an invalid state with respect to the submodules, it still serves the purpose required by devtool: To take further commits, and generate patch files from them. We are still able to run these commands to examine any submodules, where git submodule--helper reports paths relative to the top of the checkout: $ git ls-files --stage | grep ^160000 160000 5feee12d6e974dd8c0614cf5b593380b046439a5 0 repo2 $ git submodule--helper list 160000 5feee12d6e974dd8c0614cf5b593380b046439a5 0 repo2 When a recipe sets EXTERNALSRC to a subdirectory of the git checkout, we test for the existence of the reported submodule paths within the EXTERNALSRC directory. The latest versions of git submodule--helper accept a path to a subdirectory and correctly report no submodules within that subdirectory. Regrettably, we still support git versions that don't accept a path to a subdirectory. [YOCTO #14333] (From OE-Core rev: 4d961d6b794b389f8a2d062d5e7c0ae1ddc49e36) Signed-off-by: Douglas Royds Signed-off-by: Richard Purdie (cherry picked from commit 2055718fdd19f925e236d67823017323bbd92a4b) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

externalsrc: Detect code changes in submodules

2021-03-20T18:54:56+00:00

Further to 50ff9afb39, only detect code changes in submodules that are subdirectories of the EXTERNALSRC directory. git submodule status returns a path relative to the cwd for each submodule. We don't add submodules that are not within our source subtree. (From OE-Core rev: 4525310d49d115a37705f04ac5c03d639e5e8f8c) Signed-off-by: Douglas Royds Signed-off-by: Richard Purdie

externalsrc: Pass through npmsw URIs in SRC_URI

2021-02-26T15:21:21+00:00

NPM shrinkwrap files need to stay in SRC_URI even when using externalsrc so that npm_do_fetch can run to fetch the required dependencies. (From OE-Core rev: e889cbecf9a1731b199e57d79a14e574398586bb) Signed-off-by: Mike Crowe Signed-off-by: Richard Purdie

externalsrc: Detect code changes in submodules

2021-01-28T23:12:05+00:00

The srctree_hash was calculated only from main source directory ignoring changes in submodules. [YOCTO #13748] Use submodule--helper to determine all submodules, and calculate hash from all git tree objects names combined. (From OE-Core rev: 50ff9afb3990bcf60b4fa1f937506cb84028c32d) Signed-off-by: Tomasz Dziendzielski Signed-off-by: Richard Purdie

externalsrc: Fix parsing error with devtool non-git sources

2021-01-23T17:08:53+00:00

If srcdir is under poky directory (e.g. devtool poky/build/workspace/sources) and is not a git repository then ${@srctree_hash_files(d)} will run "git rev-parse --git-dir" and detect poky directory as git-dir and run "'git', 'add', '-A', '.'], cwd=s_dir" trying to add srcdir but build dir is in .gitignore and latest git will fail with "The following paths are ignored by one of your .gitignore files: build" which will end with "ExpansionError during parsing". In this commit I added a check if git_dir is the same as git-dir from TOPDIR (which will detect poky directory) and if yes, then treat srcdir as non-git sources. (From OE-Core rev: 95fbac8dcad6c93f4c9737e9fe13e92ab6befa09) Signed-off-by: Tomasz Dziendzielski Signed-off-by: Richard Purdie

externalsrc: No single-task lock if S != B

2020-10-17T11:34:29+00:00

Allow different recipes to build things from the one external source tree in parallel, but only if the build is happening outside the source tree. (From OE-Core rev: 8c6b12683c3160a5b7c62d1fe00a9f848e062df0) Signed-off-by: Douglas Royds Signed-off-by: Richard Purdie

externalsrc: always set SRCPV

2019-12-28T23:25:42+00:00

If devupstream is in use by the recipe, then that variant has SRCPV, but the main variant may not, in which case bitbake throws an expansion error. (From OE-Core rev: 740e3f5e1d4a8abb07c7a4e4aedd804d679c7562) Signed-off-by: Alexander Kanavin Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

externalsrc: stop rebuilds of 2+ externalsrc recipes sharing the same git repo

2019-09-19T19:30:35+00:00

externalsrc do_configure task watches oe-devtool-tree-sha1 file and its checksum. That file basically contains the result of `git add -A ${EXTERNALSRC} && git write-tree` which is the hash of temporary "commit" of the non committed changes. This file is stored in the .git directory of the git repo of the externalsrc recipe. do_configure then depends on the checksum of oe-devtool-tree-sha1 file. If 2+ recipes with different externalsrc paths but same git repo (e.g. one recipe at /some/path and the other at /some/path/subdir) are parsed, this oe-devtool-tree-sha1 will be overwritten by those recipes at parsing time since .git is shared between those recipes. If there is one non committed git change in /some/path but not in /some/path/subdir, the oe-devtool-tree-sha1 of both recipes will be different. What will happen is that recipe1 will watch over the oe-devtool-tree-sha1 with a specific checksum, fill in file-checksums for do_configure correctly, then recipe2 will watch over the identically named file with different content also fill in the file-checksums varflag. When do_configure of recipe1 will be evaluated for re-execution, oe-devtool-tree-sha1 will be of the value of what is watched over by recipe2, thus triggering a rebuild of recipe1. This behavior is not always reproducible which I'm guessing is due to a small window between recipe1 putting info into oe-devtool-tree-sha1 and calculating the checksum of that file and recipe2 putting its content into oe-devtool-tree-sha1. By appending the name of the recipe to oe-devtool-tree-sha1, we make sure that a recipe won't have its oe-devtool-tree-sha1 overwritten by another recipe sharing the same externalsrc git repo. (From OE-Core rev: 1b727dd7295a7a7fe17800f8038242efbf7fe2b7) Signed-off-by: Quentin Schulz Signed-off-by: Richard Purdie

externalsrc.bbclass: Set BB_DONT_CACHE for non-target recipes

2018-10-08T13:13:54+00:00

BB_DONT_CACHE was not set for non-virtual recipes where PN != BPN, such as quilt-native. Recipes that do not set BBCLASSEXTEND should always have BB_DONT_CACHE set by externalsrc. (From OE-Core rev: 4eff427a0ee629a1541a420a9591411648569a97) Signed-off-by: Ola x Nilsson Signed-off-by: Richard Purdie