linux/poky.git/scripts/lib/devtool/init.py, branch 5.1_M3

devtool: Make use of oe.patch.GitApplyTree.commitIgnored()

2024-02-19T16:03:22+00:00

This makes use of the oe.patch.GitApplyTree.commitIgnored() function to create commits that shall be ignored by `devtool finish`. (From OE-Core rev: 4e1b9289450b5e7946bd5999c95a3ee214aab8a5) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie

devtool: refactor exec_fakeroot

2024-02-02T10:38:28+00:00

Provide a function exec_fakeroot_no_d which does the same like exec_fakeroot does, but is usable independenlty from bitbake. This allows to use the fanction from scripts where the d variable is not available. (From OE-Core rev: 6f5980d96844559639fc3660bb975b0bdd1aa523) Signed-off-by: Adrian Freihofer Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

devtool: add support for git submodules

2023-12-01T11:48:25+00:00

Adding the support of submodules required a lot of changes on the internal data structures: * initial_rev/startcommit used as a starting point for looking at new / updated commits was replaced by a dictionary where the keys are the submodule name ("." for main repo) and the values are the initial_rev/startcommit * the extractPatches function now extracts patch for the main repo and for all submodules and stores them in a hierarchical way describing the submodule path * store initial_rev/commit also for all submodules inside the recipe bbappend file * _export_patches now returns dictionaries that contains the 'patchdir' parameter (if any). This parameter is used to add the correct 'patchdir=' parameter on the recipe Also, recipe can extract a secondary git tree inside the workdir. By default, at the end of the do_patch function, there is a hook in devtool that commits everything that was modified to have a clean repository. It uses the command: "git add .; git commit ..." The issue here is that, it adds the secondary git tree as a submodule but in a wrong way. Doing "git add " declares a submodule but do not adds a url associated to it, and all following "git submodule foreach" commands will fail. So detect that a git tree was extracted inside S and correctly add it using "git submodule add ", so that it will be considered as a regular git submodule (From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935) Signed-off-by: Julien Stephan Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

devtool: tag all submodules

2023-12-01T11:48:25+00:00

In the case of a repository with submodules, we need to add the "devtool-base" and "devtool-patched" tag on all submodules in order to properly detect the added/removed/modified patches (From OE-Core rev: 241da68805d177d4ec4b302c8a997645cc645286) Signed-off-by: Julien Stephan Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

devtool: make sure .git/info exists before writing to .git/info/excludes

2020-10-17T11:34:29+00:00

If nothing else is specified, 'git init' uses its default repository template from the install location (e.g. /usr/share/git-core/templates), which already includes an info/ subdirectory. However, when setting init.templateDir to a different template path in ~/.gitconfig, this isn't necessarily the case, and it can lead to setup_git_repo() failing with stack traces like: File: '.../scripts/lib/devtool/__init__.py', lineno: 234, function: setup_git_repo 0230: pass 0231: if 'singletask.lock\n' not in excludes: 0232: excludes.append('singletask.lock\n') 0233: bb.warn("try writing excludefile") *** 0234: with open(excludefile, 'w') as f: 0235: for line in excludes: 0236: f.write(line) 0237: 0238: bb.process.run('git checkout -b %s' % devbranch, cwd=repodir) Exception: FileNotFoundError: [Errno 2] No such file or directory: '.../devtooltmp-6m36b181/workdir/foobar-1.0.1/.git/info/exclude' Fix this edge case by creating the .git/info/ directory first. Fixes: 334ba846c795fc0d8c73 (2018-02-01, "devtool: set up git repos so that singletask.lock is ignored") (From OE-Core rev: 148a23e4d5ceaf655ccacb52deca4ba501f12975) Signed-off-by: Roland Hieber Signed-off-by: Richard Purdie

devtool: use -f and don't use --exclude-standard when adding files to workspace

2020-06-04T12:27:29+00:00

* I see a case where a tarball contains .gitignore and bunch of files which are normally ignored in git, but still included in the tarball (e.g. configure script next to configure.ac) * when devtool is creating a git repo in workspace it won't include these files from tarball in the initial devtool-base commit, because git ls-files won't list them * but then the first .patch file (without git headers) when applied with GitApplyTree._applypatch() will add all these still ignored files to a commit which used to only modify some files, because it's using -f: # Add all files shellcmd = ["git", "add", "-f", "-A", "."] output += runcmd(["sh", "-c", " ".join(shellcmd)], self.dir) at least in this case it would be better to add all ignored files in the initial devtool-base commit and then --force-patch-refresh will just include the small modification as before instead of adding unrelated files, just because they were initially ignored - this behavior will also match with the do_patch task in the actual build where the .gitignore is ignored when unpacking some tarball * my use-case is fixed in setup_git_repo, but similar function is in devtool upgrade, I've changed it there as well (From OE-Core rev: 06a24a615549af3550302a56ea08147000a608f3) Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie

meta/lib+scripts: Convert to SPDX license headers

2019-05-09T15:31:55+00:00

This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie

devtool: prevent starting git gc in background

2019-03-24T17:00:52+00:00

Devtool creates a git repository for extracted sources in a temporary directory and then moves it to a final destination after patching is done. Unfortunately devtool is not aware that some of its git operations may have caused git garbage collector to start in background. If timing is just right a repository move fails because GC is removing git objects while they are being moved. The issue was hit on Krogoth release, but the code that moves created git repository is still the same. Disable gc.autodetach to make GC run synchronously and block git until it is done. Traceback (most recent call last): File "/openembedded-core/scripts/devtool", line 342, in ret = main() File "/openembedded-core/scripts/devtool", line 329, in main ret = args.func(args, config, basepath, workspace) File "/openembedded-core/scripts/lib/devtool/standard.py", line 352, in extract initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, rd) File "/openembedded-core/scripts/lib/devtool/standard.py", line 644, in _extract_source shutil.move(srcsubdir, srctree) File "/usr/lib/python2.7/shutil.py", line 299, in move copytree(src, real_dst, symlinks=True) File "/usr/lib/python2.7/shutil.py", line 208, in copytree raise Error, errors Error: [('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e', '/build/grpc/grpc_src/.git/objects/5e', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e'"), ('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57', '/build/grpc/grpc_src/.git/objects/57', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57'"), many git objects ... ] (From OE-Core rev: 3da578a79195a63fb6d4bbd68d8f1548defd205f) Signed-off-by: Taras Kondratiuk Signed-off-by: Richard Purdie

devtool: set up git repos so that singletask.lock is ignored

2018-02-06T11:06:27+00:00

singletask.lock is written out while certain tasks execute for recipes that have externalsrc.bbclass enabled - this includes recipes in devtool's workspace. It appears that there's a race where singletask.lock will be there one minute and then when we try to get the file checksum of it (since we want to know if anything in the source tree has changed) it will be gone, and git chokes. To fix that, add singletask.lock to .git/info/exclude in the repository, regardless of whether we created the repository or not. In any case singletask.lock should never be tracked by git, so this is a good thing to be doing for that reason as well. This fixes oe-selftest failures in test_devtool_modify that we've seen on the Yocto Project autobuilder: bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception CalledProcessError: Command '['git', 'add', '-A', '.']' returned non-zero exit status 128. Note that this only fixes this issue for devtool; if you are using externalsrc independently of devtool there's a chance this will still be an issue unless you add singletask.lock to your .gitignore. (From OE-Core rev: 334ba846c795fc0d8c73ce05a1b0882739c86650) Signed-off-by: Paul Eggleton Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

devtool: finish: ensure repository is clean before proceeding

2017-11-11T12:14:27+00:00

If the git repository for a recipe in the workspace has uncommitted changes in it then it's possible that the user has forgotten to commit something, so check and exit if there are any. Provide a -f/--force option to continue in the case where the uncommitted changes aren't needed. Separately, if the repository is in the middle of a rebase or git am / apply then error out (without the opportunity to force) since the user really needs to sort this out before finishing. (From OE-Core rev: bfebd18982c0c82ef2da63ec8f22175c93b2e308) Signed-off-by: Paul Eggleton Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

linux/poky.git/scripts/lib/devtool/__init__.py, branch 5.1_M3