linux/poky.git/scripts/lib/devtool/init.py, branch yocto-4.0.31

devtool: make sure .git/info exists before writing to .git/info/excludes

2020-10-17T11:34:29+00:00

If nothing else is specified, 'git init' uses its default repository template from the install location (e.g. /usr/share/git-core/templates), which already includes an info/ subdirectory. However, when setting init.templateDir to a different template path in ~/.gitconfig, this isn't necessarily the case, and it can lead to setup_git_repo() failing with stack traces like: File: '.../scripts/lib/devtool/__init__.py', lineno: 234, function: setup_git_repo 0230: pass 0231: if 'singletask.lock\n' not in excludes: 0232: excludes.append('singletask.lock\n') 0233: bb.warn("try writing excludefile") *** 0234: with open(excludefile, 'w') as f: 0235: for line in excludes: 0236: f.write(line) 0237: 0238: bb.process.run('git checkout -b %s' % devbranch, cwd=repodir) Exception: FileNotFoundError: [Errno 2] No such file or directory: '.../devtooltmp-6m36b181/workdir/foobar-1.0.1/.git/info/exclude' Fix this edge case by creating the .git/info/ directory first. Fixes: 334ba846c795fc0d8c73 (2018-02-01, "devtool: set up git repos so that singletask.lock is ignored") (From OE-Core rev: 148a23e4d5ceaf655ccacb52deca4ba501f12975) Signed-off-by: Roland Hieber Signed-off-by: Richard Purdie

devtool: use -f and don't use --exclude-standard when adding files to workspace

2020-06-04T12:27:29+00:00

* I see a case where a tarball contains .gitignore and bunch of files which are normally ignored in git, but still included in the tarball (e.g. configure script next to configure.ac) * when devtool is creating a git repo in workspace it won't include these files from tarball in the initial devtool-base commit, because git ls-files won't list them * but then the first .patch file (without git headers) when applied with GitApplyTree._applypatch() will add all these still ignored files to a commit which used to only modify some files, because it's using -f: # Add all files shellcmd = ["git", "add", "-f", "-A", "."] output += runcmd(["sh", "-c", " ".join(shellcmd)], self.dir) at least in this case it would be better to add all ignored files in the initial devtool-base commit and then --force-patch-refresh will just include the small modification as before instead of adding unrelated files, just because they were initially ignored - this behavior will also match with the do_patch task in the actual build where the .gitignore is ignored when unpacking some tarball * my use-case is fixed in setup_git_repo, but similar function is in devtool upgrade, I've changed it there as well (From OE-Core rev: 06a24a615549af3550302a56ea08147000a608f3) Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie

meta/lib+scripts: Convert to SPDX license headers

2019-05-09T15:31:55+00:00

This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie

devtool: prevent starting git gc in background

2019-03-24T17:00:52+00:00

Devtool creates a git repository for extracted sources in a temporary directory and then moves it to a final destination after patching is done. Unfortunately devtool is not aware that some of its git operations may have caused git garbage collector to start in background. If timing is just right a repository move fails because GC is removing git objects while they are being moved. The issue was hit on Krogoth release, but the code that moves created git repository is still the same. Disable gc.autodetach to make GC run synchronously and block git until it is done. Traceback (most recent call last): File "/openembedded-core/scripts/devtool", line 342, in ret = main() File "/openembedded-core/scripts/devtool", line 329, in main ret = args.func(args, config, basepath, workspace) File "/openembedded-core/scripts/lib/devtool/standard.py", line 352, in extract initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, rd) File "/openembedded-core/scripts/lib/devtool/standard.py", line 644, in _extract_source shutil.move(srcsubdir, srctree) File "/usr/lib/python2.7/shutil.py", line 299, in move copytree(src, real_dst, symlinks=True) File "/usr/lib/python2.7/shutil.py", line 208, in copytree raise Error, errors Error: [('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e', '/build/grpc/grpc_src/.git/objects/5e', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e'"), ('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57', '/build/grpc/grpc_src/.git/objects/57', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57'"), many git objects ... ] (From OE-Core rev: 3da578a79195a63fb6d4bbd68d8f1548defd205f) Signed-off-by: Taras Kondratiuk Signed-off-by: Richard Purdie

devtool: set up git repos so that singletask.lock is ignored

2018-02-06T11:06:27+00:00

singletask.lock is written out while certain tasks execute for recipes that have externalsrc.bbclass enabled - this includes recipes in devtool's workspace. It appears that there's a race where singletask.lock will be there one minute and then when we try to get the file checksum of it (since we want to know if anything in the source tree has changed) it will be gone, and git chokes. To fix that, add singletask.lock to .git/info/exclude in the repository, regardless of whether we created the repository or not. In any case singletask.lock should never be tracked by git, so this is a good thing to be doing for that reason as well. This fixes oe-selftest failures in test_devtool_modify that we've seen on the Yocto Project autobuilder: bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception CalledProcessError: Command '['git', 'add', '-A', '.']' returned non-zero exit status 128. Note that this only fixes this issue for devtool; if you are using externalsrc independently of devtool there's a chance this will still be an issue unless you add singletask.lock to your .gitignore. (From OE-Core rev: 334ba846c795fc0d8c73ce05a1b0882739c86650) Signed-off-by: Paul Eggleton Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

devtool: finish: ensure repository is clean before proceeding

2017-11-11T12:14:27+00:00

If the git repository for a recipe in the workspace has uncommitted changes in it then it's possible that the user has forgotten to commit something, so check and exit if there are any. Provide a -f/--force option to continue in the case where the uncommitted changes aren't needed. Separately, if the repository is in the middle of a rebase or git am / apply then error out (without the opportunity to force) since the user really needs to sort this out before finishing. (From OE-Core rev: bfebd18982c0c82ef2da63ec8f22175c93b2e308) Signed-off-by: Paul Eggleton Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

devtool: show some warnings for upgrade versions

2017-11-11T12:14:27+00:00

* Show a warning in devtool upgrade if the version is less than the current version suggesting that the user may need to bump PE in the recipe * Show a warning in devtool add and devtool upgrade if the version looks like a pre-release version suggesting using a version number that won't mess up the progression when you come to upgrade to the final release version. (From OE-Core rev: 92c4d9be9ed365c449ea1ac5a18ddb15e4a36b2d) Signed-off-by: Paul Eggleton Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

devtool: ensure recipes devtool is working on are unlocked within the eSDK

2017-09-18T10:07:29+00:00

Alongside reworking the way devtool extracts source, we now need to ensure that within the extensible SDK where task signatures are locked, the signatures of the tasks for the recipes being worked on get unlocked at the right time or otherwise we'll now get taskhash mismatches when running devtool modify on a recipe that was included in the eSDK such as the kernel (due to a separate bug). The existing mechanism for auto-unlocking recipes was a little weak and was happening too late, so I've reimplemented it so that: (a) it gets triggered immediately when the recipe/append is created (b) we avoid writing to the unlocked signatures file unnecessarily (since it's a global configuration file) and (c) within the eSDK configuration we whitelist SIGGEN_UNLOCKED_RECIPES to avoid unnecessary reparses every time we perform one of the devtool operations that does need to change this list. Fixes [YOCTO #11883] (not the underlying cause, but this manifestation of the issue). (From OE-Core rev: 4e9a0be32fc30fb87d65da7cd1a4015c99533aff) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

devtool: import: new plugin to import the devtool workspace

2017-08-23T07:47:02+00:00

Takes a tar archive created by 'devtool export' and imports (untars) it into the workspace. Currently the whole tar archive is imported, there is no way to limit what is imported. https://bugzilla.yoctoproject.org/show_bug.cgi?id=10510 [YOCTO #10510] (From OE-Core rev: 2de8ba89ef10fefcc97246dfeb4b8d1e48ee8232) Signed-off-by: Leonardo Sandoval Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

devtool: Setup logging before calling prepare

2017-07-21T07:44:25+00:00

This avoids test failures like: INFO - ====================================================================== INFO - FAIL [1.755s]: test_devtool_layer_plugins (devtool.DevtoolTests) INFO - ---------------------------------------------------------------------- INFO - Traceback (most recent call last): File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f return func(*args, **kwargs) File "/media/build1/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 1354, in test_devtool_layer_plugins self.assertEqual(result.output, s[::-1]) AssertionError: "NOTE: Starting bitbake server...\noY senu[36 chars]rciM" != "oY senuZ s'enoynA morF tiforP oN edaM tfosorciM" - NOTE: Starting bitbake server... oY senuZ s'enoynA morF tiforP oN edaM tfosorciM INFO - ---------------------------------------------------------------------- since there is corruption in the output. Setting the logging up before calling tinfoil.prepare() resolves this. (From OE-Core rev: 3c479fb17ae4d3e7e5f0889af0f68257ef66475c) Signed-off-by: Richard Purdie

linux/poky.git/scripts/lib/devtool/__init__.py, branch yocto-4.0.31