linux/poky.git/scripts/lib/devtool/upgrade.py, branch rocko-18.0.3

devtool: ensure recipes devtool is working on are unlocked within the eSDK

2017-09-18T10:07:29+00:00

Alongside reworking the way devtool extracts source, we now need to ensure that within the extensible SDK where task signatures are locked, the signatures of the tasks for the recipes being worked on get unlocked at the right time or otherwise we'll now get taskhash mismatches when running devtool modify on a recipe that was included in the eSDK such as the kernel (due to a separate bug). The existing mechanism for auto-unlocking recipes was a little weak and was happening too late, so I've reimplemented it so that: (a) it gets triggered immediately when the recipe/append is created (b) we avoid writing to the unlocked signatures file unnecessarily (since it's a global configuration file) and (c) within the eSDK configuration we whitelist SIGGEN_UNLOCKED_RECIPES to avoid unnecessary reparses every time we perform one of the devtool operations that does need to change this list. Fixes [YOCTO #11883] (not the underlying cause, but this manifestation of the issue). (From OE-Core rev: 4e9a0be32fc30fb87d65da7cd1a4015c99533aff) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

devtool: rework source extraction so that dependencies are handled

2017-09-18T10:07:29+00:00

Since it was first implemented, devtool's source extraction (as used by the devtool modify, extract and upgrade subcommands) ignored other recipe dependencies - so for example if you ran devtool modify on a recipe that fetches from svn or is compressed using xz then it would fail if those dependencies hadn't been built first. Now that we can execute tasks in the normal way (i.e. tinfoil.build_targets()) then we can rework it to use that. This is slightly tricky in that the source extraction needs to insert some logic in between tasks; luckily we can use a helper class that conditionally adds prefuncs to make that possible. Some side-effects / aspects of this change worth noting: * Operations are a little slower because we have to go through the task dependency graph generation and other startup processing. There's not really any way to avoid this though. * devtool extract didn't used to require a workspace, now it does because it needs to create a temporary bbappend for the recipe. (As with other commands the workspace be created on the fly if it doesn't already exist.) * I want any existing sysroot files and stamps to be left alone during extraction since we are running the tasks off to the side, and especially devtool extract should be able to be used without touching these. However, this was hampered by the automatic removal process in sstate.bbclass triggered by bb.event.ReachableStamps when the task signatures change, thus I had to introduce a way to disable this removal on a per-recipe basis (we still want it to function for any dependencies that we aren't working on). To implement this I elected to use a file written to tmp/sstate-control which gets deleted automatically after reading so that there's less chance of stale files affecting future sessions. I could have used a variable but this would have needed to be whitelisted and I'd have to have poked its value in using the setVariable command. Fixes [YOCTO #11198]. (From OE-Core rev: 830dbd66992cbb9e731b48d56fddf8f220349666) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

devtool: upgrade: check that user has configured git properly

2017-09-11T16:30:30+00:00

If user.name or user.email haven't been set then git rebase can't really work properly. Check that the user has set these and error out if not. (Elsewhere we are relying on OE's git patch functionality which forces a dummy OE value - that's OK there as it's completely under OE's control and therefore it's OK for a dummy OE user to be the committer, but here the rebase may require intervention so it's reasonable to have the user's actual name and email on the operation.) Fixes [YOCTO #11947]. (From OE-Core rev: 129a3be07e272013be2db17552c13b4d8cc2cf6e) (From OE-Core rev: 802829f1c38d8c5eee11ba1d9ddd37cf02597f6e) Signed-off-by: paul Signed-off-by: Richard Purdie

devtool: upgrade: workaround for recipes which apply patches conditional upon class

2017-08-31T22:30:02+00:00

If we're upgrading a recipe that appends additional patches for, say, class-native, and we're just upgrading the target variant, then when we copied the recipe into the workspace we skipped copying the additional patches for the native variant. This caused warnings because the workspace recipe is preferred. Look at SRC_URI for all variants when copying files to work around this. More work is needed to make it easier to work with recipes that use BBCLASSEXTEND where you need to build more than one variant at once, but this at least fixes the immediate ugliness. (From OE-Core rev: 56bf5e93358187e31160d7893f57906bb3dc7ad7) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

devtool: upgrade: fix handling of non-absolute paths

2017-08-31T22:30:02+00:00

If your BBLAYERS has non-absolute paths in it (e.g. "${COREBASE}/../something") then none of the paths matched in copy_recipe_files() with the result that no files got copied and you ended up with an error later on because the recipe file couldn't be found at the destination. Fix this as well as adding an explicit check to see if no files got copied - error out earlier if so. Fixes [YOCTO #10981]. (From OE-Core rev: 3861486ad06f90c8644ebab119bbc5ddb9e693ca) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

devtool: upgrade: enable branch checking when revision is provided

2017-08-23T07:47:02+00:00

When devtool upgrade is run on a recipe with revision specified that is not on master branch, and branch isn't set by --srcbranch or -B, then we should get the correct branch and append the branch to the URL. If the revision was found on multiple branches, we will display error to inform user to provide a correct branch and exit. [YOCTO #11484] (From OE-Core rev: 29ced7387a92aed17b7fe93b1654790a981734c1) Signed-off-by: Chang Rebecca Swee Fun Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

recipetool: create: reimplement fetching with normal fetch/unpack tasks

2017-07-21T07:44:25+00:00

Now that we have the ability to run the tasks in a more standard context through tinfoil, change recipetool's fetching code to use that to fetch files using it. This has the major advantage that any dependencies of do_fetch and do_unpack (e.g. for subversion or npm) will be handled automatically. This also has the beneficial side-effect of fixing a recent regression that prevented this fetch operation from working with memory resident bitbake. Also fix devtool's usage of fetch_uri() at the same time so that we can completely replace it. Fixes [YOCTO #11710]. (From OE-Core rev: 9a47a6690052ef943c0d4760630ee630fb012153) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

devtool: upgrade: fix error of import recipeutils

2017-02-23T20:49:49+00:00

upgrade.py imports oe.recipeutils in meta/lib/ but path to oe.recipeutils is not provided. This fails populate_sdk_ext. (From OE-Core rev: 5f140359f859fea9cfe8c8d9c9584bceec875adb) Signed-off-by: Luck Hoang Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

meta/scripts: Various getVar/getVarFlag expansion parameter fixes

2017-01-09T13:39:11+00:00

There were a few straggling expansion parameter removals left for getVar/getVarFlag where the odd whitespace meant they were missed on previous passes. There were also some plain broken ussages such as: d.getVar('ALTERNATIVE_TARGET', old_name, True) path = d.getVar('PATH', d, True) d.getVar('IMAGE_ROOTFS', 'True') which I've corrected (they happend to work by luck). (From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f) Signed-off-by: Richard Purdie

scripts: remove True option to getVar calls

2016-12-16T10:23:23+00:00

getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?$( ?[^,()]*), True$ (From OE-Core rev: 0a36bd96e6b29fd99a296efc358ca3e9fb5af735) Signed-off-by: Joshua Lock Signed-off-by: Ross Burton Signed-off-by: Richard Purdie