devtool: rework source extraction so that dependencies are handled

Since it was first implemented, devtool's source extraction (as used by
the devtool modify, extract and upgrade subcommands) ignored other recipe
dependencies - so for example if you ran devtool modify on a recipe that
fetches from svn or is compressed using xz then it would fail if those
dependencies hadn't been built first. Now that we can execute tasks in
the normal way (i.e. tinfoil.build_targets()) then we can rework it to
use that. This is slightly tricky in that the source extraction needs to
insert some logic in between tasks; luckily we can use a helper class
that conditionally adds prefuncs to make that possible.

Some side-effects / aspects of this change worth noting:
* Operations are a little slower because we have to go through the task
  dependency graph generation and other startup processing. There's not
  really any way to avoid this though.
* devtool extract didn't used to require a workspace, now it does
  because it needs to create a temporary bbappend for the recipe. (As
  with other commands the workspace be created on the fly if it doesn't
  already exist.)
* I want any existing sysroot files and stamps to be left alone during
  extraction since we are running the tasks off to the side, and
  especially devtool extract should be able to be used without touching
  these. However, this was hampered by the automatic removal process in
  sstate.bbclass triggered by bb.event.ReachableStamps when the task
  signatures change, thus I had to introduce a way to disable this
  removal on a per-recipe basis (we still want it to function for any
  dependencies that we aren't working on). To implement this I elected
  to use a file written to tmp/sstate-control which gets deleted
  automatically after reading so that there's less chance of stale files
  affecting future sessions. I could have used a variable but this would
  have needed to be whitelisted and I'd have to have poked its value in
  using the setVariable command.

Fixes [YOCTO #11198].

(From OE-Core rev: 830dbd66992cbb9e731b48d56fddf8f220349666)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 5 insertions, 5 deletions

diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 1dfef599e6..3c537ee071 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -1073,7 +1073,7 @@ class DevtoolTests(DevtoolBase):
 
     @OETestID(1628)
     def test_devtool_update_recipe_local_files_subdir(self):
-        # Try devtool extract on a recipe that has a file with subdir= set in
+        # Try devtool update-recipe on a recipe that has a file with subdir= set in
         # SRC_URI such that it overwrites a file that was in an archive that
         # was also in SRC_URI
         # First, modify the recipe
@@ -1103,10 +1103,10 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         # Try devtool extract
         self.track_for_cleanup(tempdir)
+        self.track_for_cleanup(self.workspacedir)
+        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
         result = runCmd('devtool extract matchbox-terminal %s' % tempdir)
         self.assertExists(os.path.join(tempdir, 'Makefile.am'), 'Extracted source could not be found')
-        # devtool extract shouldn't create the workspace
-        self.assertNotExists(self.workspacedir)
         self._check_src_repo(tempdir)
 
     @OETestID(1379)
@@ -1114,10 +1114,10 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         # Try devtool extract
         self.track_for_cleanup(tempdir)
+        self.track_for_cleanup(self.workspacedir)
+        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
         result = runCmd('devtool extract virtual/make %s' % tempdir)
         self.assertExists(os.path.join(tempdir, 'Makefile.am'), 'Extracted source could not be found')
-        # devtool extract shouldn't create the workspace
-        self.assertNotExists(self.workspacedir)
         self._check_src_repo(tempdir)
 
     @OETestID(1168)