sstate.bbclass: Optimize the generation and install path fixups

The fixmepath file that is generated contains a list of all of the files
that need their paths fixed.  In the previous version the fixmepath was
generated to include all of the files that sed may have changed.  In the
new version, we first grep the files to see if they contain a path that
needs to be changed, only then do we perform the sed operation on those files.

This results in a modest performance increate in the creation of the sstate
file.  The following numbers include the do_package and do_populate_sysroot
tasks on the perl recipe.

Before the change:
real    4m23.018s
user    1m57.067s
sys     1m33.327s

After the change:
real    4m13.083s
user    1m54.062s
sys     1m26.064s

However, a more significnt performance gain is felt during the
extraction/install of sstate cache files, as the fixmepaths file now has a
significantly smaller list of files to modify.

Before the change:
real    0m39.798s
user    0m11.158s
sys     0m12.642s

After the change:
real    0m25.511s
user    0m8.408s
sys     0m5.077s

(All numbers above were recorded with a cold filesystem cache on a machine
with 12 GB of ram.)

(From OE-Core rev: 46067264bedeff8248a2b2441733420fe6651f84)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 6 insertions, 1 deletions

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index ad7d121f25..3fc615d1e7 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -321,10 +321,13 @@ def sstate_hardcode_path(d):
         sstate_builddir = d.getVar('SSTATE_BUILDDIR', True)
 
         if bb.data.inherits_class('native', d) or bb.data.inherits_class('nativesdk', d) or bb.data.inherits_class('crosssdk', d) or bb.data.inherits_class('cross-canadian', d):
+                sstate_grep_cmd = "grep -l -e '%s'" % (staging)
                 sstate_sed_cmd = "sed -i -e 's:%s:FIXMESTAGINGDIR:g'" % (staging)
         elif bb.data.inherits_class('cross', d):
+                sstate_grep_cmd = "grep -l -e '(%s|%s)'" % (staging_target, staging)
                 sstate_sed_cmd = "sed -i -e 's:%s:FIXMESTAGINGDIRTARGET:g; s:%s:FIXMESTAGINGDIR:g'" % (staging_target, staging)
         else:
+                sstate_grep_cmd = "grep -l -e '%s'" % (staging_host)
                 sstate_sed_cmd = "sed -i -e 's:%s:FIXMESTAGINGDIRHOST:g'" % (staging_host)
         
         sstate_scan_cmd = d.getVar('SSTATE_SCAN_CMD', True)
@@ -333,7 +336,9 @@ def sstate_hardcode_path(d):
         # fixmepath file needs relative paths, drop sstate_builddir prefix
         sstate_filelist_relative_cmd = "sed -i -e 's:^%s::g' %sfixmepath" % (sstate_builddir, sstate_builddir)
 
-        sstate_hardcode_cmd = "%s | %s | xargs %s" % (sstate_scan_cmd, sstate_filelist_cmd, sstate_sed_cmd)
+        # Limit the fixpaths and sed operations based on the initial grep search
+        # This has the side effect of making sure the vfs cache is hot
+        sstate_hardcode_cmd = "%s | xargs %s | %s | xargs %s" % (sstate_scan_cmd, sstate_grep_cmd, sstate_filelist_cmd, sstate_sed_cmd)
 
         print "Removing hardcoded paths from sstate package: '%s'" % (sstate_hardcode_cmd)
         os.system(sstate_hardcode_cmd)