summaryrefslogtreecommitdiffstats
path: root/meta
diff options
context:
space:
mode:
authorRobert Yang <liezhi.yang@windriver.com>2018-11-23 10:37:56 +0800
committerRichard Purdie <richard.purdie@linuxfoundation.org>2019-01-08 20:14:42 +0000
commit90e0658f2ee057994e4f73aae2f21a04a942c317 (patch)
tree60e40b4e1dd1560e71928ed7c5b2f685ba0366c1 /meta
parent037a69934c9749fbb23ec7a3860ccc273d974e7b (diff)
downloadpoky-90e0658f2ee057994e4f73aae2f21a04a942c317.tar.gz
sstate.bbclass: Only remove sstate file when task is existed
This can improve the performance a lot for "bitbake <recipe-native/cross/crosssdk> -ccleansstate" when there are a lot of sstate files. For example: * Before $ bitbake quilt-native -ccleansstate - Check log.do_cleansstate: Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz* Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_qa.tgz* Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_write_rpm.tgz* Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_packagedata.tgz* Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz* Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz* There are no package tasks for quilt-native, so the first 4 lines doesn't make any sense, but the glob pattern "sstate-cache/*/*" is very time consuming when there are no disk caches. E.g., I have more than 600,000 sstate files: - Without disk caches # echo 3 >/proc/sys/vm/drop_caches $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")' real 4m32.583s user 0m5.768s sys 0m12.892s - With disk caches (e.g., run it in the second time) $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")' real 0m5.128s user 0m2.772s sys 0m2.308s So the 4 removing *package* commands cost more than 20s or 272s in theory. * After $ bitbake quilt-native -ccleansstate - Check log.do_cleansstate: Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz* Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz* We can see that it saved 20s or 272s in theory. (From OE-Core rev: bb2d6349ea87f090c58001f0d4348b24c2982cde) (From OE-Core rev: aa35fbeb995b62523d44c8fad17f67d9852c594f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'meta')
-rw-r--r--meta/classes/sstate.bbclass5
1 files changed, 3 insertions, 2 deletions
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index efb0096c70..9f059a04a0 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -423,8 +423,9 @@ def sstate_clean_cachefile(ss, d):
423 import oe.path 423 import oe.path
424 424
425 sstatepkgfile = d.getVar('SSTATE_PATHSPEC') + "*_" + ss['task'] + ".tgz*" 425 sstatepkgfile = d.getVar('SSTATE_PATHSPEC') + "*_" + ss['task'] + ".tgz*"
426 bb.note("Removing %s" % sstatepkgfile) 426 if d.getVarFlag('do_%s' % ss['task'], 'task'):
427 oe.path.remove(sstatepkgfile) 427 bb.note("Removing %s" % sstatepkgfile)
428 oe.path.remove(sstatepkgfile)
428 429
429def sstate_clean_cachefiles(d): 430def sstate_clean_cachefiles(d):
430 for task in (d.getVar('SSTATETASKS') or "").split(): 431 for task in (d.getVar('SSTATETASKS') or "").split():