bitbake/codeparser: Improve cache handling

The current codeparser cache handling hurts performance badly even
over a couple of cores and certainly on many core systems, it can
spent huge amounts of time in the codeparser cache save functions.

This patch reworks the cache handling so that each parsing thread
saves out its own "differences" file compared to any existing core
cache and then the main bitbake thread picks these up and merges
things back together.

This was tested on systems with small and large numbers of cores
and was found to perform orders of magnitude better in all cases
despite the more complex code.

(Bitbake rev: 9f27563d66523f5af1028f173d53ee75e0877d46)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 1 insertions, 0 deletions

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index 9ce64dbca3..397352945e 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -1161,6 +1161,7 @@ class CookerParser(object):
         sync = threading.Thread(target=self.bb_cache.sync)
         sync.start()
         atexit.register(lambda: sync.join())
+        bb.codeparser.parser_cache_savemerge(self.cooker.configuration.data)
 
     def load_cached(self):
         for filename, appends in self.fromcache: