bitbake: cooker/siggen: Empty siggen cache during parsing

When parsing recipes its apparent the memory usage of bitbake rises linearly
with number of recipes parsed. It shouldn't.

Using tracemalloc (thanks for the tip Joshua Lock) it was clear that the
dependency information left behind in siggen was the culprit. Add a new
method to allow us to drop this information. We don't need it after the recipe
has been parsed and hashes calculated (at runtime its different but only the
currently executing task would be in memory).

This should give signficant memory usage improvements for bitbake and that
in turn should help speed on more constrained systems, as well as when used in
multiconfig environments.

(Bitbake rev: 5d98d8e39bba42f458532b1eef3619f2321d8a2b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 2 insertions, 1 deletions

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index 1124332186..a05630d52d 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -1940,7 +1940,8 @@ class Parser(multiprocessing.Process):
                 except IndexError:
                     break
                 result = self.parse(*job)
-
+                # Clear the siggen cache after parsing to control memory usage, its huge
+                bb.parse.siggen.postparsing_clean_cache()
             try:
                 self.results.put(result, timeout=0.25)
             except queue.Full: