bitbake: cooker: Fix memory resident cache invalidation issue

We've been seeing weird PRServ failures on the autobuilder. These had
one failure always followed by a second. Whilst I can't reproduce the first,
if I made that test fail, I could reproduce the second with memory resident
bitbake. This was with the tests:

prservice.BitbakePrTests.test_import_export_replace_db
and then
prservice.BitbakePrTests.test_pr_service_deb_arch_dep

which was giving a strange looking error:

NOTE: Running task 1053 of 1055 (/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/recipes-devtools/m4/m4_1.4.19.bb:do_package_write_rpm)
NOTE: Running task 1054 of 1055 (/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/recipes-devtools/m4/m4_1.4.19.bb:do_package_qa)
ERROR: No such task: do_package_write_rpm
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/recipes-devtools/m4/m4_1.4.19.bb:do_package_write_rpm) failed with exit code '1'

where the issue is that selftest.inc written by the test framework
and containing PACKAGE_CLASSES = "package_deb" was being ignored.

The issue is the cached_statements{} within BBHandler() is not being
invalidated at the right time.

This patch changes the code to ensure base configuration is not parsed
until inotify updates have been processed.

(Bitbake rev: cada37c6b9e5862ca2c5a54ad6fd1e1f1939cd9c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 10 insertions, 6 deletions

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index c631ec7e6d..1797a1d4ca 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -339,6 +339,14 @@ class BBCooker:
                         providerlog.error("Root privilege is required to modify max_user_watches.")
                     raise
 
+    def handle_inotify_updates(self):
+        # reload files for which we got notifications
+        for p in self.inotify_modified_files:
+            bb.parse.update_cache(p)
+            if p in bb.parse.BBHandler.cached_statements:
+                del bb.parse.BBHandler.cached_statements[p]
+        self.inotify_modified_files = []
+
     def sigterm_exception(self, signum, stackframe):
         if signum == signal.SIGTERM:
             bb.warn("Cooker received SIGTERM, shutting down...")
@@ -368,6 +376,7 @@ class BBCooker:
             if mod not in self.orig_sysmodules:
                 del sys.modules[mod]
 
+        self.handle_inotify_updates()
         self.setupConfigWatcher()
 
         # Need to preserve BB_CONSOLELOG over resets
@@ -1614,12 +1623,7 @@ class BBCooker:
         if self.state == state.running:
             return
 
-        # reload files for which we got notifications
-        for p in self.inotify_modified_files:
-            bb.parse.update_cache(p)
-            if p in bb.parse.BBHandler.cached_statements:
-                del bb.parse.BBHandler.cached_statements[p]
-        self.inotify_modified_files = []
+        self.handle_inotify_updates()
 
         if not self.baseconfig_valid:
             logger.debug("Reloading base configuration data")