package_manager/oe-pkgdata-util: fix complementary package installation

We currently have a problem regarding complementary package installation,
that is, if 'oe-pkgdata-util glob' maps out packages that are not in
the oe-rootfs-repo, we will get error like below:

  No match for argument: lib32-glibc-locale-en-gb
  Error: Unable to find a match: lib32-glibc-locale-en-gb

Here are the steps to reproduce the issue:
1. Add the following lines to local.conf:
   require conf/multilib.conf
   MULTILIBS ?= "multilib:lib32"
   DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32"
   IMAGE_INSTALL:append = " lib32-sysstat"
2. bitbake lib32-glibc-locale && bitbake core-image-full-cmdline

This problem appears because:
1) At do_rootfs time, we first contruct a repo with a filtering
   mechanism to ensure we don't pull in unneeded packages.[1]
2) oe-pkgdata-util uses the pkgdata without filtering.

In order to avoid any hardcoding that might grow in the future[2], we need
to give 'oe-pkgdata-util glob' some filtering ability.

So this patch does the following things:
1) Add a new option, '-a/--allpkgs', to 'oe-pkgdata-util glob'.
   This gives it a filtering mechanism. As it's an option, people who use
   'oe-pkgdata-util glob' command could use it as before.
2) Add to package_manager 'list_all' function implementations which
   list all available functions in our filtered repo.

[1] https://git.openembedded.org/openembedded-core/commit/?id=85e72e129362db896b0d368077033e4a2e373cf9
[2] https://lists.openembedded.org/g/openembedded-core/message/221449

(From OE-Core rev: 16c52f992cf35769eecb3e3863e1ad14d4cb9848)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 14 insertions, 0 deletions

diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
index 44ae40549a..5b7cd768a4 100755
--- a/scripts/oe-pkgdata-util
+++ b/scripts/oe-pkgdata-util
@@ -51,6 +51,15 @@ def glob(args):
 
     skippedpkgs = set()
     mappedpkgs = set()
+    allpkgs = set()
+    if args.allpkgs:
+        with open(args.allpkgs, 'r') as f:
+            for line in f:
+                fields = line.rstrip().split()
+                if not fields:
+                    continue
+                else:
+                    allpkgs.add(fields[0])
     with open(args.pkglistfile, 'r') as f:
         for line in f:
             fields = line.rstrip().split()
@@ -136,6 +145,10 @@ def glob(args):
                         logger.debug("%s is not a valid package!" % (pkg))
                         break
 
+                if args.allpkgs:
+                    if mappedpkg not in allpkgs:
+                        continue
+
                 if mappedpkg:
                     logger.debug("%s (%s) -> %s" % (pkg, g, mappedpkg))
                     mappedpkgs.add(mappedpkg)
@@ -592,6 +605,7 @@ def main():
     parser_glob.add_argument('pkglistfile', help='File listing packages (one package name per line)')
     parser_glob.add_argument('glob', nargs="+", help='Glob expression for package names, e.g. *-dev')
     parser_glob.add_argument('-x', '--exclude', help='Exclude packages matching specified regex from the glob operation')
+    parser_glob.add_argument('-a', '--allpkgs', help='File listing all available packages (one package name per line)')
     parser_glob.set_defaults(func=glob)