Merge pull request #294 from advancedtelematic/bugfix/kRejectAll-pyro

Bugfix/k reject all pyro

15 files changed, 366 insertions, 134 deletions

diff --git a/CONTRIBUTING.adoc b/CONTRIBUTING.adoc
index 7255a72..df7a717 100644
--- a/CONTRIBUTING.adoc
+++ b/CONTRIBUTING.adoc
@@ -1,15 +1,4 @@
 = Contributing
 
-We welcome pull requests from everyone. Here are some notes that are useful for people working on meta-updater (this repository) and https://github.com/advancedtelematic/aktualizr[aktualizr].
-
-== Options for local.conf/site.conf
-
-[options="header"]
-|======================
-| Option                              | Effect
-| `require classes/sota_bleeding.inc` | Always build the latest master of Aktualizr
-| `BRANCH_pn-aktualizr = "mybranch"`
-`BRANCH_pn-aktualizr-native = "mybranch"` | Build `mybranch` of Aktualizr (note that both of these need to be set). This will normally be used with `require classes/sota_bleeding.inc`
-| `TOOLCHAIN_HOST_TASK_append = " nativesdk-cmake "` | Use with `bitbake -c populate_sdk core-image-minimal` to build a SDK
-
+We welcome pull requests from everyone. It may be helpful to read the README and other documentation for link:README.adoc[this repo], https://github.com/advancedtelematic/aktualizr[aktualizr], and the https://github.com/advancedtelematic/updater-repo/[updater-repo], particularly the sections and development and debugging.
 
diff --git a/README.adoc b/README.adoc
index e074913..65162fc 100644
--- a/README.adoc
+++ b/README.adoc
@@ -1,4 +1,6 @@
 = meta-updater
+:toc: macro
+:toc-title:
 
 This layer enables over-the-air updates (OTA) with https://github.com/ostreedev/ostree[OSTree] and https://github.com/advancedtelematic/aktualizr[Aktualizr].
 
@@ -6,6 +8,11 @@ https://github.com/ostreedev/ostree[OSTree] is a tool for atomic full file syste
 
 https://github.com/advancedtelematic/aktualizr[Aktualizr] (and https://github.com/advancedtelematic/rvi_sota_client[RVI SOTA client]) add authentication and provisioning capabilities to OTA and are integrated with OSTree. You can connect with the open-source https://github.com/advancedtelematic/rvi_sota_server[RVI SOTA server] or sign up for a free account at https://app.atsgarage.com[ATS Garage] to get started.
 
+[discrete]
+== Table of Contents
+
+toc::[]
+
 == Build
 
 === Quickstart
@@ -20,7 +27,7 @@ If you already have a Yocto-based project and you want to add atomic filesystem
 2.  Clone BSP integration layer (`meta-updater-$\{PLATFORM}`, e.g. https://github.com/advancedtelematic/meta-updater-raspberrypi[meta-updater-raspberrypi]) and add it to your `conf/bblayers.conf`. If your board isn't supported yet, you could write a BSP integration for it yourself. See the <<Adding support for your board>> section for the details.
 3.  Set up your https://www.yoctoproject.org/docs/2.1/ref-manual/ref-manual.html#var-DISTRO[distro]. If you are using "poky", the default distro in Yocto, you can change it in your `conf/local.conf` to "poky-sota". Alternatively, if you are using your own or third party distro configuration, you can add `INHERIT += " sota"` to it, thus combining capabilities of your distro with meta-updater features.
 
-You can then build your image as usual, with bitbake. After building the root file system, bitbake will then create an https://ostree.readthedocs.io/en/latest/manual/adapting-existing/[OSTree-enabled version] of it, commit it to your local OSTree repo and (optionally) push it to a remote server. Additionally, a live disk image will be created (normally named `$\{IMAGE_NAME}.-sdimg-ota` e.g. `core-image-raspberrypi3.rpi-sdimg-ota`). You can control this behaviour through <<variables in your local.conf,OSTree-related variables in your local.conf>>.
+You can then build your image as usual, with bitbake. After building the root file system, bitbake will then create an https://ostree.readthedocs.io/en/latest/manual/adapting-existing/[OSTree-enabled version] of it, commit it to your local OSTree repo and (optionally) push it to a remote server. Additionally, a live disk image will be created (normally named `$\{IMAGE_NAME}.-sdimg-ota` e.g. `core-image-raspberrypi3.rpi-sdimg-ota`). You can control this behaviour through <<sota-related-variables-in-localconf,variables in your local.conf>>.
 
 === Build in AGL
 
@@ -63,7 +70,7 @@ You may take a look into https://github.com/advancedtelematic/meta-updater-minno
 
 Although we have used U-Boot so far, other boot loaders can be configured work with OSTree as well.
 
-== SOTA-related variables in `local.conf`
+== SOTA-related variables in local.conf
 
 * `OSTREE_REPO` - path to your OSTree repository. Defaults to `$\{DEPLOY_DIR_IMAGE}/ostree_repo`
 * `OSTREE_OSNAME` - OS deployment name on your target device. For more information about deployments and osnames see the https://ostree.readthedocs.io/en/latest/manual/deployment/[OSTree documentation]. Defaults to "poky".
@@ -123,9 +130,26 @@ The https://github.com/advancedtelematic/aktualizr[aktualizr repo] contains a to
 garage-push --repo=/path/to/ostree-repo --ref=mybranch --credentials=/path/to/credentials.zip
 ....
 
-You can set SOTA_PACKED_CREDENTIALS in your local.conf to make your build results be automatically synchronized with a remote server. Credentials are stored in the JSON format described in the https://github.com/advancedtelematic/aktualizr/blob/master/README.sotatools.adoc[garage-push README]. This JSON file can be optionally stored inside a zip file, although if it is stored this way, the JSON file must be named treehub.json.
+You can set `SOTA_PACKED_CREDENTIALS` in your `local.conf` to automatically synchronize your build results with a remote server. Credentials are stored in an archive as described in the https://github.com/advancedtelematic/aktualizr/blob/master/docs/credentials.adoc[aktualizr documentation].
+
+== Development configuration
+
+There are a few settings that can be controlled in `local.conf` to simplify the development process:
 
-== QA with `oe-selftest`
+[options="header"]
+|======================
+| Option                              | Effect
+| `require classes/sota_bleeding.inc` | Build the latest head (by default, using the master branch) of Aktualizr
+| `BRANCH_pn-aktualizr = "mybranch"`
+
+`BRANCH_pn-aktualizr-native = "mybranch"` | Build `mybranch` of Aktualizr. Note that both of these need to be set. This is normally used in conjunction with `require classes/sota_bleeding.inc`
+| `SRCREV_pn-aktualizr = "1004efa3f86cef90c012b34620992b5762b741e3"`
+
+`SRCREV_pn-aktualizr-native = "1004efa3f86cef90c012b34620992b5762b741e3"` | Build the specified revision of Aktualizr. Note that both of these need to be set. This can be used in conjunction with `BRANCH_pn-aktualizr` and `BRANCH_pn-aktualizr-native` but will conflict with `require classes/sota_bleeding.inc`
+| `TOOLCHAIN_HOST_TASK_append = " nativesdk-cmake "` | Use with `bitbake -c populate_sdk core-image-minimal` to build an SDK. See the https://github.com/advancedtelematic/aktualizr#developing-against-an-openembedded-system[aktualizr repo] for more information.
+|======================
+
+== QA with oe-selftest
 
 This layer relies on the test framework oe-selftest for quality assurance. Follow the steps below to run the tests:
 
@@ -141,13 +165,15 @@ SANITY_TESTED_DISTROS = ""
 IMAGE_INSTALL_append = " dropbear "
 ```
 
-3. To be able to build an image for the grub tests, you will need to install https://github.com/tianocore/tianocore.github.io/wiki/OVMF[TianoCore's ovmf] package on your host system. On Debian-like systems, you can do so with this command:
+3. Some tests require that `SOTA_PACKED_CREDENTIALS` is set in your `conf/local.conf`. See the <<sota-related-variables-in-localconf,SOTA-related variables in local.conf>> section.
+
+4. To be able to build an image for the grub tests, you will need to install https://github.com/tianocore/tianocore.github.io/wiki/OVMF[TianoCore's ovmf] package on your host system. On Debian-like systems, you can do so with this command:
 +
 ```
 sudo apt install ovmf
 ```
 
-4. Run oe-selftest:
+5. Run oe-selftest:
 +
 ```
 oe-selftest --run-tests updater
diff --git a/classes/image_types_ostree.bbclass b/classes/image_types_ostree.bbclass
index dcc376d..2d1fb25 100644
--- a/classes/image_types_ostree.bbclass
+++ b/classes/image_types_ostree.bbclass
@@ -11,7 +11,6 @@ IMAGE_DEPENDS_ostree = "ostree-native:do_populate_sysroot \
 
 export OSTREE_REPO
 export OSTREE_BRANCHNAME
-
 export GARAGE_TARGET_NAME
 
 RAMDISK_EXT ?= ".ext4.gz"
@@ -198,7 +197,9 @@ IMAGE_CMD_garagesign () {
         fi
 
         rm -rf ${GARAGE_SIGN_REPO}
-        garage-sign init --repo tufrepo --home-dir ${GARAGE_SIGN_REPO} --credentials ${SOTA_PACKED_CREDENTIALS}
+        garage-sign init --repo tufrepo \
+                         --home-dir ${GARAGE_SIGN_REPO} \
+                         --credentials ${SOTA_PACKED_CREDENTIALS}
 
         ostree_target_hash=$(cat ${OSTREE_REPO}/refs/heads/${OSTREE_BRANCHNAME})
 
@@ -206,11 +207,23 @@ IMAGE_CMD_garagesign () {
         #   in which case targets.json should be pulled again and the whole procedure repeated
         push_success=0
         for push_retries in $( seq 3 ); do
-            garage-sign targets pull --repo tufrepo --home-dir ${GARAGE_SIGN_REPO}
-            garage-sign targets add --repo tufrepo --home-dir ${GARAGE_SIGN_REPO} --name ${GARAGE_TARGET_NAME} --format OSTREE --version ${ostree_target_hash} --length 0 --url "https://example.com/" --sha256 ${ostree_target_hash} --hardwareids ${MACHINE}
-            garage-sign targets sign --repo tufrepo --home-dir ${GARAGE_SIGN_REPO} --key-name=targets
+            garage-sign targets pull --repo tufrepo \
+                                     --home-dir ${GARAGE_SIGN_REPO}
+            garage-sign targets add --repo tufrepo \
+                                    --home-dir ${GARAGE_SIGN_REPO} \
+                                    --name ${GARAGE_TARGET_NAME} \
+                                    --format OSTREE \
+                                    --version ${ostree_target_hash} \
+                                    --length 0 \
+                                    --url "https://example.com/" \
+                                    --sha256 ${ostree_target_hash} \
+                                    --hardwareids ${MACHINE}
+            garage-sign targets sign --repo tufrepo \
+                                     --home-dir ${GARAGE_SIGN_REPO} \
+                                     --key-name=targets
             errcode=0
-            garage-sign targets push --repo tufrepo --home-dir ${GARAGE_SIGN_REPO} || errcode=$?
+            garage-sign targets push --repo tufrepo \
+                                     --home-dir ${GARAGE_SIGN_REPO} || errcode=$?
             if [ "$errcode" -eq "0" ]; then
                 push_success=1
                 break
diff --git a/lib/oeqa/selftest/updater.py b/lib/oeqa/selftest/updater.py
index 8fbc857..c114456 100644
--- a/lib/oeqa/selftest/updater.py
+++ b/lib/oeqa/selftest/updater.py
@@ -15,6 +15,7 @@ class SotaToolsTests(oeSelfTest):
 
     @classmethod
     def setUpClass(cls):
+        super(SotaToolsTests, cls).setUpClass()
         logger = logging.getLogger("selftest")
         logger.info('Running bitbake to build aktualizr-native tools')
         bitbake('aktualizr-native')
@@ -63,7 +64,6 @@ class GeneralTests(oeSelfTest):
                          "Java not found. Do you have a JDK installed on your host machine?")
 
     def test_add_package(self):
-        print('')
         deploydir = get_bb_var('DEPLOY_DIR_IMAGE')
         imagename = get_bb_var('IMAGE_LINK_NAME', 'core-image-minimal')
         image_path = deploydir + '/' + imagename + '.otaimg'
@@ -97,6 +97,7 @@ class AktualizrToolsTests(oeSelfTest):
 
     @classmethod
     def setUpClass(cls):
+        super(AktualizrToolsTests, cls).setUpClass()
         logger = logging.getLogger("selftest")
         logger.info('Running bitbake to build aktualizr-native tools')
         bitbake('aktualizr-native')
@@ -132,20 +133,34 @@ class AktualizrToolsTests(oeSelfTest):
         self.assertTrue(os.path.getsize(ca_path) > 0, "Client certificate at %s is empty." % ca_path)
 
 
-class QemuTests(oeSelfTest):
+class AutoProvTests(oeSelfTest):
 
-    @classmethod
-    def setUpClass(cls):
-        cls.qemu, cls.s = qemu_launch(machine='qemux86-64')
+    def setUpLocal(self):
+        layer = "meta-updater-qemux86-64"
+        result = runCmd('bitbake-layers show-layers')
+        if re.search(layer, result.output) is None:
+            # Assume the directory layout for finding other layers. We could also
+            # make assumptions by using 'show-layers', but either way, if the
+            # layers we need aren't where we expect them, we are out of like.
+            path = os.path.abspath(os.path.dirname(__file__))
+            metadir = path + "/../../../../"
+            self.meta_qemu = metadir + layer
+            runCmd('bitbake-layers add-layer "%s"' % self.meta_qemu)
+        else:
+            self.meta_qemu = None
+        self.append_config('MACHINE = "qemux86-64"')
+        self.append_config('SOTA_CLIENT_PROV = " aktualizr-auto-prov "')
+        self.qemu, self.s = qemu_launch(machine='qemux86-64')
 
-    @classmethod
-    def tearDownClass(cls):
-        qemu_terminate(cls.s)
+    def tearDownLocal(self):
+        qemu_terminate(self.s)
+        if self.meta_qemu:
+            runCmd('bitbake-layers remove-layer "%s"' % self.meta_qemu, ignore_status=True)
 
     def qemu_command(self, command):
         return qemu_send_command(self.qemu.ssh_port, command)
 
-    def test_qemu(self):
+    def test_provisioning(self):
         print('Checking machine name (hostname) of device:')
         stdout, stderr, retcode = self.qemu_command('hostname')
         self.assertEqual(retcode, 0, "Unable to check hostname. " +
@@ -153,10 +168,10 @@ class QemuTests(oeSelfTest):
         machine = get_bb_var('MACHINE', 'core-image-minimal')
         self.assertEqual(stderr, b'', 'Error: ' + stderr.decode())
         # Strip off line ending.
-        value_str = stdout.decode()[:-1]
-        self.assertEqual(value_str, machine,
-                         'MACHINE does not match hostname: ' + machine + ', ' + value_str)
-        print(value_str)
+        value = stdout.decode()[:-1]
+        self.assertEqual(value, machine,
+                         'MACHINE does not match hostname: ' + machine + ', ' + value)
+        print(value)
         print('Checking output of aktualizr-info:')
         ran_ok = False
         for delay in [0, 1, 2, 5, 10, 15]:
@@ -167,31 +182,122 @@ class QemuTests(oeSelfTest):
                 break
         self.assertTrue(ran_ok, 'aktualizr-info failed: ' + stderr.decode() + stdout.decode())
 
+        verifyProvisioned(self, machine)
+
+
+class RpiTests(oeSelfTest):
+
+    def setUpLocal(self):
+        # Add layers before changing the machine type, otherwise the sanity
+        # checker complains loudly.
+        layer_python = "meta-openembedded/meta-python"
+        layer_rpi = "meta-raspberrypi"
+        layer_upd_rpi = "meta-updater-raspberrypi"
+        result = runCmd('bitbake-layers show-layers')
+        # Assume the directory layout for finding other layers. We could also
+        # make assumptions by using 'show-layers', but either way, if the
+        # layers we need aren't where we expect them, we are out of like.
+        path = os.path.abspath(os.path.dirname(__file__))
+        metadir = path + "/../../../../"
+        if re.search(layer_python, result.output) is None:
+            self.meta_python = metadir + layer_python
+            runCmd('bitbake-layers add-layer "%s"' % self.meta_python)
+        else:
+            self.meta_python = None
+        if re.search(layer_rpi, result.output) is None:
+            self.meta_rpi = metadir + layer_rpi
+            runCmd('bitbake-layers add-layer "%s"' % self.meta_rpi)
+        else:
+            self.meta_rpi = None
+        if re.search(layer_upd_rpi, result.output) is None:
+            self.meta_upd_rpi = metadir + layer_upd_rpi
+            runCmd('bitbake-layers add-layer "%s"' % self.meta_upd_rpi)
+        else:
+            self.meta_upd_rpi = None
+
+        # This is trickier that I would've thought. The fundamental problem is
+        # that the qemu layer changes the u-boot file extension to .rom, but
+        # raspberrypi still expects .bin. To prevent this, the qemu layer must
+        # be temporarily removed if it is present. It has to be removed by name
+        # without the complete path, but to add it back when we are done, we
+        # need the full path.
+        p = re.compile(r'meta-updater-qemux86-64\s*(\S*meta-updater-qemux86-64)\s')
+        m = p.search(result.output)
+        if m and m.lastindex > 0:
+            self.meta_qemu = m.group(1)
+            runCmd('bitbake-layers remove-layer meta-updater-qemux86-64')
+        else:
+            self.meta_qemu = None
+
+        self.append_config('MACHINE = "raspberrypi3"')
+        self.append_config('SOTA_CLIENT_PROV = " aktualizr-auto-prov "')
+
+    def tearDownLocal(self):
+        if self.meta_qemu:
+            runCmd('bitbake-layers add-layer "%s"' % self.meta_qemu, ignore_status=True)
+        if self.meta_upd_rpi:
+            runCmd('bitbake-layers remove-layer "%s"' % self.meta_upd_rpi, ignore_status=True)
+        if self.meta_rpi:
+            runCmd('bitbake-layers remove-layer "%s"' % self.meta_rpi, ignore_status=True)
+        if self.meta_python:
+            runCmd('bitbake-layers remove-layer "%s"' % self.meta_python, ignore_status=True)
+
+    def test_rpi(self):
+        logger = logging.getLogger("selftest")
+        logger.info('Running bitbake to build rpi-basic-image')
+        self.append_config('SOTA_CLIENT_PROV = "aktualizr-auto-prov"')
+        bitbake('rpi-basic-image')
+        credentials = get_bb_var('SOTA_PACKED_CREDENTIALS')
+        # Skip the test if the variable SOTA_PACKED_CREDENTIALS is not set.
+        if credentials is None:
+            raise unittest.SkipTest("Variable 'SOTA_PACKED_CREDENTIALS' not set.")
+        # Check if the file exists.
+        self.assertTrue(os.path.isfile(credentials), "File %s does not exist" % credentials)
+        deploydir = get_bb_var('DEPLOY_DIR_IMAGE')
+        imagename = get_bb_var('IMAGE_LINK_NAME', 'rpi-basic-image')
+        # Check if the credentials are included in the output image.
+        result = runCmd('tar -jtvf %s/%s.tar.bz2 | grep sota_provisioning_credentials.zip' %
+                        (deploydir, imagename), ignore_status=True)
+        self.assertEqual(result.status, 0, "Status not equal to 0. output: %s" % result.output)
+
 
 class GrubTests(oeSelfTest):
 
     def setUpLocal(self):
-        # This is a bit of a hack but I can't see a better option.
+        layer_intel = "meta-intel"
+        layer_minnow = "meta-updater-minnowboard"
+        result = runCmd('bitbake-layers show-layers')
+        # Assume the directory layout for finding other layers. We could also
+        # make assumptions by using 'show-layers', but either way, if the
+        # layers we need aren't where we expect them, we are out of like.
         path = os.path.abspath(os.path.dirname(__file__))
         metadir = path + "/../../../../"
-        grub_config = 'OSTREE_BOOTLOADER = "grub"\nMACHINE = "intel-corei7-64"'
-        self.append_config(grub_config)
-        self.meta_intel = metadir + "meta-intel"
-        self.meta_minnow = metadir + "meta-updater-minnowboard"
-        runCmd('bitbake-layers add-layer "%s"' % self.meta_intel)
-        runCmd('bitbake-layers add-layer "%s"' % self.meta_minnow)
+        if re.search(layer_intel, result.output) is None:
+            self.meta_intel = metadir + layer_intel
+            runCmd('bitbake-layers add-layer "%s"' % self.meta_intel)
+        else:
+            self.meta_intel = None
+        if re.search(layer_minnow, result.output) is None:
+            self.meta_minnow = metadir + layer_minnow
+            runCmd('bitbake-layers add-layer "%s"' % self.meta_minnow)
+        else:
+            self.meta_minnow = None
+        self.append_config('MACHINE = "intel-corei7-64"')
+        self.append_config('OSTREE_BOOTLOADER = "grub"')
+        self.append_config('SOTA_CLIENT_PROV = " aktualizr-auto-prov "')
         self.qemu, self.s = qemu_launch(efi=True, machine='intel-corei7-64')
 
     def tearDownLocal(self):
         qemu_terminate(self.s)
-        runCmd('bitbake-layers remove-layer "%s"' % self.meta_intel, ignore_status=True)
-        runCmd('bitbake-layers remove-layer "%s"' % self.meta_minnow, ignore_status=True)
+        if self.meta_intel:
+            runCmd('bitbake-layers remove-layer "%s"' % self.meta_intel, ignore_status=True)
+        if self.meta_minnow:
+            runCmd('bitbake-layers remove-layer "%s"' % self.meta_minnow, ignore_status=True)
 
     def qemu_command(self, command):
         return qemu_send_command(self.qemu.ssh_port, command)
 
     def test_grub(self):
-        print('')
         print('Checking machine name (hostname) of device:')
         stdout, stderr, retcode = self.qemu_command('hostname')
         self.assertEqual(retcode, 0, "Unable to check hostname. " +
@@ -214,16 +320,32 @@ class GrubTests(oeSelfTest):
                 break
         self.assertTrue(ran_ok, 'aktualizr-info failed: ' + stderr.decode() + stdout.decode())
 
+        verifyProvisioned(self, machine)
+
 
 class ImplProvTests(oeSelfTest):
 
     def setUpLocal(self):
+        layer = "meta-updater-qemux86-64"
+        result = runCmd('bitbake-layers show-layers')
+        if re.search(layer, result.output) is None:
+            # Assume the directory layout for finding other layers. We could also
+            # make assumptions by using 'show-layers', but either way, if the
+            # layers we need aren't where we expect them, we are out of like.
+            path = os.path.abspath(os.path.dirname(__file__))
+            metadir = path + "/../../../../"
+            self.meta_qemu = metadir + layer
+            runCmd('bitbake-layers add-layer "%s"' % self.meta_qemu)
+        else:
+            self.meta_qemu = None
+        self.append_config('MACHINE = "qemux86-64"')
         self.append_config('SOTA_CLIENT_PROV = " aktualizr-implicit-prov "')
-        # note: this will build aktualizr-native as a side-effect
         self.qemu, self.s = qemu_launch(machine='qemux86-64')
 
     def tearDownLocal(self):
         qemu_terminate(self.s)
+        if self.meta_qemu:
+            runCmd('bitbake-layers remove-layer "%s"' % self.meta_qemu, ignore_status=True)
 
     def qemu_command(self, command):
         return qemu_send_command(self.qemu.ssh_port, command)
@@ -236,10 +358,10 @@ class ImplProvTests(oeSelfTest):
         machine = get_bb_var('MACHINE', 'core-image-minimal')
         self.assertEqual(stderr, b'', 'Error: ' + stderr.decode())
         # Strip off line ending.
-        value_str = stdout.decode()[:-1]
-        self.assertEqual(value_str, machine,
-                         'MACHINE does not match hostname: ' + machine + ', ' + value_str)
-        print(value_str)
+        value = stdout.decode()[:-1]
+        self.assertEqual(value, machine,
+                         'MACHINE does not match hostname: ' + machine + ', ' + value)
+        print(value)
         print('Checking output of aktualizr-info:')
         ran_ok = False
         for delay in [0, 1, 2, 5, 10, 15]:
@@ -267,36 +389,33 @@ class ImplProvTests(oeSelfTest):
         akt_native_run(self, 'aktualizr_cert_provider -c {creds} -t root@localhost -p {port} -s -g {config}'
                        .format(creds=creds, port=self.qemu.ssh_port, config=config))
 
-        # Verify that device HAS provisioned.
-        ran_ok = False
-        for delay in [5, 5, 5, 5, 10]:
-            sleep(delay)
-            stdout, stderr, retcode = self.qemu_command('aktualizr-info')
-            if retcode == 0 and stderr == b'' and stdout.decode().find('Fetched metadata: yes') >= 0:
-                ran_ok = True
-                break
-        self.assertIn(b'Device ID: ', stdout, 'Provisioning failed: ' + stderr.decode() + stdout.decode())
-        self.assertIn(b'Primary ecu hardware ID: qemux86-64', stdout,
-                      'Provisioning failed: ' + stderr.decode() + stdout.decode())
-        self.assertIn(b'Fetched metadata: yes', stdout, 'Provisioning failed: ' + stderr.decode() + stdout.decode())
-        p = re.compile(r'Device ID: ([a-z0-9-]*)\n')
-        m = p.search(stdout.decode())
-        self.assertTrue(m, 'Device ID could not be read: ' + stderr.decode() + stdout.decode())
-        self.assertGreater(m.lastindex, 0, 'Device ID could not be read: ' + stderr.decode() + stdout.decode())
-        logger = logging.getLogger("selftest")
-        logger.info('Device successfully provisioned with ID: ' + m.group(1))
+        verifyProvisioned(self, machine)
 
 
 class HsmTests(oeSelfTest):
 
     def setUpLocal(self):
+        layer = "meta-updater-qemux86-64"
+        result = runCmd('bitbake-layers show-layers')
+        if re.search(layer, result.output) is None:
+            # Assume the directory layout for finding other layers. We could also
+            # make assumptions by using 'show-layers', but either way, if the
+            # layers we need aren't where we expect them, we are out of like.
+            path = os.path.abspath(os.path.dirname(__file__))
+            metadir = path + "/../../../../"
+            self.meta_qemu = metadir + layer
+            runCmd('bitbake-layers add-layer "%s"' % self.meta_qemu)
+        else:
+            self.meta_qemu = None
+        self.append_config('MACHINE = "qemux86-64"')
         self.append_config('SOTA_CLIENT_PROV = "aktualizr-hsm-prov"')
         self.append_config('SOTA_CLIENT_FEATURES = "hsm"')
-        # note: this will build aktualizr-native as a side-effect
         self.qemu, self.s = qemu_launch(machine='qemux86-64')
 
     def tearDownLocal(self):
         qemu_terminate(self.s)
+        if self.meta_qemu:
+            runCmd('bitbake-layers remove-layer "%s"' % self.meta_qemu, ignore_status=True)
 
     def qemu_command(self, command):
         return qemu_send_command(self.qemu.ssh_port, command)
@@ -309,10 +428,11 @@ class HsmTests(oeSelfTest):
         machine = get_bb_var('MACHINE', 'core-image-minimal')
         self.assertEqual(stderr, b'', 'Error: ' + stderr.decode())
         # Strip off line ending.
-        value_str = stdout.decode()[:-1]
-        self.assertEqual(value_str, machine,
-                         'MACHINE does not match hostname: ' + machine + ', ' + value_str)
-        print(value_str)
+        value = stdout.decode()[:-1]
+        self.assertEqual(value, machine,
+                         'MACHINE does not match hostname: ' + machine + ', ' + value +
+                         '\nIs tianocore ovmf installed?')
+        print(value)
         print('Checking output of aktualizr-info:')
         ran_ok = False
         for delay in [0, 1, 2, 5, 10, 15]:
@@ -382,24 +502,7 @@ class HsmTests(oeSelfTest):
         self.assertEqual(p11_m.group(1), hsm_m.group(1), 'Slot number does not match: ' +
                          p11_err.decode() + p11_out.decode() + hsm_err.decode() + hsm_out.decode())
 
-        # Verify that device HAS provisioned.
-        ran_ok = False
-        for delay in [5, 5, 5, 5, 10]:
-            sleep(delay)
-            stdout, stderr, retcode = self.qemu_command('aktualizr-info')
-            if retcode == 0 and stderr == b'' and stdout.decode().find('Fetched metadata: yes') >= 0:
-                ran_ok = True
-                break
-        self.assertIn(b'Device ID: ', stdout, 'Provisioning failed: ' + stderr.decode() + stdout.decode())
-        self.assertIn(b'Primary ecu hardware ID: qemux86-64', stdout,
-                      'Provisioning failed: ' + stderr.decode() + stdout.decode())
-        self.assertIn(b'Fetched metadata: yes', stdout, 'Provisioning failed: ' + stderr.decode() + stdout.decode())
-        p = re.compile(r'Device ID: ([a-z0-9-]*)\n')
-        m = p.search(stdout.decode())
-        self.assertTrue(m, 'Device ID could not be read: ' + stderr.decode() + stdout.decode())
-        self.assertGreater(m.lastindex, 0, 'Device ID could not be read: ' + stderr.decode() + stdout.decode())
-        logger = logging.getLogger("selftest")
-        logger.info('Device successfully provisioned with ID: ' + m.group(1))
+        verifyProvisioned(self, machine)
 
 
 def qemu_launch(efi=False, machine=None):
@@ -466,5 +569,25 @@ def akt_native_run(testInst, cmd, **kwargs):
     testInst.assertEqual(result.status, 0, "Status not equal to 0. output: %s" % result.output)
 
 
+def verifyProvisioned(testInst, machine):
+    # Verify that device HAS provisioned.
+    ran_ok = False
+    for delay in [5, 5, 5, 5, 10]:
+        sleep(delay)
+        stdout, stderr, retcode = testInst.qemu_command('aktualizr-info')
+        if retcode == 0 and stderr == b'' and stdout.decode().find('Fetched metadata: yes') >= 0:
+            ran_ok = True
+            break
+    testInst.assertIn(b'Device ID: ', stdout, 'Provisioning failed: ' + stderr.decode() + stdout.decode())
+    testInst.assertIn(b'Primary ecu hardware ID: ' + machine.encode(), stdout,
+                  'Provisioning failed: ' + stderr.decode() + stdout.decode())
+    testInst.assertIn(b'Fetched metadata: yes', stdout, 'Provisioning failed: ' + stderr.decode() + stdout.decode())
+    p = re.compile(r'Device ID: ([a-z0-9-]*)\n')
+    m = p.search(stdout.decode())
+    testInst.assertTrue(m, 'Device ID could not be read: ' + stderr.decode() + stdout.decode())
+    testInst.assertGreater(m.lastindex, 0, 'Device ID could not be read: ' + stderr.decode() + stdout.decode())
+    logger = logging.getLogger("selftest")
+    logger.info('Device successfully provisioned with ID: ' + m.group(1))
+
 
 # vim:set ts=4 sw=4 sts=4 expandtab:
diff --git a/recipes-core/ovmf/files/0001-Compare-c-string-with-NULL-instead-of-a-literal.patch b/recipes-core/ovmf/files/0001-Compare-c-string-with-NULL-instead-of-a-literal.patch
deleted file mode 100644
index 6bdaf7e..0000000
--- a/recipes-core/ovmf/files/0001-Compare-c-string-with-NULL-instead-of-a-literal.patch
+++ /dev/null
@@ -1,25 +0,0 @@
-From 6c730f3da7490ffbba5ad17af29ca44ed167cbfc Mon Sep 17 00:00:00 2001
-From: Patrick Vacek <patrickvacek@gmail.com>
-Date: Tue, 13 Feb 2018 16:38:15 +0100
-Subject: [PATCH] Compare c-string with NULL instead of a literal.
-
----
- BaseTools/Source/C/VfrCompile/VfrUtilityLib.cpp | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/BaseTools/Source/C/VfrCompile/VfrUtilityLib.cpp b/BaseTools/Source/C/VfrCompile/VfrUtilityLib.cpp
-index 3ca57ed741..4fa066dd9f 100644
---- a/BaseTools/Source/C/VfrCompile/VfrUtilityLib.cpp
-+++ b/BaseTools/Source/C/VfrCompile/VfrUtilityLib.cpp
-@@ -3372,7 +3372,7 @@ CVfrStringDB::GetVarStoreNameFormStringId (
-   UINT8       BlockType;

-   EFI_HII_STRING_PACKAGE_HDR *PkgHeader;

-   

--  if (mStringFileName == '\0' ) {

-+  if (mStringFileName == NULL ) {

-     return NULL;

-   }

- 

--- 
-2.14.1
-
diff --git a/recipes-core/ovmf/ovmf_%.bbappend b/recipes-core/ovmf/ovmf_%.bbappend
deleted file mode 100644
index 142fc53..0000000
--- a/recipes-core/ovmf/ovmf_%.bbappend
+++ /dev/null
@@ -1,3 +0,0 @@
-FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
-
-SRC_URI_append = "file://0001-Compare-c-string-with-NULL-instead-of-a-literal.patch"
diff --git a/recipes-sota/aktualizr/aktualizr-hsm-prov.bb b/recipes-sota/aktualizr/aktualizr-hsm-prov.bb
index 944607c..5f8da3c 100644
--- a/recipes-sota/aktualizr/aktualizr-hsm-prov.bb
+++ b/recipes-sota/aktualizr/aktualizr-hsm-prov.bb
@@ -1,5 +1,5 @@
 SUMMARY = "Aktualizr configuration with HSM support"
-DESCRIPTION = "Systemd service and configurations for Aktualizr, the SOTA Client application written in C++"
+DESCRIPTION = "Systemd service and configurations for HSM provisioning with Aktualizr, the SOTA Client application written in C++"
 HOMEPAGE = "https://github.com/advancedtelematic/aktualizr"
 SECTION = "base"
 LICENSE = "MPL-2.0"
@@ -14,14 +14,15 @@ SRC_URI = " \
 PV = "1.0"
 PR = "6"
 
-
 require environment.inc
 require credentials.inc
 
 do_install() {
     install -d ${D}${libdir}/sota
-    aktualizr_implicit_writer -c ${SOTA_PACKED_CREDENTIALS} --no-root-ca \
-        -i ${STAGING_DIR_NATIVE}${libdir}/sota/sota_hsm_prov.toml -o ${D}${libdir}/sota/sota.toml -p ${D}
+    if [ -n "${SOTA_PACKED_CREDENTIALS}" ]; then
+        aktualizr_implicit_writer -c ${SOTA_PACKED_CREDENTIALS} --no-root-ca \
+            -i ${STAGING_DIR_NATIVE}${libdir}/sota/sota_hsm_prov.toml -o ${D}${libdir}/sota/sota.toml -p ${D}
+    fi
 }
 
 FILES_${PN} = " \
diff --git a/recipes-sota/aktualizr/aktualizr-implicit-prov.bb b/recipes-sota/aktualizr/aktualizr-implicit-prov.bb
index b5bf420..cf3d22c 100644
--- a/recipes-sota/aktualizr/aktualizr-implicit-prov.bb
+++ b/recipes-sota/aktualizr/aktualizr-implicit-prov.bb
@@ -4,15 +4,15 @@ HOMEPAGE = "https://github.com/advancedtelematic/aktualizr"
 SECTION = "base"
 LICENSE = "MPL-2.0"
 LIC_FILES_CHKSUM = "file://${WORKDIR}/LICENSE;md5=9741c346eef56131163e13b9db1241b3"
+
 DEPENDS = "aktualizr-native"
 RDEPENDS_${PN} = "aktualizr"
-PV = "1.0"
-PR = "1"
 
 SRC_URI = " \
   file://LICENSE \
   "
-
+PV = "1.0"
+PR = "1"
 
 require environment.inc
 require credentials.inc
diff --git a/recipes-sota/aktualizr/aktualizr_git.bb b/recipes-sota/aktualizr/aktualizr_git.bb
index cba5f87..3af9988 100644
--- a/recipes-sota/aktualizr/aktualizr_git.bb
+++ b/recipes-sota/aktualizr/aktualizr_git.bb
@@ -5,8 +5,8 @@ SECTION = "base"
 LICENSE = "MPL-2.0"
 LIC_FILES_CHKSUM = "file://${S}/LICENSE;md5=9741c346eef56131163e13b9db1241b3"
 
-DEPENDS = "boost curl openssl libarchive libsodium "
-DEPENDS_append_class-target = "jansson ostree ${@bb.utils.contains('SOTA_CLIENT_FEATURES', 'hsm', ' libp11', '', d)} "
+DEPENDS = "boost curl openssl libarchive libsodium asn1c-native "
+DEPENDS_append_class-target = "ostree ${@bb.utils.contains('SOTA_CLIENT_FEATURES', 'hsm', ' libp11', '', d)} "
 DEPENDS_append_class-native = "glib-2.0-native "
 
 RDEPENDS_${PN}_class-target = "lshw "
@@ -20,7 +20,7 @@ SRC_URI = " \
   file://aktualizr.service \
   file://aktualizr-serialcan.service \
   "
-SRCREV = "1a6432175b9fb7326173e8db35d326cc1a1011a1"
+SRCREV = "6a2a0db0e557ef0ad95e19baee516a94b67aa566"
 BRANCH ?= "master"
 
 S = "${WORKDIR}/git"
@@ -34,11 +34,14 @@ BBCLASSEXTEND =+ "native"
 
 EXTRA_OECMAKE = "-DWARNING_AS_ERROR=OFF -DCMAKE_BUILD_TYPE=Release -DAKTUALIZR_VERSION=${PV} "
 EXTRA_OECMAKE_append_class-target = " -DBUILD_OSTREE=ON -DBUILD_ISOTP=ON ${@bb.utils.contains('SOTA_CLIENT_FEATURES', 'hsm', '-DBUILD_P11=ON', '', d)} "
-EXTRA_OECMAKE_append_class-native = " -DBUILD_SOTA_TOOLS=ON -DBUILD_OSTREE=OFF "
+EXTRA_OECMAKE_append_class-native = " -DBUILD_SOTA_TOOLS=ON -DBUILD_OSTREE=OFF -DBUILD_SYSTEMD=OFF "
 
 do_install_append () {
     rm -fr ${D}${libdir}/systemd
+    rm -f ${D}${bindir}/aktualizr-secondary
+    rm -f ${D}${bindir}/aktualizr-check-discovery
 }
+
 do_install_append_class-target () {
     rm -f ${D}${bindir}/aktualizr_cert_provider
     rm -f ${D}${bindir}/aktualizr_implicit_writer
diff --git a/recipes-sota/aktualizr/environment.inc b/recipes-sota/aktualizr/environment.inc
index cba77e7..09da6b7 100644
--- a/recipes-sota/aktualizr/environment.inc
+++ b/recipes-sota/aktualizr/environment.inc
@@ -3,7 +3,7 @@ export SOTA_VIRTUAL_SECONDARIES
 
 do_install_append() {
   if [ -n "${SOTA_LEGACY_SECONDARY_INTERFACE}" ]; then
-    AKTUALIZR_PARAMETERS_LEGACYSEC="--legacy-interface ${SOTA_LEGACY_SECONDARY_INTERFACE}";
+    AKTUALIZR_PARAMETERS_LEGACYSEC="--legacy-interface ${SOTA_LEGACY_SECONDARY_INTERFACE}"
   fi
 
   AKTUALIZR_PARAMETERS_CONFIGFILE="--config /usr/lib/sota/sota.toml"
diff --git a/recipes-sota/aktualizr/files/aktualizr.service b/recipes-sota/aktualizr/files/aktualizr.service
index b6df9d7..1c2e1df 100644
--- a/recipes-sota/aktualizr/files/aktualizr.service
+++ b/recipes-sota/aktualizr/files/aktualizr.service
@@ -8,6 +8,7 @@ Requires=network-online.target
 RestartSec=10
 Restart=always
 EnvironmentFile=/usr/lib/sota/sota.env
+EnvironmentFile=-/etc/sota/sota.env
 ExecStart=/usr/bin/aktualizr $AKTUALIZR_CMDLINE_PARAMETERS
 
 [Install]
diff --git a/recipes-sota/asn1c/asn1c.bb b/recipes-sota/asn1c/asn1c.bb
new file mode 100644
index 0000000..9d1517d
--- /dev/null
+++ b/recipes-sota/asn1c/asn1c.bb
@@ -0,0 +1,17 @@
+SUMMARY = "ASN.1 to C compiler"
+DESCRIPTION = "Generates serialization routines from ASN.1 schemas"
+HOMEPAGE = "http://lionet.info/asn1c"
+SECTION = "base"
+LICENSE = "BSD"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=ee8bfaaa7d71cf3edb079475e6716d4b"
+
+inherit autotools native
+
+PV = "0.9.28"
+SRC_URI = "https://github.com/vlm/asn1c/releases/download/v${PV}/asn1c-${PV}.tar.gz \
+           file://skeletons_dir_fix.patch"
+SRC_URI[sha256sum] = "8007440b647ef2dd9fb73d931c33ac11764e6afb2437dbe638bb4e5fc82386b9"
+
+BBCLASSEXTEND = "native nativesdk"
+
+# vim:set ts=4 sw=4 sts=4 expandtab:
diff --git a/recipes-sota/asn1c/files/skeletons_dir_fix.patch b/recipes-sota/asn1c/files/skeletons_dir_fix.patch
new file mode 100644
index 0000000..f1caa2f
--- /dev/null
+++ b/recipes-sota/asn1c/files/skeletons_dir_fix.patch
@@ -0,0 +1,44 @@
+From 1a1c2c94f700cf0f4dc5dba863950b16477fdc6d Mon Sep 17 00:00:00 2001
+From: Laurent Bonnans <laurent.bonnans@here.com>
+Date: Thu, 25 Jan 2018 09:49:41 +0100
+Subject: [PATCH] Patch the skeletons directory detection
+
+Detect `share/asn1c` from `bin/` if it exists
+---
+ asn1c/asn1c.c | 9 ++++-----
+ 1 file changed, 4 insertions(+), 5 deletions(-)
+
+diff --git a/asn1c/asn1c.c b/asn1c/asn1c.c
+index eb1eff7c..dd9fc832 100644
+--- a/asn1c/asn1c.c
++++ b/asn1c/asn1c.c
+@@ -226,22 +226,21 @@ main(int ac, char **av) {
+     if(skeletons_dir == NULL) {
+         struct stat sb;
+         skeletons_dir = DATADIR;
+-        if((av[-optind][0] == '.' || av[-optind][1] == '/')
+-           && stat(skeletons_dir, &sb)) {
++        if(stat(skeletons_dir, &sb)) {
+             /*
+              * The default skeletons directory does not exist,
+              * compute it from my file name:
+-             * ./asn1c/asn1c -> ./skeletons
++             * ./asn1c/asn1c -> ./share/asn1c
+              */
+             char *p;
+             size_t len;
+ 
+             p = a1c_dirname(av[-optind]);
+ 
+-            len = strlen(p) + sizeof("/../skeletons");
++            len = strlen(p) + sizeof("/../share/asn1c");
+             skeletons_dir = malloc(len);
+             assert(skeletons_dir);
+-            snprintf(skeletons_dir, len, "%s/../skeletons", p);
++            snprintf(skeletons_dir, len, "%s/../share/asn1c", p);
+             if(stat(skeletons_dir, &sb)) {
+                 fprintf(stderr,
+                         "WARNING: skeletons are neither in "
+-- 
+2.15.1
+
diff --git a/recipes-support/libp11/files/0001-Workaround-for-a-buggy-version-of-openssl-1.0.2m.patch b/recipes-support/libp11/files/0001-Workaround-for-a-buggy-version-of-openssl-1.0.2m.patch
new file mode 100644
index 0000000..bd233ee
--- /dev/null
+++ b/recipes-support/libp11/files/0001-Workaround-for-a-buggy-version-of-openssl-1.0.2m.patch
@@ -0,0 +1,42 @@
+From ccab5ce63dd5d3dbb4bd02998d21d34407e550f2 Mon Sep 17 00:00:00 2001
+From: Anton Gerasimov <anton.gerasimov@here.com>
+Date: Fri, 19 Jan 2018 12:44:27 +0100
+Subject: [PATCH] Workaround for a buggy version of openssl (1.0.2m)
+
+---
+ src/p11_pkey.c | 12 +++++++++---
+ 1 file changed, 9 insertions(+), 3 deletions(-)
+
+diff --git a/src/p11_pkey.c b/src/p11_pkey.c
+index 45d5ad3..75625e6 100644
+--- a/src/p11_pkey.c
++++ b/src/p11_pkey.c
+@@ -139,8 +139,14 @@ static void EVP_PKEY_meth_copy(EVP_PKEY_METHOD *dst, const EVP_PKEY_METHOD *src)
+ 
+ #endif
+ 
+-#if OPENSSL_VERSION_NUMBER < 0x100020d0L || defined(LIBRESSL_VERSION_NUMBER)
+-static void EVP_PKEY_meth_get_sign(EVP_PKEY_METHOD *pmeth,
++#if OPENSSL_VERSION_NUMBER < 0x100020f0L || defined(LIBRESSL_VERSION_NUMBER)
++
++#  if (OPENSSL_VERSION_NUMBER & 0xFFFFFFF0) == 0x100020d0L
++#    undef EVP_PKEY_meth_get_sign
++#    undef EVP_PKEY_meth_get_decrypt
++#  endif
++
++void EVP_PKEY_meth_get_sign(EVP_PKEY_METHOD *pmeth,
+                int (**psign_init) (EVP_PKEY_CTX *ctx),
+                int (**psign) (EVP_PKEY_CTX *ctx,
+                        unsigned char *sig, size_t *siglen,
+@@ -152,7 +158,7 @@ static void EVP_PKEY_meth_get_sign(EVP_PKEY_METHOD *pmeth,
+                *psign = pmeth->sign;
+ }
+ 
+-static void EVP_PKEY_meth_get_decrypt(EVP_PKEY_METHOD *pmeth,
++void EVP_PKEY_meth_get_decrypt(EVP_PKEY_METHOD *pmeth,
+                int (**pdecrypt_init) (EVP_PKEY_CTX *ctx),
+                int (**pdecrypt) (EVP_PKEY_CTX *ctx,
+                        unsigned char *out,
+-- 
+2.15.1
+
diff --git a/recipes-support/libp11/libp11_0.4.7.bb b/recipes-support/libp11/libp11_0.4.7.bb
index 877a57e..02d9e50 100644
--- a/recipes-support/libp11/libp11_0.4.7.bb
+++ b/recipes-support/libp11/libp11_0.4.7.bb
@@ -9,7 +9,8 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=fad9b3332be894bab9bc501572864b29"
 DEPENDS = "libtool openssl"
 RDEPENDS_${PN} += " opensc"
 
-SRC_URI = "git://github.com/OpenSC/libp11.git"
+SRC_URI = "git://github.com/OpenSC/libp11.git \
+           file://0001-Workaround-for-a-buggy-version-of-openssl-1.0.2m.patch"
 SRCREV = "da725ab727342083478150a203a3c80c4551feb4"
 
 S = "${WORKDIR}/git"