*.adoc: Update CONTRIBUTING and README

Add CONTRIBUTING.adoc and update README.adoc to match the
versions of both files from branch Rocko.

Suggested-by: Patrick Vacek <patrickvacek@gmail.com>
Signed-off-by: Leon Anavi <leon.anavi@konsulko.com>

2 files changed, 161 insertions, 33 deletions

diff --git a/CONTRIBUTING.adoc b/CONTRIBUTING.adoc
new file mode 100644
index 0000000..4d9e8f6
--- /dev/null
+++ b/CONTRIBUTING.adoc
@@ -0,0 +1,28 @@
+= Contributing
+
+We welcome pull requests from anyone. The master branch is the primary branch for development, and if you wish to add new functionality, it probably belongs there. We attempt to maintain recent previous branches and welcome bug fixes and backports for those. Currently, the actively maintained branches are:
+
+* thud
+* sumo
+* rocko
+
+Previously, some older branches were also regularly supported, and while they should still be stable, they have not been updated or actively maintained for a while. These branches include:
+
+* pyro
+* morty
+
+If you are developing with meta-updater, it may be helpful to read the README and other documentation for link:README.adoc[this repo], https://github.com/advancedtelematic/aktualizr[aktualizr], and the https://github.com/advancedtelematic/updater-repo/[updater-repo], particularly the sections about development and debugging.
+
+== Contributor checklist
+
+* OTA-enabled build succeeds for at least one platform, the resulting image boots, and an update can be installed. This check is absolutely necessary for every pull request unless it only touches documentation.
+* If your change touches platform code (like `classes/sota_<platform>.bbclass`), please check building and updating on this particular platform.
+* oe-selftest succeeds. To test meta-updater, run `oe-selftest -r updater` from a build directory with `MACHINE` set to `qemux86-64`. See the link:README.adoc#qa-with-oe-selftest[relevant section of the README] for more details.
+* Updates are forwards- and backwards-compatible. You should be able to update an OTA-enabled build before the change is applied to the version with change applied and vice versa. One should pay double attention to the compatibility when bootloader code is affected.
+* The patch/branch should be based on the latest version of the target branch. This may mean that rebasing is necessary if other PRs are merged before yours is approved.
+
+We understand that completing all these tasks might be overly tedious due to build times in Yocto. Please add a comment to your PR describing the tests you've done.
+
+== Approval
+
+PR approval should be accompanied by a comment describing what tests have been done by the reviewer.
diff --git a/README.adoc b/README.adoc
index 9615f65..32630d7 100644
--- a/README.adoc
+++ b/README.adoc
@@ -1,28 +1,33 @@
 = meta-updater
+:toc: macro
+:toc-title:
 
-This layer enables over-the-air updates (OTA) with https://github.com/ostreedev/ostree[OSTree] and https://github.com/advancedtelematic/rvi_sota_client[RVI SOTA client].
+This layer enables over-the-air updates (OTA) with https://github.com/ostreedev/ostree[OSTree] and https://github.com/advancedtelematic/aktualizr[Aktualizr].
 
 https://github.com/ostreedev/ostree[OSTree] is a tool for atomic full file system upgrades with rollback capability. OSTree has several advantages over traditional dual-bank systems, but the most important one is that it minimizes network bandwidth and data storage footprint by sharing files with the same contents across file system deployments.
 
-https://github.com/advancedtelematic/rvi_sota_client[RVI SOTA client] and/or https://github.com/advancedtelematic/aktualizr[aktualizr] add authentication and provisioning capabilities to OTA and are integrated with OSTree. You can connect with the open-source https://github.com/advancedtelematic/rvi_sota_server[RVI SOTA server] or sign up for a free account at https://app.atsgarage.com[ATS Garage] to get started.
+https://github.com/advancedtelematic/aktualizr[Aktualizr] (and https://github.com/advancedtelematic/rvi_sota_client[RVI SOTA client]) add authentication and provisioning capabilities to OTA and are integrated with OSTree. You can connect with these open-source applications or sign up for a free account at https://connect.ota.here.com/[HERE OTA Connect] to get started.
+
+[discrete]
+== Table of Contents
+
+toc::[]
 
 == Build
 
 === Quickstart
 
-If you don't already have a Yocto project that you want to add OTA to, you can use the https://docs.atsgarage.com/quickstarts/raspberry-pi.html[ATS Garage Quickstart] project to rapidly get up and running on a Raspberry Pi. It takes a standard https://www.yoctoproject.org/tools-resources/projects/poky[poky] distribution, and adds OTA and OSTree capabilities.
+If you don't already have a Yocto project that you want to add OTA to, you can use the https://docs.atsgarage.com/quickstarts/raspberry-pi.html[HERE OTA Connect Quickstart] project to rapidly get up and running on a Raspberry Pi. It takes a standard https://www.yoctoproject.org/tools-resources/projects/poky[poky] distribution, and adds OTA and OSTree capabilities.
 
 === Adding meta-updater capabilities to your build
 
 If you already have a Yocto-based project and you want to add atomic filesystem updates to it, you just need to do three things:
 
-1.  Clone the `meta-updater` layer and add it to your https://www.yoctoproject.org/docs/2.1/ref-manual/ref-manual.html#structure-build-conf-bblayers.conf[bblayers.conf].
-2.  Clone BSP integration layer (meta-updater-$\{PLATFORM}, e.g. https://github.com/advancedtelematic/meta-updater-raspberrypi[meta-updater-raspberrypi]) and add it to your conf/bblayers.conf. If your board isn't supported yet, you could write a BSP integration for it yourself. See the <<Adding support for your board>> section for the details.
-3.  Set up your https://www.yoctoproject.org/docs/2.1/ref-manual/ref-manual.html#var-DISTRO[distro]. If you are using "poky", the default distro in Yocto, you can change it in your conf/local.conf to "poky-sota". Alternatively, if you are using your own or third party distro configuration, you can add 'INHERIT += " sota"' to it, thus combining capabilities of your distro with meta-updater features.
-
-You can then build your image as usual, with bitbake. After building the root file system, bitbake will then create an https://ostree.readthedocs.io/en/latest/manual/adapting-existing/[OSTree-enabled version] of it, commit it to your local OSTree repo and (optionally) push it to a remote server. Additionally, a live disk image will be created (normally named $\{IMAGE_NAME}.-sdimg-ota e.g. core-image-raspberrypi3.rpi-sdimg-ota). You can control this behaviour through <<variables in your local.conf,OSTree-related variables in your local.conf>>.
+1.  Clone the `meta-updater` layer and add it to your https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#structure-build-conf-bblayers.conf[bblayers.conf].
+2.  Clone BSP integration layer (`meta-updater-$\{PLATFORM}`, e.g. https://github.com/advancedtelematic/meta-updater-raspberrypi[meta-updater-raspberrypi]) and add it to your `conf/bblayers.conf`. If your board isn't supported yet, you could write a BSP integration for it yourself. See the <<Adding support for your board>> section for the details.
+3.  Set up your https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DISTRO[distro]. If you are using "poky", the default distro in Yocto, you can change it in your `conf/local.conf` to "poky-sota". Alternatively, if you are using your own or third party distro configuration, you can add `INHERIT += " sota"` to it, thus combining capabilities of your distro with meta-updater features.
 
-=== Build with OpenIVI
+You can then build your image as usual, with bitbake. After building the root file system, bitbake will then create an https://ostree.readthedocs.io/en/latest/manual/adapting-existing/[OSTree-enabled version] of it, commit it to your local OSTree repo and (optionally) push it to a remote server. Additionally, a live disk image will be created (normally named `$\{IMAGE_NAME}.-sdimg-ota` e.g. `core-image-raspberrypi3.rpi-sdimg-ota`). You can control this behaviour through <<sota-related-variables-in-localconf,variables in your local.conf>>.
 
 === Build in AGL
 
@@ -32,19 +37,23 @@ With AGL you can just add agl-sota feature while configuring your build environm
 source meta-agl/scripts/aglsetup.sh -m porter agl-demo agl-appfw-smack agl-devel agl-sota
 ....
 
-you can then run
+You can then run:
 
 ....
 bitbake agl-demo-platform
 ....
 
-and get as a result an "ostree_repo" folder in your images directory (tmp/deploy/images/$\{MACHINE}/ostree_repo). It will contain
+and get as a result an `ostree_repo` folder in your images directory (`tmp/deploy/images/$\{MACHINE}/ostree_repo`). It will contain:
 
 * your OSTree repository, with the rootfs committed as an OSTree deployment,
-* an 'otaimg' bootstrap image, which is an OSTree physical sysroot as a burnable filesystem image, and optionally
-* some machine-dependent live images (e.g. '_.rpi-sdimg-ota' for Raspberry Pi or '_.porter-sdimg-ota' Renesas Porter board).
+* an `otaimg` bootstrap image, which is an OSTree physical sysroot as a burnable filesystem image, and optionally
+* some machine-dependent live images (e.g. `.wic` for Raspberry Pi or `.porter-sdimg-ota` Renesas Porter board).
+
+Although `aglsetup.sh` hooks provide reasonable defaults for SOTA-related variables, you may want to tune some of them.
+
+=== Build problems
 
-Although aglsetup.sh hooks provide reasonable defaults for SOTA-related variables, you may want to tune some of them.
+Ubuntu users that encounter an error due to missing `Python.h` should install `libpython2.7-dev` on their host machine.
 
 == Supported boards
 
@@ -67,35 +76,39 @@ Although we have used U-Boot so far, other boot loaders can be configured work w
 
 == SOTA-related variables in local.conf
 
-* OSTREE_REPO - path to your OSTree repository. Defaults to "$\{DEPLOY_DIR_IMAGE}/ostree_repo"
-* OSTREE_BRANCHNAME - the branch your rootfs will be committed to. Defaults to "ota"
-* OSTREE_OSNAME - OS deployment name on your target device. For more information about deployments and osnames see the https://ostree.readthedocs.io/en/latest/manual/deployment/[OSTree documentation]. Defaults to "poky".
-* OSTREE_INITRAMFS_IMAGE - initramfs/initrd image that is used as a proxy while booting into OSTree deployment. Do not change this setting unless you are sure that your initramfs can serve as such a proxy.
-* OSTREE_PUSH_CREDENTIALS - when set, your ostree commit will be pushed to a remote repo as a bitbake step. This should be the path to a JSON credentials file in https://github.com/advancedtelematic/sota-tools#credentials[the format accepted by garage-push].
+* `OSTREE_REPO` - path to your OSTree repository. Defaults to `$\{DEPLOY_DIR_IMAGE}/ostree_repo`
+* `OSTREE_OSNAME` - OS deployment name on your target device. For more information about deployments and osnames see the https://ostree.readthedocs.io/en/latest/manual/deployment/[OSTree documentation]. Defaults to "poky".
+* `OSTREE_INITRAMFS_IMAGE` - initramfs/initrd image that is used as a proxy while booting into OSTree deployment. Do not change this setting unless you are sure that your initramfs can serve as such a proxy.
+* `SOTA_PACKED_CREDENTIALS` - when set, your ostree commit will be pushed to a remote repo as a bitbake step. This should be the path to a zipped credentials file in https://github.com/advancedtelematic/aktualizr/blob/master/docs/credentials.adoc[the format accepted by garage-push].
+* `SOTA_DEPLOY_CREDENTIALS` - when set to '1' (default value), deploys credentials to the built image. Override it in `local.conf` to built a generic image that can be provisioned manually after the build.
+* `SOTA_CLIENT_PROV` - which provisioning method to use. Valid options are https://github.com/advancedtelematic/aktualizr/blob/master/docs/automatic-provisioning.adoc[`aktualizr-auto-prov`], https://github.com/advancedtelematic/aktualizr/blob/master/docs/implicit-provisioning.adoc[`aktualizr-ca-implicit-prov`], and https://github.com/advancedtelematic/aktualizr/blob/master/docs/hsm-provisioning.adoc[`aktualizr-hsm-prov`]. The default is `aktualizr-auto-prov`. This can also be set to an empty string to avoid using a provisioning recipe.
+* `SOTA_CLIENT_FEATURES` - extensions to aktualizr. The only valid options are `hsm` (to build with HSM support) and `secondary-network` (to set up a simulated 'in-vehicle' network with support for a primary node with a DHCP server and a secondary node with a DHCP client).
+* `SOTA_SECONDARY_CONFIG_DIR` - a directory containing JSON configuration files for virtual secondaries on the host. These will be installed into `/etc/sota/ecus` on the device and automatically provided to aktualizr.
+* `SOTA_HARDWARE_ID` - a custom hardware ID that will be written to the aktualizr config. Defaults to MACHINE if not set.
 
 == Usage
 
 === OSTree
 
-OSTree includes its own simple http server. It just exposes the whole OSTree repository to the network so that any remote device can pull data from it to device's local repository. To use the OSTree http server, you will need OSTree installed on your build machine. (Alternatively, you could run version built inside Yocto using bitbake's http://www.openembedded.org/wiki/Devshell[devshell].)
-
-To expose your repo, run ostree trivial-httpd using any free port:
+OSTree used to include a simple HTTP server as part of the ostree binary, but this has been removed in more recent versions. However, OSTree repositories are self-contained directories, and can be trivially served over the network using any HTTP server. For example, you could use Python's SimpleHTTPServer:
 
 ....
-ostree trivial-httpd tmp/deploy/images/qemux86-64/ostree_repo -P 57556
+cd tmp/deploy/images/qemux86-64/ostree_repo
+python -m SimpleHTTPServer <port> # port defaults to 8000
 ....
 
 You can then run ostree from inside your device by adding your repo:
 
 ....
-# agl-remote identifies the remote server in your local repo
-ostree remote add --no-gpg-verify my-remote http://192.168.7.1:57556 ota
+# This behaves like adding a Git remote; you can name it anything
+ostree remote add --no-gpg-verify my-remote http://<your-ip>:<port>
 
-# ota is a branch name in the remote repo, set in OSTREE_BRANCHNAME
-ostree pull my-remote ota
+# If OSTREE_BRANCHNAME is set in local.conf, that will be the name of the
+# branch. If not set, it defaults to the value of MACHINE (e.g. qemux86-64).
+ostree pull my-remote <branch>
 
-# poky is OS name as set in OSTREE_OSNAME
-ostree admin deploy --os=poky my-remote:ota
+# poky is the OS name as set in OSTREE_OSNAME
+ostree admin deploy --os=poky my-remote:<branch>
 ....
 
 After restarting, you will boot into the newly deployed OS image.
@@ -113,12 +126,99 @@ ostree pull agl-snapshot agl-ota
 ostree admin deploy --os=agl agl-snapshot:agl-ota
 ....
 
-=== SOTA tools
+=== garage-push
 
-SOTA tools currently contains only one tool, garage-push, which lets you push the changes in OSTree repository generated by bitbake process. It communicates with an http server capable of querying files with HEAD requests and uploading them with POST requests. In particular, this can be used with http://www.atsgarage.com/[ATS Garage]. garage-push is used as follows:
+The https://github.com/advancedtelematic/aktualizr[aktualizr repo] contains a tool, garage-push, which lets you push the changes in OSTree repository generated by bitbake process. It communicates with an http server capable of querying files with HEAD requests and uploading them with POST requests. In particular, this can be used with https://connect.ota.here.com/[HERE OTA Connect]. garage-push is used as follows:
 
 ....
-garage-push --repo=/path/to/ostree-repo --ref=mybranch --credentials=/path/to/credentials.json
+garage-push --repo=/path/to/ostree-repo --ref=mybranch --credentials=/path/to/credentials.zip
 ....
 
-You can set OSTREE_PUSH_CREDENTIALS in your local.conf to make your build results be automatically synchronized with a remote server. Credentials are stored in the JSON format described in the https://github.com/advancedtelematic/sota-tools#credentials[sota-tools README].
+You can set `SOTA_PACKED_CREDENTIALS` in your `local.conf` to automatically synchronize your build results with a remote server. Credentials are stored in an archive as described in the https://github.com/advancedtelematic/aktualizr/blob/master/docs/credentials.adoc[aktualizr documentation].
+
+=== aktualizr configuration
+
+https://github.com/advancedtelematic/aktualizr[Aktualizr] supports a variety of https://github.com/advancedtelematic/aktualizr/blob/master/docs/configuration.adoc[configuration options via a configuration file and the command line]. There are two primary ways to control aktualizr's configuration from meta-updater.
+
+First, you can set `SOTA_CLIENT_PROV` to control which provisioning recipe is used. Each recipe installs an appropriate `sota.toml` file from aktualizr according to the provisioning needs. See the <<sota-related-variables-in-localconf,SOTA-related variables in local.conf>> section for more information.
+
+Second, you can write recipes to install additional config files with customized options. A few recipes already exist to address common needs and provide an example:
+
+* link:recipes-sota/config/aktualizr-disable-send-ip.bb[aktualizr-disable-send-ip.bb] disables the reporting of networking information to the server. This is enabled by default and supported by https://connect.ota.here.com/[HERE OTA Connect]. However, if you are using a different server that does not support this feature, you may want to disable it in aktualizr.
+* link:recipes-sota/config/aktualizr-log-debug.bb[aktualizr-log-debug.bb] sets the log level of aktualizr to 0 (trace). The default is 2 (info). This recipe is intended for development and debugging purposes.
+
+To use these recipes, you will need to add them to your image with a line such as `IMAGE_INSTALL_append = " aktualizr-log-debug "` in your `local.conf`.
+
+== Development configuration
+
+There are a few settings that can be controlled in `local.conf` to simplify the development process:
+
+[options="header"]
+|======================
+| Option                              | Effect
+| `require classes/sota_bleeding.inc` | Build the latest head (by default, using the master branch) of Aktualizr
+| `BRANCH_pn-aktualizr = "mybranch"`
+
+`BRANCH_pn-aktualizr-native = "mybranch"` | Build `mybranch` of Aktualizr. Note that both of these need to be set. This is normally used in conjunction with `require classes/sota_bleeding.inc`
+| `SRCREV_pn-aktualizr = "1004efa3f86cef90c012b34620992b5762b741e3"`
+
+`SRCREV_pn-aktualizr-native = "1004efa3f86cef90c012b34620992b5762b741e3"` | Build the specified revision of Aktualizr. Note that both of these need to be set. This can be used in conjunction with `BRANCH_pn-aktualizr` and `BRANCH_pn-aktualizr-native` but will conflict with `require classes/sota_bleeding.inc`
+| `TOOLCHAIN_HOST_TASK_append = " nativesdk-cmake "` | Use with `bitbake -c populate_sdk core-image-minimal` to build an SDK. See the https://github.com/advancedtelematic/aktualizr#developing-against-an-openembedded-system[aktualizr repo] for more information.
+|======================
+
+=== Overriding target version
+*Warning: overriding target version is a dangerous operation, make sure you understand this section completely before doing it.*
+
+Every time you build an image with `SOTA_PACKED_CREDENTIALS` set, a new entry in your Uptane metadata is created and you can see it in the OTA Garage UI if you're using one. Normally this version will be equal to OSTree hash of your root file system. If you want it to be different though you can override is using one of two methods:
+
+1. Set `GARAGE_TARGET_VERSION` variable in your `local.conf`.
+2. Write a recipe or a bbclass to write the desired version to `${STAGING_DATADIR_NATIVE}/target_version`. An example of such bbclass can be found in `classes/target_version_example.bbclass`.
+
+Please note that [target name, target version] pairs are expected to be unique in the system. If you build a new target with the same target version as a previously built one, the old package will be overwritten on the update server. It can have unpredictable effect on devices that have this version installed, and it is not guaranteed that information will be reported correctly for such devices or that you will be able to update them (we're doing our best though). The easiest way to avoid problems is to make sure that your overriding version is as unique as an OSTree commit hash.
+
+== QA with oe-selftest
+
+This layer relies on the test framework oe-selftest for quality assurance. Currently, you will need to run this in a build directory with `MACHINE` set to `qemux86-64`. Follow the steps below to run the tests:
+
+1. Append the line below to `conf/local.conf` to disable the warning about supported operating systems:
++
+```
+SANITY_TESTED_DISTROS = ""
+```
+
+2. If your image does not already include an ssh daemon such as dropbear or openssh, add this line to `conf/local.conf` as well:
++
+```
+IMAGE_INSTALL_append = " dropbear "
+```
+
+3. Some tests require that `SOTA_PACKED_CREDENTIALS` is set in your `conf/local.conf`. See the <<sota-related-variables-in-localconf,SOTA-related variables in local.conf>> section.
+
+4. To be able to build an image for the grub tests, you will need to install https://github.com/tianocore/tianocore.github.io/wiki/OVMF[TianoCore's ovmf] package on your host system. On Debian-like systems, you can do so with this command:
++
+```
+sudo apt install ovmf
+```
+
+5. Run oe-selftest:
++
+```
+oe-selftest --run-tests updater
+```
+
+For more information about oe-selftest, including details about how to run individual test modules or classes, please refer to the https://wiki.yoctoproject.org/wiki/Oe-selftest[Yocto Project wiki].
+
+== Manual provisoning
+
+As described in <<sota-related-variables-in-localconf,SOTA-related variables in local.conf>> section you can set `SOTA_DEPLOY_CREDENTIALS` to `0` to prevent deploying credentials to the built `wic` image. In this case you get a generic image that you can use e.g. on a production line to flash a series of devices. The cost of this approach is that this image is half-baked and should be provisioned before it can connect to the backend.
+
+Provisioning procedure depends on your provisioning recipe, i.e. the value of `SOTA_CLIENT_PROV` (equal to `aktualizr-auto-prov` by default):
+
+* For `aktualizr-auto-prov` put your `credentials.zip` to `/var/sota/sota_provisioning_credentials.zip` on the filesystem of a running device. If you have the filesystem of our device mounted to your build machine, prefix all paths with `/ostree/deploy/poky` as in `/ostree/deploy/poky/var/sota/sota_provisioning_credentials.zip`.
+* For `aktualizr-ca-implicit-prov`
+** put URL to the backend server (together with protocol prefix and port number) at `/var/sota/gateway.url`. If you're using HERE OTA Connect, you can find the URL in the `autoprov.url` file in your credentials archive.
+** put client certificate, private key and root CA certificate (for the *server*, not for the *device*) at `/var/sota/import/client.pem`, `/var/sota/import/pkey.pem` and `/var/sota/import/root.crt` respectively.
+* For  `aktualizr-hsm-prov`
+** put URL to the server backend (together with protocol prefix and port number) at `/var/sota/gateway.url`. If you're using HERE OTA Connect, you can find the URL in the `autoprov.url` file in your credentials archive.
+** put root CA certificate (for the *server*, not for the *device*) at `/var/sota/import/root.crt`.
+** put client certificate and private key to slots 1 and 2 of the PKCS#11-compatible device.