summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--CONTRIBUTING.adoc16
-rw-r--r--README.adoc308
-rw-r--r--classes/image_types_ostree.bbclass19
-rw-r--r--classes/sota.bbclass2
-rw-r--r--lib/oeqa/selftest/cases/updater_qemux86_64.py14
-rw-r--r--recipes-core/images/initramfs-ostree-image.bb2
-rw-r--r--recipes-sota/aktualizr/aktualizr_git.bb10
-rw-r--r--scripts/qemucommand.py50
-rwxr-xr-xscripts/run-qemu-ota26
9 files changed, 129 insertions, 318 deletions
diff --git a/CONTRIBUTING.adoc b/CONTRIBUTING.adoc
index 0b40438..24916cc 100644
--- a/CONTRIBUTING.adoc
+++ b/CONTRIBUTING.adoc
@@ -1,17 +1,9 @@
1= Contributing 1= Contributing
2:aktualizr-docsroot: https://github.com/advancedtelematic/aktualizr/tree/master/docs/ota-client-guide/modules/ROOT/pages/
2 3
3We welcome pull requests from anyone. The master branch is the primary branch for development, and if you wish to add new functionality, it probably belongs there. We attempt to maintain recent previous branches and welcome bug fixes and backports for those. Currently, the actively maintained branches are: 4We welcome pull requests from anyone. The master branch is the primary branch for development, and if you wish to add new functionality, it probably belongs there. We attempt to maintain recent release branches and welcome bug fixes and backports for those. Please see the xref:{aktualizr-docsroot}yocto-release-branches.adoc[release branches] documentation for the current list of supported branches.
4 5
5* thud 6If you are developing with meta-updater, it may be helpful to read the README and other documentation for xref:README.adoc[this repo], https://github.com/advancedtelematic/aktualizr[aktualizr], and the link:https://github.com/advancedtelematic/updater-repo/[updater-repo], particularly the sections about development and debugging.
6* sumo
7* rocko
8
9Previously, some older branches were also regularly supported, and while they should still be stable, they have not been updated or actively maintained for a while. These branches include:
10
11* pyro
12* morty
13
14If you are developing with meta-updater, it may be helpful to read the README and other documentation for link:README.adoc[this repo], https://github.com/advancedtelematic/aktualizr[aktualizr], and the https://github.com/advancedtelematic/updater-repo/[updater-repo], particularly the sections about development and debugging.
15 7
16== Developer Certificate of Origin (DCO) 8== Developer Certificate of Origin (DCO)
17 9
@@ -23,7 +15,7 @@ New pull requests will automatically be checked by the https://probot.github.io/
23 15
24* OTA-enabled build succeeds for at least one platform, the resulting image boots, and an update can be installed. This check is absolutely necessary for every pull request unless it only touches documentation. 16* OTA-enabled build succeeds for at least one platform, the resulting image boots, and an update can be installed. This check is absolutely necessary for every pull request unless it only touches documentation.
25* If your change touches platform code (like `classes/sota_<platform>.bbclass`), please check building and updating on this particular platform. 17* If your change touches platform code (like `classes/sota_<platform>.bbclass`), please check building and updating on this particular platform.
26* oe-selftest succeeds. To test meta-updater, run `oe-selftest -r updater` from a build directory with `MACHINE` set to `qemux86-64`. See the link:README.adoc#qa-with-oe-selftest[relevant section of the README] for more details. 18* oe-selftest succeeds. To test meta-updater, run `oe-selftest -r updater` from a build directory with `MACHINE` set to `qemux86-64`. See the link:{aktualizr-docsroot}meta-updater-testing.adoc#qa-with-oe-selftest[relevant section of the README] for more details.
27* Updates are forwards- and backwards-compatible. You should be able to update an OTA-enabled build before the change is applied to the version with change applied and vice versa. One should pay double attention to the compatibility when bootloader code is affected. 19* Updates are forwards- and backwards-compatible. You should be able to update an OTA-enabled build before the change is applied to the version with change applied and vice versa. One should pay double attention to the compatibility when bootloader code is affected.
28* The patch/branch should be based on the latest version of the target branch. This may mean that rebasing is necessary if other PRs are merged before yours is approved. 20* The patch/branch should be based on the latest version of the target branch. This may mean that rebasing is necessary if other PRs are merged before yours is approved.
29 21
diff --git a/README.adoc b/README.adoc
index dd07425..12e0446 100644
--- a/README.adoc
+++ b/README.adoc
@@ -1,25 +1,19 @@
1= meta-updater 1= meta-updater
2:toc: macro 2:toc: macro
3:toc-title: 3:toc-title:
4:aktualizr-docsroot: https://github.com/advancedtelematic/aktualizr/tree/master/docs/ota-client-guide/modules/ROOT/pages/
4 5
5This layer enables over-the-air updates (OTA) with https://github.com/ostreedev/ostree[OSTree] and https://github.com/advancedtelematic/aktualizr[Aktualizr]. 6Meta-updater is a link:https://www.yoctoproject.org/software-overview/layers/[Yocto layer] that enables over-the-air updates (OTA) with https://github.com/ostreedev/ostree[OSTree] and https://github.com/advancedtelematic/aktualizr[Aktualizr] -- the default client for link:https://www.here.com/products/automotive/ota-technology[HERE OTA Connect].
6 7
7https://github.com/ostreedev/ostree[OSTree] is a tool for atomic full file system upgrades with rollback capability. OSTree has several advantages over traditional dual-bank systems, but the most important one is that it minimizes network bandwidth and data storage footprint by sharing files with the same contents across file system deployments. 8https://github.com/ostreedev/ostree[OSTree] is a tool for atomic full file system upgrades with rollback capability. OSTree has several advantages over traditional dual-bank systems, but the most important one is that it minimizes network bandwidth and data storage footprint by sharing files with the same contents across file system deployments.
8 9
9https://github.com/advancedtelematic/aktualizr[Aktualizr] (and https://github.com/advancedtelematic/rvi_sota_client[RVI SOTA client]) add authentication and provisioning capabilities to OTA and are integrated with OSTree. You can connect with these open-source applications or sign up for a free account at https://connect.ota.here.com/[HERE OTA Connect] to get started. 10https://github.com/advancedtelematic/aktualizr[Aktualizr] (and https://github.com/advancedtelematic/rvi_sota_client[RVI SOTA client]) add authentication and provisioning capabilities to OTA and are integrated with OSTree. You can connect with these open-source applications or sign up for a free account at https://connect.ota.here.com/[HERE OTA Connect] to get started.
10 11
11[discrete] 12== Quickstart
12== Table of Contents
13
14toc::[]
15
16== Build
17 13
18=== Quickstart 14If you don't already have a Yocto project that you want to add OTA to, you can use the xref:dev@getstarted::raspberry-pi.adoc[HERE OTA Connect Quickstart] project to rapidly get up and running on a Raspberry Pi. It takes a standard https://www.yoctoproject.org/tools-resources/projects/poky[poky] distribution, and adds OTA and OSTree capabilities.
19 15
20If you don't already have a Yocto project that you want to add OTA to, you can use the https://docs.atsgarage.com/quickstarts/raspberry-pi.html[HERE OTA Connect Quickstart] project to rapidly get up and running on a Raspberry Pi. It takes a standard https://www.yoctoproject.org/tools-resources/projects/poky[poky] distribution, and adds OTA and OSTree capabilities. 16== Dependencies
21
22=== Dependencies
23 17
24In addition to the link:https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#required-packages-for-the-build-host[standard Yocto dependencies], meta-updater generally requires a few additional dependencies, depending on your use case and target platform. To install these additional packages on Debian/Ubuntu, run this: 18In addition to the link:https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#required-packages-for-the-build-host[standard Yocto dependencies], meta-updater generally requires a few additional dependencies, depending on your use case and target platform. To install these additional packages on Debian/Ubuntu, run this:
25 19
@@ -33,268 +27,46 @@ To build for https://github.com/advancedtelematic/meta-updater-minnowboard[Minno
33sudo apt install ovmf 27sudo apt install ovmf
34.... 28....
35 29
36=== Adding meta-updater capabilities to your build 30[discrete]
37 31== Table of Contents
38If you already have a Yocto-based project and you want to add atomic filesystem updates to it, you just need to do three things:
39
401. Clone the `meta-updater` layer and add it to your https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#structure-build-conf-bblayers.conf[bblayers.conf].
412. Clone BSP integration layer (`meta-updater-$\{PLATFORM}`, e.g. https://github.com/advancedtelematic/meta-updater-raspberrypi[meta-updater-raspberrypi]) and add it to your `conf/bblayers.conf`. If your board isn't supported yet, you could write a BSP integration for it yourself. See the <<Adding support for your board>> section for the details.
423. Set up your https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DISTRO[distro]. If you are using "poky", the default distro in Yocto, you can change it in your `conf/local.conf` to "poky-sota". Alternatively, if you are using your own or third party distro configuration, you can add `INHERIT += " sota"` to it, thus combining capabilities of your distro with meta-updater features.
43
44You can then build your image as usual, with bitbake. After building the root file system, bitbake will then create an https://ostree.readthedocs.io/en/latest/manual/adapting-existing/[OSTree-enabled version] of it, commit it to your local OSTree repo and (optionally) push it to a remote server. Additionally, a live disk image will be created (normally named `$\{IMAGE_NAME}.-sdimg-ota` e.g. `core-image-raspberrypi3.rpi-sdimg-ota`). You can control this behaviour through <<sota-related-variables-in-localconf,variables in your local.conf>>.
45
46=== Build in AGL
47
48With AGL you can just add agl-sota feature while configuring your build environment:
49
50....
51source meta-agl/scripts/aglsetup.sh -m porter agl-demo agl-appfw-smack agl-devel agl-sota
52....
53
54You can then run:
55
56....
57bitbake agl-demo-platform
58....
59
60and get as a result an `ostree_repo` folder in your images directory (`tmp/deploy/images/$\{MACHINE}/ostree_repo`). It will contain:
61
62* your OSTree repository, with the rootfs committed as an OSTree deployment,
63* an `ota-ext4` bootstrap image, which is an OSTree physical sysroot as a burnable filesystem image, and optionally
64* some machine-dependent live images (e.g. `.wic` for Raspberry Pi or `.porter-sdimg-ota` for Renesas Porter board).
65
66Although `aglsetup.sh` hooks provide reasonable defaults for SOTA-related variables, you may want to tune some of them.
67
68=== Build problems
69
70Ubuntu users that encounter an error due to missing `Python.h` should install `libpython2.7-dev` on their host machine.
71
72== Supported boards
73
74Currently supported platforms are:
75
76* https://github.com/advancedtelematic/meta-updater-raspberrypi[Raspberry Pi 2 and 3]
77* https://github.com/advancedtelematic/meta-updater-minnowboard[Intel Minnowboard]
78* https://github.com/advancedtelematic/meta-updater-qemux86-64[Native QEMU emulation]
79* Renesas R-Car H3 and M3
80* https://github.com/advancedtelematic/meta-updater-ti/[TI BeagleBone Black] (rocko only, using TI SDK 05.03)
81* https://github.com/advancedtelematic/meta-updater-ti/[TI AM65x industrial development kit] (rocko only, using TI SDK 05.03)
82
83Additionally, there is community support for https://github.com/ricardosalveti/meta-updater-riscv[RISC-V] boards, in particular the Freedom U540.
84
85We also historically supported the https://github.com/advancedtelematic/meta-updater-porter[Renesas Porter] board.
86
87=== Adding support for your board
88
89If your board isn't supported yet, you can add board integration code yourself. The main purpose of this code is to provide a bootloader that will be able to use https://ostree.readthedocs.io/en/latest/manual/atomic-upgrades/[OSTree's boot directory]. In the meta-updater integration layers we have written so far, the basic steps are:
90
911. Make the board boot into http://www.denx.de/wiki/U-Boot[U-Boot]
922. Make U-boot import variables from /boot/loader/uEnv.txt and load the kernel with initramfs and kernel command line arguments according to what is set in this file.
93
94You may take a look into https://github.com/advancedtelematic/meta-updater-minnowboard[Minnowboard] or https://github.com/advancedtelematic/meta-updater-raspberrypi[Raspberry Pi] integration layers for examples.
95
96Although we have focused on U-Boot and GRUB so far, other bootloaders can be configured to work with OSTree as well.
97
98Your images will also need network connectivity to be able to reach an actual OTA backend. Our 'poky-sota' distribution does not mandate or install a default network manager but our supported platforms use the `virtual/network-configuration` recipe, which can be used as a starting example.
99
100== SOTA-related variables in local.conf
101
102* `OSTREE_BRANCHNAME` - OSTree branch name. Defaults to `${SOTA_HARDWARE_ID}`. Particularly useful for grouping similar images.
103* `OSTREE_REPO` - path to your OSTree repository. Defaults to `$\{DEPLOY_DIR_IMAGE}/ostree_repo`
104* `OSTREE_OSNAME` - OS deployment name on your target device. For more information about deployments and osnames see the https://ostree.readthedocs.io/en/latest/manual/deployment/[OSTree documentation]. Defaults to "poky".
105* `OSTREE_COMMIT_BODY` - Message attached to OSTree commit. Empty by default.
106* `OSTREE_COMMIT_SUBJECT` - Commit subject used by OSTree. Defaults to `Commit-id: ${IMAGE_NAME}`
107* `OSTREE_UPDATE_SUMMARY` - Set this to '1' to update summary of OSTree repository on each commit. '0' by default.
108* `OSTREE_DEPLOY_DEVICETREE` - Set this to '1' to include devicetree(s) to boot
109* `GARAGE_SIGN_AUTOVERSION` - Set this to '1' to automatically fetch the last version of the garage tools installed by the aktualizr-native. Otherwise use the fixed version specified in the recipe.
110* `INITRAMFS_IMAGE` - initramfs/initrd image that is used as a proxy while booting into OSTree deployment. Do not change this setting unless you are sure that your initramfs can serve as such a proxy.
111* `SOTA_PACKED_CREDENTIALS` - when set, your ostree commit will be pushed to a remote repo as a bitbake step. This should be the path to a zipped credentials file in https://github.com/advancedtelematic/aktualizr/blob/master/docs/credentials.adoc[the format accepted by garage-push].
112* `SOTA_DEPLOY_CREDENTIALS` - when set to '1' (default value), deploys credentials to the built image. Override it in `local.conf` to built a generic image that can be provisioned manually after the build.
113* `SOTA_CLIENT_PROV` - which provisioning method to use. Valid options are `aktualizr-shared-prov`, `aktualizr-device-prov`, and `aktualizr-device-prov-hsm`. For more information on these provisioning methods, see the https://docs.ota.here.com/client-config/client-provisioning-methods.html[OTA Connect documentation]. The default is `aktualizr-shared-prov`. This can also be set to an empty string to avoid using a provisioning recipe.
114* `SOTA_CLIENT_FEATURES` - extensions to aktualizr. The only valid options are `hsm` (to build with HSM support) and `secondary-network` (to set up a simulated 'in-vehicle' network with support for a primary node with a DHCP server and a secondary node with a DHCP client).
115* `SOTA_SECONDARY_CONFIG` - a file containing JSON configuration for secondaries. It will be installed into `/etc/sota/ecus` on the device and automatically provided to aktualizr. See link:https://github.com/advancedtelematic/aktualizr/blob/master/docs/posix-secondaries-bitbaking.adoc[here] for more details.
116* `SOTA_HARDWARE_ID` - a custom hardware ID that will be written to the aktualizr config. Defaults to MACHINE if not set.
117* `SOTA_MAIN_DTB` - base device tree to use with the kernel. Used together with FIT images. You can change it, and the device tree will also be changed after the update.
118* `SOTA_DT_OVERLAYS` - whitespace-separated list of used device tree overlays for FIT image. This list is OSTree-updateable as well.
119* `SOTA_EXTRA_CONF_FRAGS` - extra https://lxr.missinglinkelectronics.com/uboot/doc/uImage.FIT/overlay-fdt-boot.txt[configuration fragments] for FIT image.
120* `RESOURCE_xxx_pn-aktualizr` - controls maximum resource usage of the aktualizr service, when `aktualizr-resource-control` is installed on the image. See <<aktualizr service resource control>> for details.
121* `SOTA_POLLING_SEC` - sets polling interval for aktualizr to check for updates if aktualizr-polling-interval is included in the image.
122
123== Usage
124
125=== OSTree
126
127OSTree used to include a simple HTTP server as part of the ostree binary, but this has been removed in more recent versions. However, OSTree repositories are self-contained directories, and can be trivially served over the network using any HTTP server. For example, you could use Python's SimpleHTTPServer:
128
129....
130cd tmp/deploy/images/qemux86-64/ostree_repo
131python -m SimpleHTTPServer <port> # port defaults to 8000
132....
133
134You can then run ostree from inside your device by adding your repo:
135
136....
137# This behaves like adding a Git remote; you can name it anything
138ostree remote add --no-gpg-verify my-remote http://<your-ip>:<port>
139
140# If OSTREE_BRANCHNAME is set in local.conf, that will be the name of the
141# branch. If not set, it defaults to the value of MACHINE (e.g. qemux86-64).
142ostree pull my-remote <branch>
143
144# poky is the OS name as set in OSTREE_OSNAME
145ostree admin deploy --os=poky my-remote:<branch>
146....
147
148After restarting, you will boot into the newly deployed OS image.
149
150For example, on the raspberry pi you can try this sequence:
151
152....
153# add remote
154ostree remote add --no-gpg-verify agl-snapshot https://download.automotivelinux.org/AGL/snapshots/master/latest/raspberrypi3/deploy/images/raspberrypi3/ostree_repo/ agl-ota
155
156# pull
157ostree pull agl-snapshot agl-ota
158
159# deploy
160ostree admin deploy --os=agl agl-snapshot:agl-ota
161....
162
163=== garage-push
164
165The https://github.com/advancedtelematic/aktualizr[aktualizr repo] contains a tool, garage-push, which lets you push the changes in OSTree repository generated by bitbake process. It communicates with an http server capable of querying files with HEAD requests and uploading them with POST requests. In particular, this can be used with https://connect.ota.here.com/[HERE OTA Connect]. garage-push is used as follows:
166
167....
168garage-push --repo=/path/to/ostree-repo --ref=mybranch --credentials=/path/to/credentials.zip
169....
170
171You can set `SOTA_PACKED_CREDENTIALS` in your `local.conf` to automatically synchronize your build results with a remote server. Credentials are stored in an archive as described in the https://github.com/advancedtelematic/aktualizr/blob/master/docs/credentials.adoc[aktualizr documentation].
172
173=== aktualizr configuration
174
175https://github.com/advancedtelematic/aktualizr[Aktualizr] supports a variety of https://github.com/advancedtelematic/aktualizr/blob/master/docs/configuration.adoc[configuration options via a configuration file and the command line]. There are two primary ways to control aktualizr's configuration from meta-updater.
176
177First, you can set `SOTA_CLIENT_PROV` to control which provisioning recipe is used. Each recipe installs an appropriate `sota.toml` file from aktualizr according to the provisioning needs. See the <<sota-related-variables-in-localconf,SOTA-related variables in local.conf>> section for more information.
178
179Second, you can write recipes to install additional config files with customized options. A few recipes already exist to address common needs and provide an example:
180
181* link:recipes-sota/config/aktualizr-auto-reboot.bb[aktualizr-auto-reboot.bb] configures aktualizr to automatically reboot after new updates are installed in order to apply the updates immediately. This is only relevant for package managers (such as OSTree) that require a reboot to complete the installation process. If this is not enabled, you will need to reboot the system through other means.
182* link:recipes-sota/config/aktualizr-disable-send-ip.bb[aktualizr-disable-send-ip.bb] disables the reporting of networking information to the server. This is enabled by default and supported by https://connect.ota.here.com/[HERE OTA Connect]. However, if you are using a different server that does not support this feature, you may want to disable it in aktualizr.
183* link:recipes-sota/config/aktualizr-log-debug.bb[aktualizr-log-debug.bb] sets the log level of aktualizr to 0 (trace). The default is 2 (info). This recipe is intended for development and debugging purposes.
184
185To use these recipes, you will need to add them to your image with a line such as `IMAGE_INSTALL_append = " aktualizr-log-debug "` in your `local.conf`.
186
187=== aktualizr service resource control
188
189With systemd based images, it is possible to set resource policies for the aktualizr service. The main use case is to provide a safeguard against resource exhaustion during an unforeseen failure scenario.
190
191To enable it, install `aktualizr-resource-control` on the target image and optionally override the default resource limits set in link:recipes-sota/aktualizr/aktualizr_git.bb[aktualizr_git.bb], from your `local.conf`.
192
193For example:
194
195....
196IMAGE_INSTALL_append += " aktualizr-resource-control "
197RESOURCE_CPU_WEIGHT_pn-aktualizr = "50"
198....
199
200== Development configuration
201
202=== Logging
203
204To troubleshoot problems that you might encounter during development, we recommend that you enable persistent `systemd` logging. This setting is enabled by default for newly configured environments (see link:conf/local.conf.sample.append[]). To enable it manually, put this to your `local.conf`:
205
206....
207IMAGE_INSTALL_append += " systemd-journald-persistent"
208....
209
210It may also be helpful to run with debug logging enabled in aktualizr. To do so, add this to your `local.conf`:
211
212....
213IMAGE_INSTALL_append += " aktualizr-log-debug"
214....
215
216=== Custom aktualizr versions
217
218You can override the version of aktualizr included in your image. This requires that the version you wish to run is pushed to the https://github.com/advancedtelematic/aktualizr[aktualizr github repo]. You can then use these settings in your `local.conf` to simplify the development process:
219
220[options="header"]
221|======================
222| Option | Effect
223| `require classes/sota_bleeding.inc` | Build the latest head (by default, using the master branch) of Aktualizr
224| `BRANCH_pn-aktualizr = "mybranch"`
225
226`BRANCH_pn-aktualizr-native = "mybranch"` | Build `mybranch` of Aktualizr. Note that both of these need to be set. This is normally used in conjunction with `require classes/sota_bleeding.inc`
227| `SRCREV_pn-aktualizr = "1004efa3f86cef90c012b34620992b5762b741e3"`
228
229`SRCREV_pn-aktualizr-native = "1004efa3f86cef90c012b34620992b5762b741e3"` | Build the specified revision of Aktualizr. Note that both of these need to be set. This can be used in conjunction with `BRANCH_pn-aktualizr` and `BRANCH_pn-aktualizr-native` but will conflict with `require classes/sota_bleeding.inc`
230| `TOOLCHAIN_HOST_TASK_append = " nativesdk-cmake "` | Use with `bitbake -c populate_sdk core-image-minimal` to build an SDK. See the https://github.com/advancedtelematic/aktualizr#developing-against-an-openembedded-system[aktualizr repo] for more information.
231|======================
232
233=== Overriding target version
234*Warning: overriding target version is a dangerous operation, make sure you understand this section completely before doing it.*
235
236Every time you build an image with `SOTA_PACKED_CREDENTIALS` set, a new entry in your Uptane metadata is created and you can see it in the OTA Garage UI if you're using one. Normally this version will be equal to OSTree hash of your root file system. If you want it to be different though you can override is using one of two methods:
237
2381. Set `GARAGE_TARGET_VERSION` variable in your `local.conf`.
2392. Write a recipe or a bbclass to write the desired version to `${STAGING_DATADIR_NATIVE}/target_version`. An example of such bbclass can be found in `classes/target_version_example.bbclass`.
240
241Please note that [target name, target version] pairs are expected to be unique in the system. If you build a new target with the same target version as a previously built one, the old package will be overwritten on the update server. It can have unpredictable effect on devices that have this version installed, and it is not guaranteed that information will be reported correctly for such devices or that you will be able to update them (we're doing our best though). The easiest way to avoid problems is to make sure that your overriding version is as unique as an OSTree commit hash.
242
243== QA with oe-selftest
244 32
245This layer relies on the test framework oe-selftest for quality assurance. Currently, you will need to run this in a build directory with `MACHINE` set to `qemux86-64`. Follow the steps below to run the tests: 33The following documentation focuses on tasks that involve the meta-updater layer. If you want to get an idea of the overall developer workflow in OTA Connect, see the link:https://docs.ota.here.com/ota-client/dev/index.html[OTA Connect Developer Guide].
34[NOTE]
35====
36The following links point to files in the aktualizr repository where the source of the developer guide is stored.
37====
246 38
2471. Append the line below to `conf/local.conf` to disable the warning about supported operating systems: 39* xref:{aktualizr-docsroot}meta-updater-build.adoc[Build]
248+ 40+
249``` 41Learn how to use this layer to build a basic disk image and add it to your own Yocto project.
250SANITY_TESTED_DISTROS = ""
251```
252
2532. If your image does not already include an ssh daemon such as dropbear or openssh, add this line to `conf/local.conf` as well:
254+ 42+
255``` 43* xref:{aktualizr-docsroot}supported-boards.adoc[Supported boards]
256IMAGE_INSTALL_append = " dropbear "
257```
258
2593. Some tests require that `SOTA_PACKED_CREDENTIALS` is set in your `conf/local.conf`. See the <<sota-related-variables-in-localconf,SOTA-related variables in local.conf>> section.
260
2614. To be able to build an image for the GRUB tests, you will need to install the ovmf package as described in the <<Dependencies,dependencies>>.
262
2635. Run oe-selftest:
264+ 44+
265``` 45Find out if your board is supported and learn about the minimum hardware requirements.
266oe-selftest -r updater_native updater_qemux86_64 updater_minnowboard updater_raspberrypi updater_qemux86_64_ptest 46+
267``` 47* xref:{aktualizr-docsroot}build-configuration.adoc[SOTA-related variables in local.conf]
268 48+
269For more information about oe-selftest, including details about how to run individual test modules or classes, please refer to the https://wiki.yoctoproject.org/wiki/Oe-selftest[Yocto Project wiki]. 49Learn how to configure OTA-related functionality when building disk images.
270 50+
271== Aktualizr test suite with ptest 51* xref:{aktualizr-docsroot}meta-updater-usage.adoc[Usage]
272 52+
273The meta-updater layer includes support for running parts of the aktualizr test suite on deployed devices through link:https://wiki.yoctoproject.org/wiki/Ptest[Yocto's ptest functionality]. Since it adds significant build time cost, it is currently disabled by default. To enable it, add the following to your `conf/local.conf`: 53Learn about the `garage-push` and `garage-sign` utilities, aktualizr configuration and service resource control, and OSTree.
274 54+
275``` 55* xref:{aktualizr-docsroot}meta-updater-dev-config.adoc[Development configuration]
276PTEST_ENABLED_pn-aktualizr = "1" 56+
277IMAGE_INSTALL_append += " aktualizr-ptest ptest-runner " 57Learn how to configure logging, install custom versions of aktualizr, and override the version indicator for sofware updates.
278``` 58+
279 59* xref:{aktualizr-docsroot}meta-updater-testing.adoc#_qa_with_oe_selftest[QA with oe-selftest]
280Be aware that it will add several hundreds of MB to the generated file system. 60+
281 61Learn how to use the `oe-selftest` framework for quality assurance.
282The aktualizr tests will now be part of the deployed ptest suite, which can be run by calling `ptest-runner`. Alternatively, the required files and run script can be found in `/usr/lib/aktualizr/ptest`. 62+
283 63* xref:{aktualizr-docsroot}meta-updater-testing.adoc#_aktualizr_test_suite_with_ptest[Aktualizr test suite with ptest]
284== Manual provisoning 64+
285 65Learn how to enable Yocto's package test functionality and run parts of the aktualizr test suite.
286As described in <<sota-related-variables-in-localconf,SOTA-related variables in local.conf>> section you can set `SOTA_DEPLOY_CREDENTIALS` to `0` to prevent deploying credentials to the built `wic` image. In this case you get a generic image that you can use e.g. on a production line to flash a series of devices. The cost of this approach is that this image is half-baked and should be provisioned before it can connect to the backend. 66+
287 67* xref:{aktualizr-docsroot}meta-updater-provisioning-methods.adoc[Provisoning methods]
288Provisioning procedure depends on your provisioning recipe, i.e. the value of `SOTA_CLIENT_PROV` (equal to `aktualizr-shared-prov` by default): 68+
289 69Learn how to enable different methods for provisioning devices.
290* For `aktualizr-shared-prov` put your `credentials.zip` to `/var/sota/sota_provisioning_credentials.zip` on the filesystem of a running device. If you have the filesystem of our device mounted to your build machine, prefix all paths with `/ostree/deploy/poky` as in `/ostree/deploy/poky/var/sota/sota_provisioning_credentials.zip`.
291* For `aktualizr-device-prov`
292** put URL to the backend server (together with protocol prefix and port number) at `/var/sota/gateway.url`. If you're using HERE OTA Connect, you can find the URL in the `autoprov.url` file in your credentials archive.
293** put client certificate, private key and root CA certificate (for the *server*, not for the *device*) at `/var/sota/import/client.pem`, `/var/sota/import/pkey.pem` and `/var/sota/import/root.crt` respectively.
294* For `aktualizr-device-prov-hsm`
295** put URL to the server backend (together with protocol prefix and port number) at `/var/sota/gateway.url`. If you're using HERE OTA Connect, you can find the URL in the `autoprov.url` file in your credentials archive.
296** put root CA certificate (for the *server*, not for the *device*) at `/var/sota/import/root.crt`.
297** put client certificate and private key to slots 1 and 2 of the PKCS#11-compatible device.
298 70
299== License 71== License
300 72
diff --git a/classes/image_types_ostree.bbclass b/classes/image_types_ostree.bbclass
index 2e8e8f5..795e01b 100644
--- a/classes/image_types_ostree.bbclass
+++ b/classes/image_types_ostree.bbclass
@@ -237,10 +237,20 @@ IMAGE_CMD_garagesign () {
237 # Push may fail due to race condition when multiple build machines try to push simultaneously 237 # Push may fail due to race condition when multiple build machines try to push simultaneously
238 # in which case targets.json should be pulled again and the whole procedure repeated 238 # in which case targets.json should be pulled again and the whole procedure repeated
239 push_success=0 239 push_success=0
240 target_url="" 240 target_url=""
241 if [ -n "${GARAGE_TARGET_URL}" ]; then 241 if [ -n "${GARAGE_TARGET_URL}" ]; then
242 target_url='--url ${GARAGE_TARGET_URL}' 242 target_url="--url ${GARAGE_TARGET_URL}"
243 fi 243 fi
244 target_expiry=""
245 if [ -n "${GARAGE_TARGET_EXPIRES}" ] && [ -n "${GARAGE_TARGET_EXPIRE_AFTER}" ]; then
246 bbfatal "Both GARAGE_TARGET_EXPIRES and GARAGE_TARGET_EXPIRE_AFTER are set. Only one can be set at a time."
247 elif [ -n "${GARAGE_TARGET_EXPIRES}" ]; then
248 target_expiry="--expires ${GARAGE_TARGET_EXPIRES}"
249 elif [ -n "${GARAGE_TARGET_EXPIRE_AFTER}" ]; then
250 target_expiry="--expire-after ${GARAGE_TARGET_EXPIRE_AFTER}"
251 else
252 target_expiry="--expire-after 1M"
253 fi
244 254
245 for push_retries in $( seq 3 ); do 255 for push_retries in $( seq 3 ); do
246 garage-sign targets pull --repo tufrepo \ 256 garage-sign targets pull --repo tufrepo \
@@ -262,6 +272,7 @@ IMAGE_CMD_garagesign () {
262 fi 272 fi
263 garage-sign targets sign --repo tufrepo \ 273 garage-sign targets sign --repo tufrepo \
264 --home-dir ${GARAGE_SIGN_REPO} \ 274 --home-dir ${GARAGE_SIGN_REPO} \
275 ${target_expiry} \
265 --key-name=targets 276 --key-name=targets
266 errcode=0 277 errcode=0
267 garage-sign targets push --repo tufrepo \ 278 garage-sign targets push --repo tufrepo \
diff --git a/classes/sota.bbclass b/classes/sota.bbclass
index 5506428..5620b76 100644
--- a/classes/sota.bbclass
+++ b/classes/sota.bbclass
@@ -51,6 +51,8 @@ GARAGE_SIGN_KEYNAME ?= "garage-key"
51GARAGE_TARGET_NAME ?= "${OSTREE_BRANCHNAME}" 51GARAGE_TARGET_NAME ?= "${OSTREE_BRANCHNAME}"
52GARAGE_TARGET_VERSION ?= "" 52GARAGE_TARGET_VERSION ?= ""
53GARAGE_TARGET_URL ?= "" 53GARAGE_TARGET_URL ?= ""
54GARAGE_TARGET_EXPIRES ?= ""
55GARAGE_TARGET_EXPIRE_AFTER ?= ""
54GARAGE_CUSTOMIZE_TARGET ?= "" 56GARAGE_CUSTOMIZE_TARGET ?= ""
55 57
56SOTA_MACHINE ??="none" 58SOTA_MACHINE ??="none"
diff --git a/lib/oeqa/selftest/cases/updater_qemux86_64.py b/lib/oeqa/selftest/cases/updater_qemux86_64.py
index 24abcff..4506300 100644
--- a/lib/oeqa/selftest/cases/updater_qemux86_64.py
+++ b/lib/oeqa/selftest/cases/updater_qemux86_64.py
@@ -360,17 +360,7 @@ class IpSecondaryTests(OESelftestTestCase):
360 self._test_ctx.append_config('SOTA_CLIENT_PROV = " aktualizr-shared-prov "') 360 self._test_ctx.append_config('SOTA_CLIENT_PROV = " aktualizr-shared-prov "')
361 361
362 def is_ecu_registered(self, ecu_id): 362 def is_ecu_registered(self, ecu_id):
363 max_number_of_tries = 120 363 device_status = self.get_info()
364 try_counter = 0
365
366 # aktualizr-info is not always able to load ECU serials from DB
367 # so, let's run it a few times until it actually succeeds
368 while try_counter < max_number_of_tries:
369 device_status = self.get_info()
370 try_counter += 1
371 if device_status.find("load ECU serials") == -1:
372 break
373 sleep(1)
374 364
375 if not ((device_status.find(ecu_id[0]) != -1) and (device_status.find(ecu_id[1]) != -1)): 365 if not ((device_status.find(ecu_id[0]) != -1) and (device_status.find(ecu_id[1]) != -1)):
376 return False 366 return False
@@ -379,7 +369,7 @@ class IpSecondaryTests(OESelftestTestCase):
379 return not_reg_start == -1 or (device_status.find(ecu_id[1], not_reg_start) == -1) 369 return not_reg_start == -1 or (device_status.find(ecu_id[1], not_reg_start) == -1)
380 370
381 def get_info(self): 371 def get_info(self):
382 stdout, stderr, retcode = self.send_command('aktualizr-info') 372 stdout, stderr, retcode = self.send_command('aktualizr-info --wait-until-provisioned', timeout=620)
383 self._test_ctx.assertEqual(retcode, 0, 'Unable to run aktualizr-info: {}'.format(stderr)) 373 self._test_ctx.assertEqual(retcode, 0, 'Unable to run aktualizr-info: {}'.format(stderr))
384 return stdout 374 return stdout
385 375
diff --git a/recipes-core/images/initramfs-ostree-image.bb b/recipes-core/images/initramfs-ostree-image.bb
index b21be1c..7e03f3d 100644
--- a/recipes-core/images/initramfs-ostree-image.bb
+++ b/recipes-core/images/initramfs-ostree-image.bb
@@ -29,4 +29,4 @@ IMAGE_OVERHEAD_FACTOR = "1.0"
29 29
30BAD_RECOMMENDATIONS += "busybox-syslog" 30BAD_RECOMMENDATIONS += "busybox-syslog"
31 31
32 32IMAGE_PREPROCESS_COMMAND_remove = "buildinfo_manifest;"
diff --git a/recipes-sota/aktualizr/aktualizr_git.bb b/recipes-sota/aktualizr/aktualizr_git.bb
index 9adfce8..17aa15a 100644
--- a/recipes-sota/aktualizr/aktualizr_git.bb
+++ b/recipes-sota/aktualizr/aktualizr_git.bb
@@ -8,14 +8,14 @@ LIC_FILES_CHKSUM = "file://${S}/LICENSE;md5=9741c346eef56131163e13b9db1241b3"
8DEPENDS = "boost curl openssl libarchive libsodium sqlite3 asn1c-native" 8DEPENDS = "boost curl openssl libarchive libsodium sqlite3 asn1c-native"
9DEPENDS_append = "${@bb.utils.contains('PTEST_ENABLED', '1', ' coreutils-native net-tools-native ostree-native aktualizr-native ', '', d)}" 9DEPENDS_append = "${@bb.utils.contains('PTEST_ENABLED', '1', ' coreutils-native net-tools-native ostree-native aktualizr-native ', '', d)}"
10RDEPENDS_${PN}_class-target = "aktualizr-configs lshw" 10RDEPENDS_${PN}_class-target = "aktualizr-configs lshw"
11RDEPENDS_${PN}-host-tools = "aktualizr aktualizr-repo aktualizr-cert-provider ${@bb.utils.contains('PACKAGECONFIG', 'sota-tools', 'garage-deploy garage-push', '', d)}" 11RDEPENDS_${PN}-host-tools = "aktualizr aktualizr-cert-provider ${@bb.utils.contains('PACKAGECONFIG', 'sota-tools', 'garage-deploy garage-push', '', d)}"
12 12
13RDEPENDS_${PN}-ptest += "bash cmake curl python3-misc python3-modules openssl-bin sqlite3 valgrind" 13RDEPENDS_${PN}-ptest += "bash cmake curl python3-misc python3-modules openssl-bin sqlite3 valgrind"
14 14
15PV = "1.0+git${SRCPV}" 15PV = "1.0+git${SRCPV}"
16PR = "7" 16PR = "7"
17 17
18GARAGE_SIGN_PV = "0.7.0-19-g89ec974" 18GARAGE_SIGN_PV = "0.7.0-33-g214dfb1"
19 19
20SRC_URI = " \ 20SRC_URI = " \
21 gitsm://github.com/advancedtelematic/aktualizr;branch=${BRANCH} \ 21 gitsm://github.com/advancedtelematic/aktualizr;branch=${BRANCH} \
@@ -28,10 +28,10 @@ SRC_URI = " \
28 " 28 "
29 29
30# for garage-sign archive 30# for garage-sign archive
31SRC_URI[md5sum] = "3ed27c1142860cd9b4a2594067312b8b" 31SRC_URI[md5sum] = "66ffe8dcd61d4c15646e1c4b7dde7401"
32SRC_URI[sha256sum] = "e54eef3863118f373c3ebd9e2877f9de5bab4950ed157a15fb4f4ec575bc2ece" 32SRC_URI[sha256sum] = "7a7193ddf7e1a33ea60fbb20f98318a8bd78c325dab391d8c4ebd644a738abdc"
33 33
34SRCREV = "03778511cc937d07bf53a8092f8b268e65f5d9a6" 34SRCREV = "3bb9fe91b4c614a79373beadc721272fcf7acce2"
35BRANCH ?= "master" 35BRANCH ?= "master"
36 36
37S = "${WORKDIR}/git" 37S = "${WORKDIR}/git"
diff --git a/scripts/qemucommand.py b/scripts/qemucommand.py
index a869d4d..9b23c54 100644
--- a/scripts/qemucommand.py
+++ b/scripts/qemucommand.py
@@ -2,6 +2,7 @@ from os.path import exists, isdir, join, realpath, abspath
2from os import listdir 2from os import listdir
3import random 3import random
4import socket 4import socket
5from shutil import copyfile
5from subprocess import check_output 6from subprocess import check_output
6 7
7EXTENSIONS = { 8EXTENSIONS = {
@@ -39,6 +40,8 @@ def random_mac():
39 40
40class QemuCommand(object): 41class QemuCommand(object):
41 def __init__(self, args): 42 def __init__(self, args):
43 self.dry_run = args.dry_run
44 self.overlay = args.overlay
42 if args.machine: 45 if args.machine:
43 self.machine = args.machine 46 self.machine = args.machine
44 else: 47 else:
@@ -49,21 +52,53 @@ class QemuCommand(object):
49 self.machine = machines[0] 52 self.machine = machines[0]
50 else: 53 else:
51 raise ValueError("Could not autodetect machine type. More than one entry in %s. Maybe --machine qemux86-64?" % args.dir) 54 raise ValueError("Could not autodetect machine type. More than one entry in %s. Maybe --machine qemux86-64?" % args.dir)
55
56 # If using an overlay with U-Boot, copy the rom when we create the
57 # overlay so that we can keep it around just in case.
52 if args.efi: 58 if args.efi:
53 self.bios = 'OVMF.fd' 59 self.bios = 'OVMF.fd'
54 else: 60 else:
55 uboot = abspath(join(args.dir, self.machine, 'u-boot-qemux86-64.rom')) 61 uboot_path = abspath(join(args.dir, self.machine, 'u-boot-qemux86-64.rom'))
56 if not exists(uboot): 62 if self.overlay:
57 raise ValueError("U-Boot image %s does not exist" % uboot) 63 new_uboot_path = self.overlay + '.u-boot.rom'
58 self.bios = uboot 64 if not exists(self.overlay):
65 if not exists(uboot_path):
66 raise ValueError("U-Boot image %s does not exist" % uboot_path)
67 if not exists(new_uboot_path):
68 if self.dry_run:
69 print("cp %s %s" % (uboot_path, new_uboot_path))
70 else:
71 copyfile(uboot_path, new_uboot_path)
72 uboot_path = new_uboot_path
73 if not exists(uboot_path) and not (self.dry_run and not exists(self.overlay)):
74 raise ValueError("U-Boot image %s does not exist" % uboot_path)
75 self.bios = uboot_path
76
77 # If using an overlay, we need to keep the "backing" image around, as
78 # bitbake will often clean it up, and the overlay silently depends on
79 # the hardcoded path. The easiest solution is to keep the file and use
80 # a relative path to it.
59 if exists(args.imagename): 81 if exists(args.imagename):
60 image = args.imagename 82 image = realpath(args.imagename)
61 else: 83 else:
62 ext = EXTENSIONS.get(self.machine, 'wic') 84 ext = EXTENSIONS.get(self.machine, 'wic')
63 image = join(args.dir, self.machine, '%s-%s.%s' % (args.imagename, self.machine, ext)) 85 image = join(args.dir, self.machine, '%s-%s.%s' % (args.imagename, self.machine, ext))
64 self.image = realpath(image) 86 if self.overlay:
65 if not exists(self.image): 87 new_image_path = self.overlay + '.img'
88 if not exists(self.overlay):
89 if not exists(image):
90 raise ValueError("OS image %s does not exist" % image)
91 if not exists(new_image_path):
92 if self.dry_run:
93 print("cp %s %s" % (image, new_image_path))
94 else:
95 copyfile(image, new_image_path)
96 self.image = new_image_path
97 else:
98 self.image = realpath(image)
99 if not exists(self.image) and not (self.dry_run and not exists(self.overlay)):
66 raise ValueError("OS image %s does not exist" % self.image) 100 raise ValueError("OS image %s does not exist" % self.image)
101
67 if args.mac: 102 if args.mac:
68 self.mac_address = args.mac 103 self.mac_address = args.mac
69 else: 104 else:
@@ -86,7 +121,6 @@ class QemuCommand(object):
86 self.gui = not args.no_gui 121 self.gui = not args.no_gui
87 self.gdb = args.gdb 122 self.gdb = args.gdb
88 self.pcap = args.pcap 123 self.pcap = args.pcap
89 self.overlay = args.overlay
90 self.secondary_network = args.secondary_network 124 self.secondary_network = args.secondary_network
91 125
92 def command_line(self): 126 def command_line(self):
diff --git a/scripts/run-qemu-ota b/scripts/run-qemu-ota
index de63297..232ee11 100755
--- a/scripts/run-qemu-ota
+++ b/scripts/run-qemu-ota
@@ -2,7 +2,7 @@
2 2
3from argparse import ArgumentParser 3from argparse import ArgumentParser
4from subprocess import Popen 4from subprocess import Popen
5from os.path import exists 5from os.path import exists, dirname
6import sys 6import sys
7from qemucommand import QemuCommand 7from qemucommand import QemuCommand
8 8
@@ -39,27 +39,37 @@ def main():
39 'This can be used to test Uptane Primary/Secondary communication.') 39 'This can be used to test Uptane Primary/Secondary communication.')
40 parser.add_argument('-n', '--dry-run', help='Print qemu command line rather then run it', action='store_true') 40 parser.add_argument('-n', '--dry-run', help='Print qemu command line rather then run it', action='store_true')
41 args = parser.parse_args() 41 args = parser.parse_args()
42
43 if args.overlay and not exists(args.overlay) and dirname(args.overlay) and not dirname(args.overlay) == '.':
44 print('Error: please provide a file name in the current working directory. ' +
45 'Overlays do not work properly with other directories.')
46 sys.exit(1)
47 if args.overlay and exists(args.overlay) and args.imagename != parser.get_default('imagename'):
48 # qemu-img amend -o <filename> might work, but it has not yet been done
49 # successfully.
50 print('Warning: cannot change backing image of overlay after it has been created.')
51
42 try: 52 try:
43 qemu_command = QemuCommand(args) 53 qemu_command = QemuCommand(args)
44 except ValueError as e: 54 except ValueError as e:
45 print(e.message) 55 print(e.message)
46 sys.exit(1) 56 sys.exit(1)
47 57
48 print("Launching %s with mac address %s" % (args.imagename, qemu_command.mac_address))
49 print("To connect via SSH:")
50 print(" ssh -o StrictHostKeyChecking=no root@localhost -p %d" % qemu_command.ssh_port)
51 print("To connect to the serial console:")
52 print(" nc localhost %d" % qemu_command.serial_port)
53
54 cmdline = qemu_command.command_line() 58 cmdline = qemu_command.command_line()
55 if args.overlay and not exists(args.overlay): 59 if args.overlay and not exists(args.overlay):
56 print("Image file %s does not yet exist, creating." % args.overlay) 60 print("Overlay file %s does not yet exist, creating." % args.overlay)
57 img_cmdline = qemu_command.img_command_line() 61 img_cmdline = qemu_command.img_command_line()
58 if args.dry_run: 62 if args.dry_run:
59 print(" ".join(img_cmdline)) 63 print(" ".join(img_cmdline))
60 else: 64 else:
61 Popen(img_cmdline).wait() 65 Popen(img_cmdline).wait()
62 66
67 print("Launching %s with mac address %s" % (args.imagename, qemu_command.mac_address))
68 print("To connect via SSH:")
69 print(" ssh -o StrictHostKeyChecking=no root@localhost -p %d" % qemu_command.ssh_port)
70 print("To connect to the serial console:")
71 print(" nc localhost %d" % qemu_command.serial_port)
72
63 if args.dry_run: 73 if args.dry_run:
64 print(" ".join(cmdline)) 74 print(" ".join(cmdline))
65 else: 75 else: