diff options
| -rw-r--r-- | docs/00-INDEX | 3 | ||||
| -rw-r--r-- | docs/container-bundling.md | 339 |
2 files changed, 342 insertions, 0 deletions
diff --git a/docs/00-INDEX b/docs/00-INDEX index 6659fbee..dfcc8bda 100644 --- a/docs/00-INDEX +++ b/docs/00-INDEX | |||
| @@ -8,6 +8,9 @@ alphabetical order as well. | |||
| 8 | 00-README | 8 | 00-README |
| 9 | - info on the goals of meta-virtualization and this docs subdir | 9 | - info on the goals of meta-virtualization and this docs subdir |
| 10 | 10 | ||
| 11 | container-bundling.md | ||
| 12 | - container bundling, cross-install, and vdkr/vpdmn documentation. | ||
| 13 | |||
| 11 | openvswitch.txt | 14 | openvswitch.txt |
| 12 | - example on how to setup openvswitch with qemu/kvm. | 15 | - example on how to setup openvswitch with qemu/kvm. |
| 13 | 16 | ||
diff --git a/docs/container-bundling.md b/docs/container-bundling.md new file mode 100644 index 00000000..ee2b5e31 --- /dev/null +++ b/docs/container-bundling.md | |||
| @@ -0,0 +1,339 @@ | |||
| 1 | Container Bundling and Cross-Architecture Deployment | ||
| 2 | ===================================================== | ||
| 3 | |||
| 4 | This document describes how to bundle containers into Yocto images at build | ||
| 5 | time using the container-cross-install system. This enables deploying Docker | ||
| 6 | and Podman containers from x86_64 build systems to ARM64/x86_64 targets | ||
| 7 | without requiring container runtimes on the build host. | ||
| 8 | |||
| 9 | Prerequisites | ||
| 10 | ------------- | ||
| 11 | |||
| 12 | Enable the vcontainer distro feature in local.conf: | ||
| 13 | |||
| 14 | DISTRO_FEATURES:append = " virtualization vcontainer" | ||
| 15 | |||
| 16 | Enable multiconfig for blob builds: | ||
| 17 | |||
| 18 | BBMULTICONFIG = "vruntime-aarch64 vruntime-x86-64" | ||
| 19 | |||
| 20 | |||
| 21 | Choosing How to Bundle Containers | ||
| 22 | --------------------------------- | ||
| 23 | |||
| 24 | There are two ways to bundle containers into a host image: | ||
| 25 | |||
| 26 | 1. **BUNDLED_CONTAINERS variable** (simpler, no extra recipe needed) | ||
| 27 | |||
| 28 | # In local.conf or image recipe | ||
| 29 | BUNDLED_CONTAINERS = "container-base:docker myapp-container:docker:autostart" | ||
| 30 | |||
| 31 | 2. **container-bundle packages** (more flexible) | ||
| 32 | |||
| 33 | # Create a bundle recipe, then add to IMAGE_INSTALL | ||
| 34 | inherit container-bundle | ||
| 35 | CONTAINER_BUNDLES = "myapp-container:autostart" | ||
| 36 | |||
| 37 | ### Decision Guide | ||
| 38 | |||
| 39 | Use Case | BUNDLED_CONTAINERS | Bundle Recipe | ||
| 40 | --------------------------------------------|--------------------|-------------- | ||
| 41 | Simple: containers in one host image | recommended | overkill | ||
| 42 | Reuse containers across multiple images | repetitive | recommended | ||
| 43 | Remote containers (docker.io/library/...) | not supported | required | ||
| 44 | Package versioning and dependencies | not supported | supported | ||
| 45 | Distribute pre-built container set | not supported | supported | ||
| 46 | |||
| 47 | **For most single-image use cases, BUNDLED_CONTAINERS is simpler:** | ||
| 48 | - No bundle recipe needed | ||
| 49 | - Dependencies auto-generated at parse time | ||
| 50 | - vrunner batch-import runs once for all containers | ||
| 51 | |||
| 52 | **Use container-bundle.bbclass when you need:** | ||
| 53 | - Remote container fetching via skopeo | ||
| 54 | - A distributable/versioned package of containers | ||
| 55 | - To share the same bundle across multiple different host images | ||
| 56 | |||
| 57 | |||
| 58 | Component Relationships | ||
| 59 | ----------------------- | ||
| 60 | |||
| 61 | To bundle a local container like "myapp:autostart", three recipe types | ||
| 62 | work together: | ||
| 63 | |||
| 64 | 1. **Application Recipe** (builds the software) | ||
| 65 | |||
| 66 | recipes-demo/myapp/myapp_1.0.bb | ||
| 67 | - Compiles application binaries | ||
| 68 | - Creates installable package (myapp) | ||
| 69 | |||
| 70 | 2. **Container Image Recipe** (creates OCI image containing the app) | ||
| 71 | |||
| 72 | recipes-demo/images/myapp-container.bb | ||
| 73 | - inherit image image-oci | ||
| 74 | - IMAGE_INSTALL = "myapp" | ||
| 75 | - Produces: ${DEPLOY_DIR_IMAGE}/myapp-container-latest-oci/ | ||
| 76 | |||
| 77 | 3. **Bundle Recipe** (packages container images for deployment) - OPTIONAL | ||
| 78 | |||
| 79 | recipes-demo/bundles/my-bundle_1.0.bb | ||
| 80 | - inherit container-bundle | ||
| 81 | - CONTAINER_BUNDLES = "myapp-container:autostart" | ||
| 82 | - Creates installable package with OCI data | ||
| 83 | |||
| 84 | ### Flow Diagram | ||
| 85 | |||
| 86 | myapp_1.0.bb myapp-container.bb | ||
| 87 | (application) (container image) | ||
| 88 | │ │ | ||
| 89 | │ IMAGE_INSTALL="myapp" │ inherit image-oci | ||
| 90 | └──────────────┬────────────────┘ | ||
| 91 | │ | ||
| 92 | ▼ | ||
| 93 | myapp-container-latest-oci/ | ||
| 94 | (OCI directory in DEPLOY_DIR_IMAGE) | ||
| 95 | │ | ||
| 96 | │ BUNDLED_CONTAINERS or CONTAINER_BUNDLES | ||
| 97 | ▼ | ||
| 98 | container-image-host | ||
| 99 | (target host image with containers pre-installed) | ||
| 100 | |||
| 101 | ### Concrete Example (from meta-virtualization) | ||
| 102 | |||
| 103 | - Application: `recipes-demo/autostart-test/autostart-test_1.0.bb` | ||
| 104 | - Container image: `recipes-demo/images/autostart-test-container.bb` | ||
| 105 | - Usage: `BUNDLED_CONTAINERS = "autostart-test-container:docker:autostart"` | ||
| 106 | |||
| 107 | |||
| 108 | Using BUNDLED_CONTAINERS | ||
| 109 | ------------------------ | ||
| 110 | |||
| 111 | Set in local.conf or image recipe: | ||
| 112 | |||
| 113 | BUNDLED_CONTAINERS = "container-base:docker myapp-container:podman:autostart" | ||
| 114 | |||
| 115 | ### Format | ||
| 116 | |||
| 117 | name:runtime[:autostart][:external] | ||
| 118 | |||
| 119 | - **name**: Container image recipe name or OCI directory name | ||
| 120 | - **runtime**: `docker` or `podman` | ||
| 121 | - **autostart**: Optional - `autostart`, `always`, `unless-stopped`, `on-failure` | ||
| 122 | - **external**: Optional - skip dependency generation for third-party blobs | ||
| 123 | |||
| 124 | ### Examples | ||
| 125 | |||
| 126 | # Yocto-built containers (dependencies auto-generated) | ||
| 127 | BUNDLED_CONTAINERS = "container-base:docker" | ||
| 128 | BUNDLED_CONTAINERS = "myapp-container:podman:autostart" | ||
| 129 | |||
| 130 | # Third-party blobs (no dependency generated) | ||
| 131 | BUNDLED_CONTAINERS = "vendor-image:docker:external" | ||
| 132 | |||
| 133 | # Legacy format (still supported) | ||
| 134 | BUNDLED_CONTAINERS = "container-base-latest-oci:docker" | ||
| 135 | |||
| 136 | |||
| 137 | Using container-bundle.bbclass | ||
| 138 | ------------------------------ | ||
| 139 | |||
| 140 | Create a bundle recipe: | ||
| 141 | |||
| 142 | # recipes-demo/bundles/my-bundle_1.0.bb | ||
| 143 | inherit container-bundle | ||
| 144 | |||
| 145 | CONTAINER_BUNDLES = "\ | ||
| 146 | myapp-container:autostart \ | ||
| 147 | mydb-container \ | ||
| 148 | docker.io/library/redis:7 \ | ||
| 149 | " | ||
| 150 | |||
| 151 | # Required for remote containers: | ||
| 152 | CONTAINER_DIGESTS[docker.io_library_redis_7] = "sha256:..." | ||
| 153 | |||
| 154 | To get the digest for a remote container, use skopeo: | ||
| 155 | |||
| 156 | skopeo inspect docker://docker.io/library/redis:7 | jq -r '.Digest' | ||
| 157 | |||
| 158 | Install in your host image: | ||
| 159 | |||
| 160 | IMAGE_INSTALL:append:pn-container-image-host = " my-bundle" | ||
| 161 | |||
| 162 | |||
| 163 | Container Autostart | ||
| 164 | ------------------- | ||
| 165 | |||
| 166 | Containers can be configured to start automatically on boot: | ||
| 167 | |||
| 168 | | Policy | Description | | ||
| 169 | |--------|-------------| | ||
| 170 | | `autostart` | Alias for unless-stopped (recommended) | | ||
| 171 | | `always` | Always restart container | | ||
| 172 | | `unless-stopped` | Restart unless manually stopped | | ||
| 173 | | `on-failure` | Restart only on non-zero exit code | | ||
| 174 | |||
| 175 | **Generated files:** | ||
| 176 | - Docker: `/lib/systemd/system/container-<name>.service` | ||
| 177 | - Podman: `/etc/containers/systemd/<name>.container` (Quadlet format) | ||
| 178 | |||
| 179 | |||
| 180 | vdkr and vpdmn - Virtual Container Runtimes | ||
| 181 | =========================================== | ||
| 182 | |||
| 183 | vdkr (virtual docker) and vpdmn (virtual podman) are tools that execute | ||
| 184 | container commands inside a QEMU-emulated environment. They enable | ||
| 185 | cross-architecture container operations during Yocto builds. | ||
| 186 | |||
| 187 | | Tool | Runtime | State Directory | | ||
| 188 | |------|---------|-----------------| | ||
| 189 | | `vdkr` | Docker (dockerd + containerd) | `~/.vdkr/<arch>/` | | ||
| 190 | | `vpdmn` | Podman (daemonless) | `~/.vpdmn/<arch>/` | | ||
| 191 | |||
| 192 | Quick Start | ||
| 193 | ----------- | ||
| 194 | |||
| 195 | # Build and install the standalone SDK | ||
| 196 | MACHINE=qemux86-64 bitbake vcontainer-tarball | ||
| 197 | ./tmp/deploy/sdk/vcontainer-standalone.sh -d /tmp/vcontainer -y | ||
| 198 | source /tmp/vcontainer/init-env.sh | ||
| 199 | |||
| 200 | # List images | ||
| 201 | vdkr images | ||
| 202 | |||
| 203 | # Import an OCI container | ||
| 204 | vdkr vimport ./my-container-oci/ myapp:latest | ||
| 205 | |||
| 206 | # Export storage for deployment | ||
| 207 | vdkr --storage /tmp/docker-storage.tar vimport ./container-oci/ myapp:latest | ||
| 208 | |||
| 209 | # Clean persistent state | ||
| 210 | vdkr clean | ||
| 211 | |||
| 212 | Architecture Selection | ||
| 213 | ---------------------- | ||
| 214 | |||
| 215 | vdkr/vpdmn detect target architecture automatically. Override with: | ||
| 216 | |||
| 217 | | Method | Example | Priority | | ||
| 218 | |--------|---------|----------| | ||
| 219 | | `--arch` / `-a` flag | `vdkr -a aarch64 images` | Highest | | ||
| 220 | | Executable name | `vdkr-x86_64 images` | 2nd | | ||
| 221 | | `VDKR_ARCH` env var | `export VDKR_ARCH=aarch64` | 3rd | | ||
| 222 | | Config file | `~/.config/vdkr/arch` | 4th | | ||
| 223 | | Host architecture | `uname -m` | Lowest | | ||
| 224 | |||
| 225 | Memory Resident Mode | ||
| 226 | -------------------- | ||
| 227 | |||
| 228 | Keep QEMU VM running for fast command execution (~1s vs ~30s): | ||
| 229 | |||
| 230 | vdkr vmemres start # Start daemon | ||
| 231 | vdkr images # Fast! | ||
| 232 | vdkr pull alpine:latest # Fast! | ||
| 233 | vdkr vmemres stop # Stop daemon | ||
| 234 | |||
| 235 | Commands | ||
| 236 | -------- | ||
| 237 | |||
| 238 | ### Docker-Compatible | ||
| 239 | |||
| 240 | images, run, import, load, save, pull, tag, rmi, ps, rm, logs, start, stop, exec | ||
| 241 | |||
| 242 | ### Extended (vdkr-specific) | ||
| 243 | |||
| 244 | | Command | Description | | ||
| 245 | |---------|-------------| | ||
| 246 | | `vimport <path> [name:tag]` | Import OCI directory or tarball | | ||
| 247 | | `vrun [opts] <image> [cmd]` | Run with entrypoint cleared | | ||
| 248 | | `clean` | Remove persistent state | | ||
| 249 | | `vmemres start/stop/status` | Memory resident VM control | | ||
| 250 | |||
| 251 | |||
| 252 | How It Works | ||
| 253 | ============ | ||
| 254 | |||
| 255 | The container-cross-install system uses QEMU to run container tools | ||
| 256 | (Docker/Podman) for the target architecture during the build. This solves | ||
| 257 | the "pseudo problem" where container tools fail under Yocto's fakeroot. | ||
| 258 | |||
| 259 | Architecture: | ||
| 260 | |||
| 261 | ┌─────────────────────────────────────────────────────────────────┐ | ||
| 262 | │ QEMU VM (target architecture) │ | ||
| 263 | │ ┌───────────────────────────────────────────────────────────┐ │ | ||
| 264 | │ │ rootfs.img (squashfs, ~100-130MB) │ │ | ||
| 265 | │ │ - Docker or Podman tools │ │ | ||
| 266 | │ │ - Processes OCI containers │ │ | ||
| 267 | │ │ - Outputs storage tar via console │ │ | ||
| 268 | │ └───────────────────────────────────────────────────────────┘ │ | ||
| 269 | │ │ | ||
| 270 | │ Drive Layout: │ | ||
| 271 | │ /dev/vda = rootfs.img (tools) │ | ||
| 272 | │ /dev/vdb = input disk (OCI containers) │ | ||
| 273 | │ /dev/vdc = state disk (Docker/Podman storage) │ | ||
| 274 | └─────────────────────────────────────────────────────────────────┘ | ||
| 275 | |||
| 276 | Blobs are built via multiconfig and deployed to: | ||
| 277 | |||
| 278 | tmp-vruntime-aarch64/deploy/images/qemuarm64/vdkr/ | ||
| 279 | tmp-vruntime-x86-64/deploy/images/qemux86-64/vdkr/ | ||
| 280 | |||
| 281 | |||
| 282 | Testing | ||
| 283 | ======= | ||
| 284 | |||
| 285 | cd /opt/bruce/poky/meta-virtualization | ||
| 286 | |||
| 287 | # Run container-cross-install tests | ||
| 288 | pytest tests/test_container_cross_install.py -v | ||
| 289 | |||
| 290 | # Run vdkr/vpdmn CLI tests | ||
| 291 | pytest tests/test_vdkr.py tests/test_vpdmn.py -v --vdkr-dir /tmp/vcontainer | ||
| 292 | |||
| 293 | |||
| 294 | Quick Reference Commands | ||
| 295 | ======================== | ||
| 296 | |||
| 297 | Build host image with bundled containers: | ||
| 298 | |||
| 299 | cd /opt/bruce/poky | ||
| 300 | source oe-init-build-env build | ||
| 301 | |||
| 302 | # Ensure local.conf has: | ||
| 303 | # DISTRO_FEATURES:append = " virtualization vcontainer" | ||
| 304 | # BBMULTICONFIG = "vruntime-aarch64 vruntime-x86-64" | ||
| 305 | # BUNDLED_CONTAINERS = "container-base:docker autostart-test-container:docker:autostart" | ||
| 306 | |||
| 307 | MACHINE=qemux86-64 bitbake container-image-host | ||
| 308 | |||
| 309 | Build the vdkr/vpdmn SDK tarball: | ||
| 310 | |||
| 311 | # Build blobs for desired architectures (sequentially to avoid deadlocks) | ||
| 312 | bitbake mc:vruntime-x86-64:vdkr-initramfs-create mc:vruntime-x86-64:vpdmn-initramfs-create | ||
| 313 | bitbake mc:vruntime-aarch64:vdkr-initramfs-create mc:vruntime-aarch64:vpdmn-initramfs-create | ||
| 314 | |||
| 315 | # Build SDK tarball | ||
| 316 | MACHINE=qemux86-64 bitbake vcontainer-tarball | ||
| 317 | |||
| 318 | # Output: tmp/deploy/sdk/vcontainer-standalone.sh | ||
| 319 | |||
| 320 | Install and test SDK: | ||
| 321 | |||
| 322 | ./tmp/deploy/sdk/vcontainer-standalone.sh -d /tmp/vcontainer -y | ||
| 323 | source /tmp/vcontainer/init-env.sh | ||
| 324 | |||
| 325 | vdkr images | ||
| 326 | vdkr vimport /path/to/container-base-latest-oci/ test:latest | ||
| 327 | vdkr vmemres start | ||
| 328 | vdkr images | ||
| 329 | vdkr vmemres stop | ||
| 330 | vdkr clean | ||
| 331 | |||
| 332 | |||
| 333 | See Also | ||
| 334 | ======== | ||
| 335 | |||
| 336 | - `classes/container-cross-install.bbclass` - Main bundling class | ||
| 337 | - `classes/container-bundle.bbclass` - Package-based bundling | ||
| 338 | - `recipes-containers/vcontainer/README.md` - vdkr/vpdmn detailed documentation | ||
| 339 | - `tests/README.md` - Test documentation | ||
