summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--README.md3
-rw-r--r--recipes-containers/vcontainer/README.md368
2 files changed, 371 insertions, 0 deletions
diff --git a/README.md b/README.md
index d22d0501..8657f269 100644
--- a/README.md
+++ b/README.md
@@ -39,6 +39,9 @@ that may also be enabled:
39 - systemd: enable systemd services and unit files (for recipes for support) 39 - systemd: enable systemd services and unit files (for recipes for support)
40 - sysvinit: enable sysvinit scripts (for recipes with support) 40 - sysvinit: enable sysvinit scripts (for recipes with support)
41 - seccomp: enable seccomp support for packages that have the capability. 41 - seccomp: enable seccomp support for packages that have the capability.
42 - vcontainer: enables vdkr/vpdmn virtual container runtime support, including
43 kernel config fragments (9p, squashfs, overlayfs) and busybox utilities
44 (base64, hexdump) required for container cross-install during image builds.
42 45
43Dependencies 46Dependencies
44------------ 47------------
diff --git a/recipes-containers/vcontainer/README.md b/recipes-containers/vcontainer/README.md
new file mode 100644
index 00000000..4b7b03ed
--- /dev/null
+++ b/recipes-containers/vcontainer/README.md
@@ -0,0 +1,368 @@
1# vdkr & vpdmn - Emulated Docker/Podman for Cross-Architecture
2
3Execute Docker or Podman commands inside a QEMU-emulated target environment.
4
5| Tool | Runtime | State Directory |
6|------|---------|-----------------|
7| `vdkr` | Docker (dockerd + containerd) | `~/.vdkr/<arch>/` |
8| `vpdmn` | Podman (daemonless) | `~/.vpdmn/<arch>/` |
9
10## Quick Start
11
12```bash
13# Build vdkr
14bitbake vdkr-native
15
16# List images (uses host architecture by default)
17vdkr images
18
19# Explicit architecture
20vdkr -a aarch64 images
21
22# Import an OCI container
23vdkr vimport ./my-container-oci/ myapp:latest
24
25# Export storage for deployment
26vdkr --storage /tmp/docker-storage.tar vimport ./container-oci/ myapp:latest
27
28# Clean persistent state
29vdkr clean
30```
31
32## Architecture Selection
33
34vdkr detects the target architecture automatically. Override with:
35
36| Method | Example | Priority |
37|--------|---------|----------|
38| `--arch` / `-a` flag | `vdkr -a aarch64 images` | Highest |
39| Executable name | `vdkr-x86_64 images` | 2nd |
40| `VDKR_ARCH` env var | `export VDKR_ARCH=aarch64` | 3rd |
41| Config file | `~/.config/vdkr/arch` | 4th |
42| Host architecture | `uname -m` | Lowest |
43
44**Set default architecture:**
45```bash
46mkdir -p ~/.config/vdkr
47echo "aarch64" > ~/.config/vdkr/arch
48```
49
50**Backwards-compatible symlinks:**
51```bash
52vdkr-aarch64 images # Same as: vdkr -a aarch64 images
53vdkr-x86_64 images # Same as: vdkr -a x86_64 images
54```
55
56## Commands
57
58### Docker-Compatible (same syntax as Docker)
59
60| Command | Description |
61|---------|-------------|
62| `images` | List images |
63| `run [opts] <image> [cmd]` | Run a command in a container |
64| `import <tarball> [name:tag]` | Import rootfs tarball |
65| `load -i <file>` | Load Docker image archive |
66| `save -o <file> <image>` | Save image to archive |
67| `pull <image>` | Pull image from registry |
68| `tag <source> <target>` | Tag an image |
69| `rmi <image>` | Remove an image |
70| `ps`, `rm`, `logs`, `start`, `stop` | Container management |
71| `exec [opts] <container> <cmd>` | Execute in running container |
72
73### Extended Commands (vdkr-specific)
74
75| Command | Description |
76|---------|-------------|
77| `vimport <path> [name:tag]` | Import OCI directory, tarball, or directory (auto-detect) |
78| `vrun [opts] <image> [cmd]` | Run with entrypoint cleared (command runs directly) |
79| `clean` | Remove persistent state |
80| `memres start [-p port:port]` | Start memory resident VM with optional port forwards |
81| `memres stop` | Stop memory resident VM |
82| `memres restart [--clean]` | Restart VM (optionally clean state) |
83| `memres status` | Show memory resident VM status |
84| `memres list` | List all running memres instances |
85
86### run vs vrun
87
88| Command | Behavior |
89|---------|----------|
90| `run` | Docker-compatible - entrypoint honored |
91| `vrun` | Clears entrypoint when command given - runs command directly |
92
93## Options
94
95| Option | Description |
96|--------|-------------|
97| `--arch, -a <arch>` | Target architecture (x86_64 or aarch64) |
98| `--instance, -I <name>` | Use named instance (shortcut for `--state-dir ~/.vdkr/<name>`) |
99| `--stateless` | Don't use persistent state |
100| `--storage <file>` | Export Docker storage to tar after command |
101| `--state-dir <path>` | Override state directory |
102| `--no-kvm` | Disable KVM acceleration |
103| `-v, --verbose` | Enable verbose output |
104
105## Memory Resident Mode
106
107Keep QEMU VM running for fast command execution (~1s vs ~30s):
108
109```bash
110vdkr memres start # Start daemon
111vdkr images # Fast!
112vdkr pull alpine:latest # Fast!
113vdkr run -it alpine /bin/sh # Interactive mode works via daemon!
114vdkr memres stop # Stop daemon
115```
116
117Interactive mode (`run -it`, `vrun -it`, `exec -it`) now works directly via the daemon using virtio-serial passthrough - no need to stop/restart the daemon.
118
119Note: Interactive mode combined with volume mounts (`-v`) still requires stopping the daemon temporarily.
120
121## Port Forwarding
122
123Forward ports from host to containers for SSH, web servers, etc:
124
125```bash
126# Start daemon with port forwarding
127vdkr memres start -p 8080:80 # Host:8080 -> Guest:80
128vdkr memres start -p 8080:80 -p 2222:22 # Multiple ports
129
130# Run container with host networking (shares guest's network)
131vdkr run -d --rm --network=host nginx:alpine
132
133# Access from host
134curl http://localhost:8080 # Access nginx
135```
136
137**How it works:**
138```
139Host:8080 → (QEMU hostfwd) → Guest:80 → (--network=host) → Container on port 80
140```
141
142Containers must use `--network=host` because Docker runs with `--bridge=none` inside the guest. This means the container shares the guest VM's network stack directly.
143
144**Options:**
145- `-p <host_port>:<guest_port>` - TCP forwarding (default)
146- `-p <host_port>:<guest_port>/udp` - UDP forwarding
147- Multiple `-p` options can be specified
148
149**Managing instances:**
150```bash
151vdkr memres list # Show all running instances
152vdkr memres start -p 9000:80 # Prompts if instance already running
153vdkr -I web memres start -p 8080:80 # Start named instance "web"
154vdkr -I web images # Use named instance
155vdkr -I backend run -d --network=host my-api:latest
156```
157
158## Exporting Images
159
160Two ways to export, for different purposes:
161
162```bash
163# Export a single image as Docker archive (portable, can be `docker load`ed)
164vdkr save -o /tmp/myapp.tar myapp:latest
165
166# Export entire Docker storage for deployment to target rootfs
167vdkr --storage /tmp/docker-storage.tar images
168```
169
170| Method | Output | Use case |
171|--------|--------|----------|
172| `save -o file image:tag` | Docker archive | Share image, load on another Docker |
173| `--storage file` | `/var/lib/docker` tar | Deploy to target rootfs |
174
175## Persistent State
176
177By default, Docker state persists in `~/.vdkr/<arch>/`. Images imported in one session are available in the next.
178
179```bash
180vdkr vimport ./container-oci/ myapp:latest
181vdkr images # Shows myapp:latest
182
183# Later...
184vdkr images # Still shows myapp:latest
185
186# Start fresh
187vdkr --stateless images # Empty
188
189# Clear state
190vdkr clean
191```
192
193## Standalone Distribution
194
195Create a self-contained redistributable tarball that works without Yocto:
196
197```bash
198# Build the standalone tarball
199MACHINE=qemux86-64 bitbake vdkr-native -c create_tarball
200
201# Output: tmp/deploy/vdkr/vdkr-standalone-x86_64.tar.gz
202```
203
204The tarball includes:
205- `vdkr` - Main CLI script
206- `vdkr-aarch64`, `vdkr-x86_64` - Symlinks (only for available architectures)
207- `vrunner.sh` - QEMU runner
208- `vdkr-blobs/` - Kernel and initramfs per architecture
209- `qemu/` - QEMU system emulators with wrapper scripts
210- `lib/` - Shared libraries for QEMU
211- `share/qemu/` - QEMU firmware files
212- `socat` - Socket communication for memres mode
213- `init-env.sh` - Environment setup script
214
215Usage:
216```bash
217tar -xzf vdkr-standalone-x86_64.tar.gz
218cd vdkr-standalone
219source init-env.sh
220vdkr images
221```
222
223## Interactive Mode
224
225Run containers with an interactive shell:
226
227```bash
228# Interactive shell in a container
229vdkr run -it alpine:latest /bin/sh
230
231# Using vrun (clears entrypoint)
232vdkr vrun -it alpine:latest /bin/sh
233
234# Inside the container:
235/ # apk add curl
236/ # exit
237```
238
239## Networking
240
241vdkr supports outbound networking via QEMU's slirp user-mode networking:
242
243```bash
244# Pull an image from a registry
245vdkr pull alpine:latest
246
247# Images persist in state directory
248vdkr images # Shows alpine:latest
249```
250
251## Volume Mounts
252
253Mount host directories into containers using `-v` (requires memory resident mode):
254
255```bash
256# Start memres first
257vdkr memres start
258
259# Mount a host directory
260vdkr vrun -v /tmp/data:/data alpine cat /data/file.txt
261
262# Mount multiple directories
263vdkr vrun -v /home/user/src:/src -v /tmp/out:/out alpine /src/build.sh
264
265# Read-only mount
266vdkr vrun -v /etc/config:/config:ro alpine cat /config/settings.conf
267
268# With run command (same syntax)
269vdkr run -v ./local:/app --rm myapp:latest /app/run.sh
270```
271
272**How it works:**
273- Host files are copied to the virtio-9p share directory before container runs
274- Container accesses them via the shared filesystem mount
275- For `:rw` mounts (default), changes are synced back to host after container exits
276- For `:ro` mounts, changes in container are discarded
277
278**Limitations:**
279- Requires daemon mode (memres) - volume mounts don't work in regular mode
280- Interactive + volumes (`-it -v`) requires stopping daemon temporarily (share directory conflict)
281- Changes sync after container exits (not real-time)
282- Large directories may be slow to copy
283
284**Debugging with volumes:**
285```bash
286# Run non-interactively with a shell command to inspect volume contents
287vdkr vrun -v /tmp/data:/data alpine ls -la /data
288
289# Or start the container detached and exec into it
290vdkr run -d --name debug -v /tmp/data:/data alpine sleep 3600
291vdkr exec debug ls -la /data
292vdkr rm -f debug
293```
294
295## Testing
296
297See `tests/README.md` for the pytest-based test suite:
298
299```bash
300# Build standalone tarball
301MACHINE=qemux86-64 bitbake vdkr-native -c create_tarball
302
303# Extract and run tests
304cd /tmp && tar -xzf .../vdkr-standalone-x86_64.tar.gz
305cd /opt/bruce/poky/meta-virtualization
306pytest tests/test_vdkr.py -v --vdkr-dir /tmp/vdkr-standalone
307```
308
309## vpdmn (Podman)
310
311vpdmn provides the same functionality as vdkr but uses Podman instead of Docker:
312
313```bash
314# Pull and run with Podman
315vpdmn-x86_64 pull alpine:latest
316vpdmn-x86_64 vrun alpine:latest echo hello
317
318# Override entrypoint
319vpdmn-x86_64 run --rm --entrypoint /bin/cat alpine:latest /etc/os-release
320
321# Import OCI container
322vpdmn-x86_64 vimport ./my-container-oci/ myapp:latest
323```
324
325Key differences from vdkr:
326- **Daemonless** - No containerd/dockerd startup, faster boot (~5s vs ~10-15s)
327- **Separate state** - Uses `~/.vpdmn/<arch>/` (images not shared with vdkr)
328- **Same commands** - `images`, `pull`, `run`, `vrun`, `vimport`, etc. all work
329
330## Recipes
331
332| Recipe | Purpose |
333|--------|---------|
334| `vdkr-native_1.0.bb` | Main vdkr (Docker) CLI and blobs |
335| `vpdmn-native_1.0.bb` | Main vpdmn (Podman) CLI and blobs |
336| `vcontainer-native_1.0.bb` | Unified tarball with both tools |
337| `vdkr-initramfs-create_1.0.bb` | Build vdkr initramfs blobs |
338| `vpdmn-initramfs-create_1.0.bb` | Build vpdmn initramfs blobs |
339
340## Files
341
342| File | Purpose |
343|------|---------|
344| `vdkr.sh` | Docker CLI wrapper |
345| `vpdmn.sh` | Podman CLI wrapper |
346| `vrunner.sh` | Shared QEMU runner script |
347| `vdkr-init.sh` | Docker init script (baked into initramfs) |
348| `vpdmn-init.sh` | Podman init script (daemonless) |
349
350## Testing
351
352```bash
353# Build unified standalone tarball
354bitbake vcontainer-native -c create_tarball
355
356# Extract
357cd /tmp && tar -xzf .../vcontainer-standalone-*.tar.gz
358
359# Run tests for both tools
360cd /opt/bruce/poky/meta-virtualization
361pytest tests/test_vdkr.py tests/test_vpdmn.py -v --vdkr-dir /tmp/vcontainer-standalone
362```
363
364## See Also
365
366- `classes/container-cross-install.bbclass` for bundling containers into Yocto images
367- `classes/container-bundle.bbclass` for creating container bundle packages
368- `tests/README.md` for test documentation