xen: document guest import system and add tests

Add 3rd-party guest import section to README-xen.md covering
import types, kernel modes, Alpine example, and how to add
custom import handlers.

Add test_xen_guest_bundle.py with 46 pytest tests covering
bbclass structure, import handlers, kernel modes, license
warning, Alpine recipe, and README content.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>

1 files changed, 129 insertions, 95 deletions

diff --git a/recipes-extended/images/README-xen.md b/recipes-extended/images/README-xen.md
index 82d72364..f16aa743 100644
--- a/recipes-extended/images/README-xen.md
+++ b/recipes-extended/images/README-xen.md
@@ -20,154 +20,188 @@ to differentiate).
 
 It creates tarballs, ext4 and qcow images for testing purposes.
 
-bundling
+Bundling
 --------
 
-Guests can be bundled automatically through the following mechanisms:
+There are two ways to bundle Xen guests into a Dom0 host image:
 
-  - via the variable XEN_BUNDLED_GUESTS
-  - via a xen configuration file in the deploy directory of the format
-    xen-guest-bundle-*.cfg
+| Use Case | `BUNDLED_XEN_GUESTS` | Bundle Recipe |
+|---|---|---|
+| Simple: guests in one host image | recommended | overkill |
+| Reuse guests across multiple host images | repetitive | recommended |
+| Package versioning and dependencies | not supported | supported |
+| Distribute pre-built guest sets | not supported | supported |
 
-The guests can be built via OE, or be 3rd party guests. They just
-must be in the deploy directory so they can be copied into the rootfs
-of the xen host image
+### Variable-driven (BUNDLED_XEN_GUESTS)
 
-Type 1) XEN_BUNDLED_GUESTS
+Guests can be bundled into the host image automatically using
+`xen-guest-cross-install.bbclass` (inherited by xen-image-minimal).
 
-If XEN_BUNDLED_GUESTS is used, it is simply a colon separated list of
-rootfs:kernels. Normal variable rules apply, so it can be set in a
-local.conf, or in a bbappend to the image recipe.
+Set `BUNDLED_XEN_GUESTS` in local.conf or the image recipe:
 
-An example would be:
+  BUNDLED_XEN_GUESTS = "xen-guest-image-minimal:autostart"
 
- XEN_BUNDLED_GUESTS = "xen-guest-image-minimal-qemuarm64.rootfs.ext4:Image"
+Each entry is a recipe name with optional tags:
 
-These point at symlinks created in the image deploy directory, or they
-can be specific images/kernels without the symlink.
+  recipe-name[:autostart][:external]
 
-Type 2) A Xen guest configuration file
+  - recipe-name: Yocto image recipe that produces the guest rootfs
+  - autostart: Creates symlink in /etc/xen/auto/ for xendomains
+  - external: Skip dependency generation (3rd-party guest)
 
-If xen guest configuration files are found in the deploy directories
-the kernel and disk information contained within them will be processed
-and modified for the xen host. The kernel and guest image will be
-copied to the appropriate location, and the config made to match.
+Examples:
 
-These files following the naming convention: xen-guest-bundle*.cfg
+  # Single guest with autostart (default recommendation)
+  BUNDLED_XEN_GUESTS = "xen-guest-image-minimal:autostart"
 
-Guests of type #1 generate a configuration file that is picked up as
-type #2.
+  # Guest without autostart
+  BUNDLED_XEN_GUESTS = "xen-guest-image-minimal"
 
-An example config file follows:
+  # External/3rd-party guest (no build dependency)
+  BUNDLED_XEN_GUESTS = "my-vendor-guest:external"
 
-   name = "xen-guest"
-   memory = 512
-   vcpus = 1
-   disk = ['file:xen-guest-image-minimal-qemuarm64.rootfs.ext4,xvda,rw']
-   vif = ['bridge=xenbr0']
-   kernel = "Image"
-   extra = "root=/dev/xvda ro console=hvc0 ip=dhcp"
+Per-guest configuration via varflags:
 
-It should also be noted that when a xen-guest-image-minimal is built
-with the XEN_GUEST_AUTO_BUNDLE varaible set to True, a configuration
-file for type #2 will be generated and the guest bundled automatically
-when the host image is built.
+  XEN_GUEST_MEMORY[xen-guest-image-minimal] = "1024"
+  XEN_GUEST_VCPUS[xen-guest-image-minimal] = "2"
+  XEN_GUEST_VIF[xen-guest-image-minimal] = "bridge=xenbr0"
+  XEN_GUEST_EXTRA[xen-guest-image-minimal] = "root=/dev/xvda ro console=hvc0 ip=dhcp"
+
+Custom config file (replaces auto-generation):
+
+  SRC_URI += "file://my-custom-guest.cfg"
+  BUNDLED_XEN_GUESTS = "xen-guest-image-minimal:autostart"
+  XEN_GUEST_CONFIG_FILE[xen-guest-image-minimal] = "${UNPACKDIR}/my-custom-guest.cfg"
+
+Explicit rootfs/kernel for external guests:
+
+  XEN_GUEST_ROOTFS[my-vendor-guest] = "vendor-rootfs.ext4"
+  XEN_GUEST_KERNEL[my-vendor-guest] = "vendor-kernel"
+
+### Package-based (xen-guest-bundle.bbclass)
+
+For reusable guest sets, create a bundle recipe that inherits
+`xen-guest-bundle`:
+
+  # recipes-extended/xen-guest-bundles/my-guests_1.0.bb
+  inherit xen-guest-bundle
+
+  XEN_GUEST_BUNDLES = "xen-guest-image-minimal:autostart"
+  XEN_GUEST_MEMORY[xen-guest-image-minimal] = "1024"
+
+Then install the bundle in the host image:
+
+  IMAGE_INSTALL:append:pn-xen-image-minimal = " my-guests"
+
+The bundle package includes rootfs, kernel, and config files. At
+image time, `merge_installed_xen_bundles()` deploys them to the
+same target locations as the variable-driven path.
+
+Custom config files work the same way via SRC_URI + varflag:
+
+  SRC_URI += "file://my-custom-guest.cfg"
+  XEN_GUEST_CONFIG_FILE[xen-guest-image-minimal] = "${UNPACKDIR}/my-custom-guest.cfg"
+
+See `example-xen-guest-bundle_1.0.bb` for a complete example.
+
+### 3rd-party guest import
+
+The import system converts fetched source formats (tarballs, qcow2 images,
+etc.) into Xen-ready disk images at build time. This is for guests that
+are not built by Yocto (e.g., Alpine minirootfs, Debian cloud images).
+
+Per-guest varflags control the import:
+
+  XEN_GUEST_SOURCE_TYPE[guest] = "rootfs_dir"   # import handler type
+  XEN_GUEST_SOURCE_FILE[guest] = "alpine-rootfs" # file/dir in UNPACKDIR
+  XEN_GUEST_IMAGE_SIZE[guest] = "128"            # target image size in MB
+
+Built-in import types:
+
+| Type | Input | Output | Tool |
+|---|---|---|---|
+| `rootfs_dir` | Extracted rootfs directory | ext4 image | `mkfs.ext4 -F -d` |
+| `qcow2` | QCOW2 disk image | raw image | `qemu-img convert` |
+| `ext4` | ext4 image file | ext4 (copy) | `cp` |
+| `raw` | raw disk image | raw (copy) | `cp` |
+
+Native tool dependencies are resolved automatically at parse time.
+
+Kernel modes (per-guest via `XEN_GUEST_KERNEL` varflag):
+
+  - (not set): Shared host kernel from DEPLOY_DIR_IMAGE
+  - `"path"`: Custom kernel from UNPACKDIR or DEPLOY_DIR_IMAGE
+  - `"none"`: HVM guest, no kernel (omits kernel= from config)
+
+Alpine example (`alpine-xen-guest-bundle_3.23.bb`):
+
+  inherit xen-guest-bundle
+
+  SRC_URI = "https://...alpine-minirootfs-${ALPINE_VERSION}-${ALPINE_ARCH}.tar.gz;subdir=alpine-rootfs"
+
+  XEN_GUEST_BUNDLES = "alpine:autostart:external"
+  XEN_GUEST_SOURCE_TYPE[alpine] = "rootfs_dir"
+  XEN_GUEST_SOURCE_FILE[alpine] = "alpine-rootfs"
+  XEN_GUEST_IMAGE_SIZE[alpine] = "128"
+  XEN_GUEST_MEMORY[alpine] = "256"
+  XEN_GUEST_EXTRA[alpine] = "root=/dev/xvda ro console=hvc0"
+
+Adding custom import types: define a shell function
+`xen_guest_import_<type>(source_path, output_path, size_mb)` in a
+bbclass, recipe, or bbappend and set the corresponding
+`XEN_GUEST_IMPORT_DEPENDS_<type>` variable for native tool dependencies.
+
+Target layout
+-------------
 
 kernel and rootfs are copied to the target in /var/lib/xen/images/
 
 configuration files are copied to: /etc/xen
 
-Guests can be launched after boot with: xl create -c /etc/xen/<config file>
+autostart symlinks are created in: /etc/xen/auto/
+
+Guests can be launched after boot with: xl create -c /etc/xen/<guest>.cfg
 
 Build and boot
 --------------
 
-Using a reference qmeuarm64 MACHINE, the following are the commands
+Using a reference qemuarm64 MACHINE, the following are the commands
 to build and boot a guest.
 
 local.conf contains:
 
-   XEN_BUNDLED_GUESTS = "xen-guest-image-minimal-qemuarm64.rootfs.ext4:Image"
+   BUNDLED_XEN_GUESTS = "xen-guest-image-minimal:autostart"
 
  % bitbake xen-guest-image-minimal
  % bitbake xen-image-minimal
 
- % runqemu qemuarm64 nographic slirp qemuparams="-m 4096" tmp/deploy/images/qemuarm64/xen-image-minimal-qemuarm64.rootfs.ext4
+ % runqemu qemuarm64 nographic slirp qemuparams="-m 4096"
 
 Poky (Yocto Project Reference Distro) 5.1 qemuarm64 hvc0
 
 qemuarm64 login: root
 
-WARNING: Poky is a reference Yocto Project distribution that should be used for
-testing and development purposes only. It is recommended that you create your
-own distribution for production use.
-
- root@qemuarm64:~# uname -a
-Linux qemuarm64 6.10.11-yocto-standard #1 SMP PREEMPT Fri Sep 20 22:32:26 UTC 2024 aarch64 GNU/Linux
 root@qemuarm64:~# ls /etc/xen/
 auto
 cpupool
 scripts
-xen-guest-bundle-xen-guest-image-minimal-qemuarm64--20241112174803.cfg
+xen-guest-image-minimal.cfg
 xl.conf
 root@qemuarm64:~# ls /var/lib/xen/images/
 Image--6.10.11+git0+4bf82718cf_6c956b2ea6-r0-qemuarm64-20241018190311.bin
-xen-guest-image-minimal-qemuarm64.rootfs-20241111222814.ext4
-
- root@qemuarm64:~# ip a s
-1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
-    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
-    inet 127.0.0.1/8 scope host lo
-       valid_lft forever preferred_lft forever
-    inet6 ::1/128 scope host noprefixroute 
-       valid_lft forever preferred_lft forever
-2: enp0s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master xenbr0 qlen 1000
-    link/ether 52:54:00:12:35:02 brd ff:ff:ff:ff:ff:ff
-3: sit0@NONE: <NOARP> mtu 1480 qdisc noop qlen 1000
-    link/sit 0.0.0.0 brd 0.0.0.0
-4: xenbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
-    link/ether ee:e4:a8:24:24:e7 brd ff:ff:ff:ff:ff:ff
-    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic xenbr0
-       valid_lft 86354sec preferred_lft 86354sec
-    inet6 fec0::ece4:a8ff:fe24:24e7/64 scope site dynamic noprefixroute flags 100 
-       valid_lft 86356sec preferred_lft 14356sec
-    inet6 fe80::ece4:a8ff:fe24:24e7/64 scope link 
-       valid_lft forever preferred_lft forever
-
- root@qemuarm64:~# xl create -c /etc/xen/xen-guest-bundle-xen-guest-image-minimal-qemuarm64--20241112174803.cfg
+xen-guest-image-minimal-qemuarm64-20241111222814.ext4
 
-qemuarm64 login: root
+ root@qemuarm64:~# xl create -c /etc/xen/xen-guest-image-minimal.cfg
 
-WARNING: Poky is a reference Yocto Project distribution that should be used for
-testing and development purposes only. It is recommended that you create your
-own distribution for production use.
+qemuarm64 login: root
 
 root@qemuarm64:~# uname -a
 Linux qemuarm64 6.10.11-yocto-standard #1 SMP PREEMPT Fri Sep 20 22:32:26 UTC 2024 aarch64 GNU/Linux
 
-root@qemuarm64:~# wget example.com
-Connecting to example.com (93.184.215.14:80)
-wget: can't open 'index.html': File exists
-root@qemuarm64:~# rm index.html 
-root@qemuarm64:~# wget example.com
-Connecting to example.com (93.184.215.14:80)
-saving to 'index.html'
-index.html           100% |********************************|  1256  0:00:00 ETA
-'index.html' saved
-
 From the host:
 
-Connection to 127.0.0.1 closed.
-build4 [/home/bruc.../qemuarm64]> ssh -p 2222 root@127.0.0.1
-Last login: Tue Nov 12 20:42:57 2024 from 10.0.2.2
-
-WARNING: Poky is a reference Yocto Project distribution that should be used for
-testing and development purposes only. It is recommended that you create your
-own distribution for production use.
-
 root@qemuarm64:~# xl list
 Name                                        ID   Mem VCPUs      State   Time(s)
 Domain-0                                     0   192     4     r-----     696.2
-xen-guest                                    1   512     1     -b----     153.0
-root@qemuarm64:~# xl destroy xen-guest
-
+xen-guest-image-minimal                      1   512     1     -b----     153.0
+root@qemuarm64:~# xl destroy xen-guest-image-minimal