summaryrefslogtreecommitdiffstats
path: root/classes
Commit message (Collapse)AuthorAgeFilesLines
* image-oci: don't preserve ownership in directories/files/host layer copiesTim Orling14 days1-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The multi-layer 'directories', 'files', and 'host' branches in IMAGE_CMD:oci copy delta content into the OCI bundle rootfs with 'cp -a'. 'cp -a' implies '--preserve=all', which calls lchown() on the destination to copy ownership from the source. When a directories/files layer copies a symbolic link whose target does not exist at build time (for example, the '/dev/stdout' and '/dev/stderr' log forwarding symlinks used by the official nginx Docker image), lchown() can return EINVAL under pseudo and 'cp' aborts with: cp: failed to preserve ownership for .../var/log/nginx/access.log: Invalid argument failing the whole do_image_oci task. The single-layer rootfs copy already handles this correctly: cp -r -a --no-preserve=ownership ${IMAGE_ROOTFS}/* $image_bundle_name/rootfs and the multi-layer 'packages' branch uses 'rsync -a --no-owner --no-group' for the same reason. Bring the three remaining cp -a sites in line by adding '--no-preserve=ownership'. Ownership inside an OCI image is set by umoci based on the image config and source ownership has no meaning for symlinks to runtime device nodes anyway, so dropping preservation is the correct behaviour. Reproduce: declare a directories: layer that copies a path containing a symlink to '/dev/stdout' or '/dev/stderr' (e.g. a postprocess that creates /var/log/nginx/{access,error}.log -> /dev/{stdout,stderr} to mirror the upstream nginx Docker image). Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* oci-multiarch: fix MC defaults, deploy dependency, and OCI layoutBruce Ashfield2026-05-101-12/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Three issues prevented oci-multiarch.bbclass from producing usable multi-architecture container images: 1. MC defaults pointed to vruntime-* multiconfigs, whose BBMASK blocks OCI tooling. Changed to container-* multiconfigs which use the new vcontainer distro without BBMASK. 2. mcdepends targeted do_image_oci, but the OCI output is only deployed to deploy/images/ by the later do_image_complete task. The bbclass then failed to find the OCI directory at the expected deploy path. The original implementation assumed do_image_oci was the final step, but OE-core's image pipeline has a separate deploy phase. 3. The OCI Image Index was written directly into index.json with multiple manifest entries. This is valid per the OCI spec but skopeo requires index.json to reference a single entry when there are multiple images. The fix writes the multi-platform image index as a blob in blobs/sha256/ and has index.json reference it with a single entry of mediaType application/vnd.oci.image.index.v1+json. This nested layout is what tools like buildah and crane produce for multi-arch images, and is required for 'skopeo copy --all' to work. Also adds container-base-multiarch.bb recipe that wires up container-base for aarch64 + x86_64 builds via the oci-multiarch class. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* oe-go-mod-fetcher: add license scanning for Go module dependenciesBruce Ashfield2026-04-281-4/+84
| | | | | | | | | | | | | | | | | | | | | | | Add --scan-licenses to oe-go-mod-fetcher.py which scans Go module zips for license files and generates go-mod-licenses.inc with LICENSE and LIC_FILES_CHKSUM entries matching OE-core's go-mod-update-modules format. License detection uses OE-core's glob patterns and MD5 + crunched MD5 matching against known SPDX licenses. The hash database resolves from: 1. --common-license-dir (explicit path) 2. Auto-detected poky tree common-licenses 3. Bundled scripts/data/license-hashes.csv (offline fallback) New files: - scripts/generate-license-hashes.py: regenerate bundled CSV - scripts/data/license-hashes.csv: pre-computed hash DB (704 entries) bbclass changes: - go-mod-discovery: pass --scan-licenses during do_generate_modules - GO_MOD_DISCOVERY_SKIP_LICENSES variable to bypass scanning - do_update_license_hashes task to refresh bundled CSV Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* go-mod-vcs.bbclass: handle pure hybrid mode gracefullyBruce Ashfield2026-03-261-1/+7
| | | | | | | | | | | | | When a recipe uses only gomod:// fetches with no VCS (git://) modules, GO_MODULE_CACHE_DATA is an empty list. Previously do_create_module_cache would run with nothing to process and do_sync_go_files would fatal with "No modules found in cache". Detect the empty module list and return early with an informational message in both tasks, allowing pure hybrid recipes (like yq) to build without workarounds. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-discovery/oe-go-mod-fetcher: add GO_MOD_VCS_EXCLUDE for deleted reposBruce Ashfield2026-03-191-1/+12
| | | | | | | | | | | | | | | | | | Some upstream Go module repositories get deleted from GitHub (e.g., github.com/vtolstov/go-ioctl). While the Go module proxy still serves cached archives, VCS mode cannot git clone a deleted repo. This causes both do_fetch failures and generator verification failures. Add GO_MOD_VCS_EXCLUDE recipe variable (space-separated module path prefixes) and corresponding --exclude-module CLI flag. Excluded modules are filtered out before verification and SRC_URI generation. Recipes must provide a gomod:// SRC_URI entry for excluded modules as fallback. Usage in recipe: SRC_URI += "gomod://example.com/deleted-repo;version=v1.0.0;sha256sum=..." GO_MOD_VCS_EXCLUDE = "example.com/deleted-repo" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-bundle: add explicit do_fetch dependency on skopeo-nativeBruce Ashfield2026-03-121-0/+3
| | | | | | | | | | | | do_fetch_containers runs as a postfunc of do_fetch and needs skopeo from the native sysroot. However, DEPENDS only gates do_prepare_recipe_sysroot which runs after do_fetch, so with a clean tmp/ directory skopeo-native may not be built when do_fetch runs. Add an explicit do_fetch[depends] on skopeo-native:do_populate_sysroot when remote containers are configured so the ordering is correct. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-registry: make IMAGE_FEATURES local to image recipesBruce Ashfield2026-03-121-3/+0
| | | | | | | | | | | | | | | Remove the global IMAGE_FEATURES[validitems] registration entirely. Setting it in layer.conf or a globally-inherited bbclass changes the varflag value, which gets pulled into the signature of every recipe that depends on IMAGE_FEATURES — causing yocto-check-layer signature change failures. Image recipes that use the container-registry feature already set IMAGE_FEATURES[validitems] locally (e.g. container-image-host.bb). Users who want the feature in their own images add the one-liner: IMAGE_FEATURES[validitems] += "container-registry" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-registry: make IMAGE_FEATURES conditional on distro featuresBruce Ashfield2026-03-111-0/+3
| | | | | | | | | | | | | Move the container-registry IMAGE_FEATURES[validitems] registration from layer.conf into meta-virt-container-cfg.bbclass where it can be gated on DISTRO_FEATURES. The validitems varflag is now only registered when vcontainer or virtualization is in DISTRO_FEATURES. layer.conf is parsed before distro features are known, so inline Python cannot be used there. The bbclass is loaded via USER_CLASSES (deferred parsing) and already handles container profile configuration. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* xen-guest-bundle: drop QA skip for installed-vs-shippedBruce Ashfield2026-03-111-1/+1
| | | | | | | | | | While we need the buildpaths QA skip due to the 3rd party nature of some bundles, we don't need the installed-vs-shipped. There's no reason why a bundled guest isn't packaging all its files (or deleting them). Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* sanity: default virtualization distro check to falseBruce Ashfield2026-03-101-0/+2
| | | | | | | | | | | | | | | | | We've terrorized the users of meta-virt for long enough with the warning about the virtualization distro feature not being set. Modern OE has the features_check capability so recipes and functionality that need the distro feature to be set can just check it themselves. No need to warn over and over again. For now, it just default to off, but the check can be completely removed in the future (since it does cause some parse time overhead). Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: update oci-iamge-tools + runc instructionsBruce Ashfield2026-03-031-13/+34
| | | | | | | | | | | | | | Our runc is now sufficiently far ahead of the oci-image-tools such that they aren't compatible. But They can still be made to work with a runc generated spec. Update the comments to show a flow that works if using runc directly is desired. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* xen: add guest bundling infrastructure with cross-install classBruce Ashfield2026-02-261-0/+437
| | | | | | | | | | | | | Add xen-guest-cross-install.bbclass for bundling Xen guest images into Dom0 host images at build time. Supports per-guest configuration via varflags (memory, vcpus, vif, kernel extra), custom config files, external/pre-built guests, and autostart via /etc/xen/auto/ symlinks. Also add example-xen-guest-bundle recipe as a template, and simplify xen-guest-image-minimal by removing the old XEN_GUEST_AUTO_BUNDLE do_deploy mechanism in favor of the new class-based approach. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* xen-guest-bundle: add guest import system for 3rd-party imagesBruce Ashfield2026-02-261-18/+254
| | | | | | | | | | | | | | | | | | | | | | | | | | Add extensible import system to convert fetched source formats (tarballs, qcow2, etc.) into Xen-ready disk images at build time. Built-in import types: - rootfs_dir: extracted directory → ext4 via mkfs.ext4 -d - qcow2: QCOW2 → raw via qemu-img convert - ext4/raw: copy passthrough Per-guest varflags control the import: XEN_GUEST_SOURCE_TYPE[guest] = "rootfs_dir" XEN_GUEST_SOURCE_FILE[guest] = "alpine-rootfs" XEN_GUEST_IMAGE_SIZE[guest] = "128" Also adds three kernel modes for resolve_bundle_kernel(): - (not set): shared host kernel from DEPLOY_DIR_IMAGE - "path": custom kernel, checks UNPACKDIR then DEPLOY_DIR_IMAGE - "none": HVM guest, omits kernel= from config Native tool dependencies and fakeroot are resolved automatically at parse time. External guests emit a single license warning at do_compile time (prefunc, not parse-time). Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* xen-guest-bundle: add bbclass for packaging Xen guest bundlesBruce Ashfield2026-02-261-0/+405
| | | | | | | | | | | | | | | | New bbclass that creates installable packages bundling Xen guest images (rootfs + kernel + config). When installed via IMAGE_INSTALL into a Dom0 image that inherits xen-guest-cross-install, guests are automatically deployed by merge_installed_xen_bundles(). Features: - Parse-time dependency generation from XEN_GUEST_BUNDLES - Per-guest varflags for memory, vcpus, vif, extra, disk, name - Custom config file support via XEN_GUEST_CONFIG_FILE varflag - Explicit rootfs/kernel path overrides for external guests - Manifest-based packaging for cross-install integration Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container tasks: move network access out of build chainBruce Ashfield2026-02-101-3/+10
| | | | | | | | | | | | | | | | | | | yocto-check-layer reports an error for any task between do_fetch and do_build that has network enabled. Two changes fix this: container-bundle.bbclass: Move do_fetch_containers from a standalone task into a do_fetch postfunc. When remote containers are configured, the anonymous function adds extend_recipe_sysroot as a do_fetch prefunc (so skopeo-native is available) and do_fetch_containers as a postfunc. Network access during do_fetch is permitted by the QA check. container-registry-index: Remove do_container_registry_index from the build dependency chain (drop "before do_build"). Registry push is a deployment action requiring explicit invocation: bitbake container-registry-index -c container_registry_index The default do_build task now prints usage instructions. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci-umoci: create usrmerge symlinks in multi-layer modeBruce Ashfield2026-02-101-0/+10
| | | | | | | | | | ROOTFS_PREPROCESS_COMMAND creates usrmerge symlinks (/bin -> usr/bin, etc.) during do_rootfs, but multi-layer OCI mode installs packages to separate layer rootfs directories that bypass this step. Call the existing create_merged_usr_symlinks function on the first packages layer to fix this. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer: fix yocto-check-layer mcdepends parse errorBruce Ashfield2026-02-092-11/+17
| | | | | | | | | | | | | | | | | | Fix yocto-check-layer failure: ERROR: Multiconfig dependency mc::vruntime-x86-64:vpdmn-initramfs-create:do_deploy depends on nonexistent multiconfig configuration named configuration vruntime-x86-64 Several recipes and classes declared static mcdepends referencing vruntime-aarch64 and vruntime-x86-64 multiconfigs. When parsed without BBMULTICONFIG set (e.g. yocto-check-layer), BitBake validates these and fails because the referenced multiconfigs don't exist. Move mcdepends into anonymous python functions and only set them when the target multiconfig exists in BBMULTICONFIG, following the pattern established in meta/classes-recipe/kernel-fit-image.bbclass. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-vcs: fix do_rm_work permission failure on module cacheBruce Ashfield2026-02-091-12/+4
| | | | | | | | | | | | | go build creates read-only files in the module cache during do_compile. The previous do_fix_go_mod_permissions task ran before do_compile, so it could not catch these files, causing do_rm_work to fail with permission errors. Replace the standalone task with a do_compile postfunc that fixes module cache permissions after compilation finishes. This covers all go-mod-vcs recipes regardless of how they invoke go build. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: fix process substitution for dash/busybox compatibilityBruce Ashfield2026-02-091-6/+7
| | | | | | | | | | | Replace bash-specific process substitution (< <(find ...)) with POSIX-compatible piped find | while constructs. Replace $((...)) arithmetic with expr for broader shell compatibility. This fixes OCI image delta-copy on systems where /bin/sh is dash or busybox ash rather than bash. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-registry: add target image TLS integrationBruce Ashfield2026-02-091-0/+10
| | | | | | | | | | | | | | | | | | | | | | | Install CA certificates and registry configuration into target images so they can pull from the secure registry at runtime. docker-registry-config.bb: When CONTAINER_REGISTRY_SECURE=1, install the CA cert to /etc/docker/certs.d/{host}/ca.crt instead of adding insecure-registries to daemon.json. Translates localhost/127.0.0.1 to 10.0.2.2 for QEMU targets where the host registry is accessed via slirp networking. container-oci-registry-config.bb: Same secure mode support for podman/CRI-O with insecure=false in registries.conf. container-registry-ca.bb: New recipe that installs the CA certificate to Docker, podman/CRI-O, and system trust store paths on the target. container-cross-install.bbclass: Auto-add docker-registry-config or container-oci-registry-config to IMAGE_INSTALL when CONTAINER_REGISTRY_SECURE=1, based on the configured container engine. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-registry: add secure registry infrastructure with TLS and authBruce Ashfield2026-02-091-1/+165
| | | | | | | | | | | | | | | | | | | | | | | | | | | Add opt-in secure registry mode with auto-generated TLS certificates and htpasswd authentication. New BitBake variables: CONTAINER_REGISTRY_SECURE - Enable TLS (HTTPS) for local registry CONTAINER_REGISTRY_AUTH - Enable htpasswd auth (requires SECURE=1) CONTAINER_REGISTRY_USERNAME/PASSWORD - Credential configuration CONTAINER_REGISTRY_CERT_DAYS/CA_DAYS - Certificate validity CONTAINER_REGISTRY_CERT_SAN - Custom SAN entries The bbclass validates conflicting settings (AUTH without SECURE) and provides credential helper functions for skopeo push operations. PKI infrastructure (CA + server cert with SAN) is auto-generated at bitbake build time via openssl-native. The generated helper script supports both TLS-only and TLS+auth modes. The script now supports environment variable overrides for CONTAINER_REGISTRY_STORAGE, CONTAINER_REGISTRY_URL, and CONTAINER_REGISTRY_NAMESPACE, uses per-port PID files to allow multiple instances, and auto-generates config files when running from an overridden storage path. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-cross-install: add CONTAINER_SERVICE_FILE supportBruce Ashfield2026-02-092-1/+236
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for custom systemd service files (Docker) or Quadlet container files (Podman) instead of auto-generated ones for container autostart. For containers requiring specific startup configuration (ports, volumes, capabilities, dependencies), users can now provide custom service files using the CONTAINER_SERVICE_FILE varflag: CONTAINER_SERVICE_FILE[container-name] = "${UNPACKDIR}/myservice.service" For BUNDLED_CONTAINERS in image recipes: SRC_URI += "file://myapp.service" BUNDLED_CONTAINERS = "myapp-container:docker:autostart" CONTAINER_SERVICE_FILE[myapp-container] = "${UNPACKDIR}/myapp.service" For container-bundle packages: SRC_URI = "file://myapp.service" CONTAINER_BUNDLES = "myapp-container:autostart" CONTAINER_SERVICE_FILE[myapp-container] = "${UNPACKDIR}/myapp.service" Implementation: - container-cross-install.bbclass: Add get_container_service_file_map() to build varflag map, install_custom_service() for BUNDLED_CONTAINERS, and install_custom_service_from_bundle() for bundle packages - container-bundle.bbclass: Install custom service files to ${datadir}/container-bundles/${runtime}/services/ Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add host layer type and delta-only copyingBruce Ashfield2026-02-092-14/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add two enhancements to multi-layer OCI image support: 1. Delta-only copying for directories/files layers: - directories and files layers now only copy content that doesn't already exist in the bundle rootfs from earlier layers - Prevents duplication when a directories layer references paths that were already populated by a packages layer - Logs show "delta: N copied, M skipped" for visibility 2. New 'host' layer type for build machine content: - Copies files from the build machine filesystem (outside Yocto) - Format: name:host:source_path:dest_path - Multiple pairs: name:host:src1:dst1+src2:dst2 - Emits warning at parse time about reproducibility impact - Fatal error if source path doesn't exist - Use case: deployment-specific config, certificates, keys that cannot be packaged in recipes Example: OCI_LAYERS = "\ base:packages:busybox \ app:directories:/opt/myapp \ certs:host:/etc/ssl/certs/ca.crt:/etc/ssl/certs/ca.crt \ " Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer-tarball: build all architectures via single bitbake commandBruce Ashfield2026-02-091-0/+5
| | | | | | | | | | | | | | | | | | | | Previously, building vcontainer-tarball required multiple bitbake invocations or complex command lines to build both x86_64 and aarch64 blobs. This was a usability issue. Changes: - mcdepends now triggers builds for BOTH architectures automatically - VCONTAINER_ARCHITECTURES defaults to "x86_64 aarch64" (was auto-detect) - Add informational banner at parse time showing what will be built - Fix duplicate sanity check messages when multiconfig is active Usage is now simply: bitbake vcontainer-tarball To build only one architecture, set in local.conf: VCONTAINER_ARCHITECTURES = "x86_64" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer: add sanity checks and auto-enable virtfs for QEMUBruce Ashfield2026-02-091-2/+4
| | | | | | | | | | | | | | | | | Fix virtio-9p (virtfs) support for container-cross-install batch imports which provides ~50x speedup over base64-over-serial. The issue was that native recipes don't see target DISTRO_FEATURES, so qemu-system-native wasn't getting virtfs enabled. Fix by: - layer.conf: Propagate virtualization to DISTRO_FEATURES_NATIVE when vcontainer or virtualization is in target DISTRO_FEATURES - qemu-system-native: Check DISTRO_FEATURES_NATIVE for virtfs enable - container-cross-install: Prepend native sysroot to PATH so vrunner finds the QEMU with virtfs support Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer: add sanity checks and auto-enable virtfs for QEMUBruce Ashfield2026-02-091-0/+36
| | | | | | | | | | | Add sanity check that warns when vcontainer distro feature is enabled but BBMULTICONFIG is missing the required vruntime-* multiconfigs. Add qemu-system-native bbappend to auto-enable virtfs (virtio-9p) when vcontainer or virtualization distro feature is set. This is required for the fast batch-import path in container-cross-install. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer: add multi-arch OCI supportBruce Ashfield2026-02-091-0/+243
| | | | | | | | | | | | | | | | | | | | | Add functions to detect and handle multi-architecture OCI Image Index format with automatic platform selection during import. Also add oci-multiarch.bbclass for build-time multi-arch OCI creation. Runtime support (vcontainer-common.sh): - is_oci_image_index() - detect multi-arch OCI images - get_oci_platforms() - list available platforms - select_platform_manifest() - select manifest for target architecture - extract_platform_oci() - extract single platform to new OCI dir - normalize_arch_to_oci/from_oci() - architecture name mapping - Update vimport to auto-select platform from multi-arch images Build-time support (oci-multiarch.bbclass): - Create OCI Image Index from multiconfig builds - Collect images from vruntime-aarch64, vruntime-x86-64 - Combine blobs and create unified manifest list Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-cross-install: fix image naming and default runtimeBruce Ashfield2026-02-091-17/+56
| | | | | | | | | | | | Fix extract_container_info() to properly handle multi-part container names and add automatic runtime detection based on CONTAINER_PROFILE. - Fix multi-part name parsing (app-container-multilayer-latest-oci now correctly becomes app-container-multilayer:latest) - Add CONTAINER_DEFAULT_RUNTIME from CONTAINER_PROFILE - Add CONTAINER_IMPORT_TIMEOUT_BASE/PER for dynamic timeout scaling Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add layer caching for multi-layer OCI buildsBruce Ashfield2026-02-092-2/+201
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add layer caching to speed up multi-layer OCI image rebuilds. When enabled, pre-installed package layers are cached to disk and restored on subsequent builds, avoiding repeated package installation. New variables: - OCI_LAYER_CACHE: Enable/disable caching (default "1") - OCI_LAYER_CACHE_DIR: Cache location (default ${TOPDIR}/oci-layer-cache/${MACHINE}) Cache key is computed from: - Layer name and type - Sorted package list - Package versions from PKGDATA_DIR - MACHINE and TUNE_PKGARCH Cache automatically invalidates when: - Package versions change - Layer definition changes - Architecture changes Benefits: - First build: ~10-30s per layer (cache miss, packages installed) - Subsequent builds: ~1s per layer (cache hit, files copied) - Shared across recipes with identical layer definitions Build log shows cache status: NOTE: OCI Cache HIT: Layer 'base' (be88c180f651416b) NOTE: OCI: Pre-installed packages for 3 layers (cache: 3 hits, 0 misses) Also adds comprehensive pytest suite for multi-layer OCI functionality including tests for 1/2/3 layer modes and cache behavior. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add multi-layer OCI image support with OCI_LAYERSBruce Ashfield2026-02-092-23/+365
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for creating multi-layer OCI images with explicit layer definitions via OCI_LAYERS variable. This enables fine-grained control over container layer composition. New variables: - OCI_LAYER_MODE: Set to "multi" for explicit layer definitions - OCI_LAYERS: Define layers as "name:type:content" entries - packages: Install specific packages in a layer - directories: Copy directories from IMAGE_ROOTFS - files: Copy specific files from IMAGE_ROOTFS Package installation uses Yocto's package manager classes (RpmPM, OpkgPM) for consistency with do_rootfs, rather than calling dnf/opkg directly. Example usage: OCI_LAYER_MODE = "multi" OCI_LAYERS = "\ base:packages:base-files+base-passwd+netbase \ shell:packages:busybox \ app:packages:curl \ " This creates a 3-layer OCI image with discrete base, shell, and app layers that can be shared and cached independently. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-bundle: add CONTAINER_BUNDLE_DEPLOY for base layer useBruce Ashfield2026-02-091-0/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | Add CONTAINER_BUNDLE_DEPLOY variable to enable dual-use of container-bundle: 1. Target packages (existing): Creates installable packages for target container storage (Docker/Podman) 2. Base layer source (new): When CONTAINER_BUNDLE_DEPLOY = "1", also deploys the fetched OCI image to DEPLOY_DIR_IMAGE for use as a base layer via OCI_BASE_IMAGE This enables fetching external images (docker.io, quay.io) and using them as base layers for Yocto-built container images. Example usage: # recipes-containers/oci-base-images/alpine-oci-base_3.19.bb inherit container-bundle CONTAINER_BUNDLES = "docker.io/library/alpine:3.19" CONTAINER_DIGESTS[docker.io_library_alpine_3.19] = "sha256:..." CONTAINER_BUNDLE_DEPLOY = "1" # Then in your app container recipe: OCI_BASE_IMAGE = "alpine-oci-base" IMAGE_INSTALL = "myapp" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add multi-layer OCI support and CMD defaultBruce Ashfield2026-02-092-15/+395
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for multi-layer OCI images, enabling base + app layer builds: Multi-layer support: - Add OCI_BASE_IMAGE variable to specify base layer (recipe name or path) - Add OCI_BASE_IMAGE_TAG for selecting base image tag (default: latest) - Resolve base image type (recipe/path/remote) at parse time - Copy base OCI layout before adding new layer via umoci repack - Fix merged-usr whiteout ordering issue for non-merged-usr base images (replaces problematic whiteouts with filtered entries to avoid Docker pull failures when layering merged-usr on traditional layout) CMD/ENTRYPOINT behavior change: - Add OCI_IMAGE_CMD variable (default: "/bin/sh") - Change OCI_IMAGE_ENTRYPOINT default to empty string - This makes `docker run image /bin/sh` work as expected (like Docker Hub images) - OCI_IMAGE_ENTRYPOINT_ARGS still works for legacy compatibility - Fix shlex.split() for proper shell quoting in CMD/ENTRYPOINT values The multi-layer feature requires umoci backend (default). The sloci backend only supports single-layer images and will error if OCI_BASE_IMAGE is set. Example usage: OCI_BASE_IMAGE = "container-base" IMAGE_INSTALL = "myapp" OCI_IMAGE_CMD = "/usr/bin/myapp" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add build-time metadata labels for traceabilityBruce Ashfield2026-02-092-1/+81
| | | | | | | | | | | | | | | | | | | | | | | Automatically embed source and build information into OCI images using standard OCI annotations (opencontainers.org image-spec): - org.opencontainers.image.revision: git commit SHA - org.opencontainers.image.ref.name: git branch name - org.opencontainers.image.created: ISO 8601 build timestamp - org.opencontainers.image.version: PV (if meaningful) New variables: - OCI_IMAGE_REVISION: explicit SHA override (auto-detects from TOPDIR) - OCI_IMAGE_BRANCH: explicit branch override (auto-detects from TOPDIR) - OCI_IMAGE_BUILD_DATE: explicit timestamp override (auto-generated) - OCI_IMAGE_APP_RECIPE: hook for future cross-recipe extraction Set any variable to "none" to disable that specific label. This enables 1:1 traceability between container images and source code, following industry best practices for CI/CD and release management. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-registry: add industry-standard tag strategiesBruce Ashfield2026-02-091-4/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | Add comprehensive tag support for registry push operations: Tag strategies (CONTAINER_REGISTRY_TAG_STRATEGY): - sha/git: short git commit hash for traceability - branch: git branch name (sanitized) for dev workflows - semver: nested SemVer tags (1.2.3 -> 1.2.3, 1.2, 1) - timestamp: YYYYMMDD-HHMMSS format - version: single version tag from PV - latest: the "latest" tag - arch: append architecture suffix Helper script enhancements: - push --tag <tag>: explicit tags (repeatable) - push --strategy <strategies>: override tag strategy - push --version <ver>: version for semver strategy - Baked-in defaults from bitbake variables - Environment variable overrides supported This aligns with industry practices: - Git SHA for CI/CD traceability - SemVer nested tags for release management - Branch tags for feature development Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-registry: add local OCI registry infrastructureBruce Ashfield2026-02-091-0/+203
| | | | | | | | | | | Add container registry support for Yocto container workflows: - container-registry.bbclass with helper functions - container-registry-index.bb generates helper script with baked paths - docker-registry-config.bb for Docker daemon on targets - container-oci-registry-config.bb for Podman/Skopeo/Buildah targets - IMAGE_FEATURES container-registry for easy target configuration Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer: add auto-start daemon with idle timeoutBruce Ashfield2026-02-091-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | Add automatic daemon startup and idle timeout cleanup for vdkr/vpdmn: - vmemres daemon auto-starts on first command (no manual start needed) - Daemon auto-stops after idle timeout (default: 30 minutes) - --no-daemon flag for ephemeral mode (single-shot QEMU) - New config keys: idle-timeout, auto-daemon Changes: - vcontainer-init-common.sh: Parse idle_timeout from cmdline, add read -t timeout to daemon loop for auto-shutdown - vrunner.sh: Add --idle-timeout option, pass to kernel cmdline - vcontainer-common.sh: Auto-start logic in run_runtime_command(), --no-daemon flag, config defaults - container-cross-install.bbclass: Add --no-daemon for explicit ephemeral mode during Yocto builds Configuration: vdkr vconfig idle-timeout 3600 # 1 hour timeout vdkr vconfig auto-daemon false # Disable auto-start Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: fix symlink loop on image rebuildBruce Ashfield2026-02-092-6/+9
| | | | | | | | | | | | | Use ln -sfn instead of ln -sf to prevent creating symlinks inside existing symlink target directories. Without -n, if the convenience symlink (e.g., container-base-latest-oci) already exists and points to a directory, ln -sf creates the new symlink inside that directory rather than replacing it, resulting in a self-referencing symlink. This manifests as "Too many levels of symbolic links" errors when container-bundle tries to copy OCI images after a rebuild. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* docs: fix dead references to vdkr-native and obsolete test classesBruce Ashfield2026-02-091-1/+1
| | | | | | | | | Update references to reflect the current architecture: - Change vdkr-native/vpdmn-native to vcontainer-native in comments - Remove TestContainerCrossTools and TestContainerCrossInitramfs from README - Fix build command: vdkr-native → vcontainer-tarball Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* classes: factor out shared functions to container-common.bbclassBruce Ashfield2026-02-093-66/+58
| | | | | | | | | | | | | | Create container-common.bbclass with shared Python functions used by both container-bundle.bbclass and container-cross-install.bbclass: - get_vruntime_multiconfig(d): Maps TARGET_ARCH to multiconfig name - get_vruntime_machine(d): Maps TARGET_ARCH to MACHINE for deploy path - get_blob_arch(d): Maps TARGET_ARCH to blob directory name This removes ~55 lines of duplicated code and ensures consistency between the two bbclass files. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-cross-install: add bbclass for bundling containers into imagesBruce Ashfield2026-02-091-0/+911
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This class enables bundling pre-built OCI containers into Yocto images at build time. It uses vdkr/vpdmn to process containers via QEMU, producing properly formatted storage that can be merged into the target rootfs. Two mechanisms for bundling containers: 1. BUNDLED_CONTAINERS variable (direct specification): - Set in local.conf or image recipe - Format: "name:runtime[:autostart][:external]" - Dependencies auto-generated at parse time - Example: BUNDLED_CONTAINERS = "container-base:docker:autostart" 2. container-bundle packages (package-based): - Recipes inherit container-bundle.bbclass to create packages - Installing packages via IMAGE_INSTALL triggers processing - merge_installed_bundles() scans installed OCI directories - Runs vrunner once in batch-import mode for efficiency Automatic dependency generation for BUNDLED_CONTAINERS: - Parses entries at recipe parse time via python __anonymous() - Derives recipe name from OCI dir names (strips -latest-oci/-oci suffix) - Generates do_rootfs[depends] on recipe:do_image_complete - Use :external tag to skip dependency for third-party blobs - Single "bitbake container-image-host" builds containers + image Usage in image recipe: inherit container-cross-install Features: - Automatic detection of target architecture - Support for both Docker and Podman runtimes - Batch import mode for efficiency with multiple containers - Autostart support via systemd services (Docker) or Quadlet (Podman) - Integration with vrunner for QEMU-based processing - Backwards compatible with legacy -latest-oci naming convention The class processes containers during do_rootfs and merges the resulting storage into /var/lib/docker or /var/lib/containers. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-bundle: add package-based container bundling supportBruce Ashfield2026-02-091-0/+441
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This class creates installable packages that bundle pre-processed container images. When installed via IMAGE_INSTALL, containers are automatically merged into the target image's container storage. Component relationships for bundling a local container: 1. Application Recipe (builds the software) recipes-demo/myapp/myapp_1.0.bb - Compiles application binaries - Creates installable package (myapp) 2. Container Image Recipe (creates OCI image containing the app) recipes-demo/images/myapp-container.bb - inherit image image-oci - IMAGE_INSTALL = "myapp" - Produces: ${DEPLOY_DIR_IMAGE}/myapp-container-latest-oci/ 3. Bundle Recipe (packages container images for deployment) recipes-demo/bundles/my-bundle_1.0.bb - inherit container-bundle - CONTAINER_BUNDLES = "myapp-container:autostart" - Creates installable package with OCI data Flow: application recipe -> container image recipe -> bundle recipe -> IMAGE_INSTALL in host image -> container deployed on target Usage: inherit container-bundle CONTAINER_BUNDLES = "myapp-container:autostart redis-container" CONTAINER_BUNDLES format: source[:autostart-policy] - source: Container IMAGE recipe name or remote registry URL - autostart-policy: Optional (autostart, always, unless-stopped, on-failure) Features: - Auto-generates dependencies on container image recipes (do_image_complete) - Supports remote containers via skopeo (requires CONTAINER_DIGESTS) - Runtime auto-detected from CONTAINER_PROFILE (docker/podman) - Produces OCI directories and metadata for container-cross-install Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-fetcher: fix shallow clone handling, duplicates, and discovery workflowBruce Ashfield2026-01-041-2/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | oe-go-mod-fetcher.py: - Remove BB_GIT_SHALLOW_EXTRA_REFS generation - refs must be present in ALL repositories which isn't the case for module dependencies. Instead, use tag= parameter in individual SRC_URI entries. - Add tag=<tagname> to SRC_URI when ref is a tag, allowing BitBake's shallow clone to include the necessary tag (with BB_GIT_SHALLOW=1) - Remove premature _ref_points_to_commit() check that was clearing ref_hints before repos were fetched, preventing tag= from being added - Fix pseudo-version verification: only use shallow fetch for actual tags (refs/tags/...), not branch refs. Pseudo-versions with branch refs (refs/heads/...) now correctly use unshallow path to reach historical commits that aren't fetchable with depth=1 oe-go-mod-fetcher-hybrid.py: - Fix duplicate SRC_URI entries when multiple modules share the same git repo/commit (e.g., errdefs and errdefs/pkg). Track added vcs_hashes to skip duplicates. - Add --discovery-cache option to calculate module sizes from discovery cache .zip files, enabling size recommendations during discover_and_generate go-mod-discovery.bbclass: - Add automatic hybrid mode recommendations after generate_modules, showing module sizes and suggested --git prefixes for conversion - Add GO_MOD_DISCOVERY_SKIP_VERIFY variable to skip commit verification on retries (useful after fixing verification issues) - Pass --discovery-cache to hybrid script for accurate size calculations Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod: introduce hybrid modeBruce Ashfield2025-12-081-0/+224
| | | | | | | | | | | | | | | | | | | | | | Introduce the ability to have hybrid gomod:// and git:// repositories. This allows SRCREV bumping when fixing bugs, and using the git archiver for some, all or none of the modules in a go mod project. Example: k3s Hybrid Conversion 1. Ensure VCS mode works first bitbake k3s 2. Get recommendations bitbake k3s -c go_mod_recommend 3. Convert with recommended prefixes (keep containerd, k8s.io as git://) python3 ./meta-virtualization/scripts/oe-go-mod-fetcher-hybrid.py \ --recipedir ./meta-virtualization/recipes-containers/k3s/ \ --git "github.com/containerd,k8s.io,sigs.k8s.io,github.com/rancher" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod: make extracted files writeableBruce Ashfield2025-12-081-0/+13
| | | | | | | | Make extracted files writable so BitBake can clean them later Go's module cache is read-only by design, but this breaks rm -rf and our subsquent task runs will fail. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-vcs: declare cache defaultsBruce Ashfield2025-12-081-2/+9
| | | | | | | | | | There are variables which are common to all recipes, declare defaults to make recipe integration simpler. Also, when creating the cache the messages are normal about synthesis, etc, so reduce them to bb.note. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-vcs: missing go.mod is not a warningBruce Ashfield2025-12-081-1/+2
| | | | Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-discovery: support multi-patckage buildBruce Ashfield2025-12-081-1/+5
| | | | | | | When building multiple packages (./... or multiple targets), go build requires the output to be a directory. Create the directory and use it. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-discovery: move patching after discovery buildBruce Ashfield2025-12-081-2/+5
| | | | | | | | | | | | When updating a SRCREV and then running discovery, we expect that some patches will fail. We don't want that to block discovery as these patches are not normally ADDING dependencies that would be found. So we move discovery before patch and patch failures can be dealt with on a full build. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* discover: manage go.* filesBruce Ashfield2025-12-082-0/+27
| | | | | | | | | | | | | | The discovery repository can be in an incosistent state after a build. By restoring the go.* files, we ensure a stable start point. Add automatic removal of go.sum files from git-fetched dependencies in vcs_cache during do_create_module_cache. This prevents checksum mismatch errors caused by stale go.sum files in dependencies having different checksums than the git-sourced modules. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-discovery: split into more tasksBruce Ashfield2025-12-081-214/+307
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To make it easier to run individual components of the go module discovery and generation process, we add invidual tasks as well as a combined task: AVAILABLE TASKS: bitbake <recipe> -c discover_modules Build project and download modules from proxy.golang.org This populates the discovery cache but does NOT extract or generate bitbake <recipe> -c extract_modules Extract module metadata from discovery cache to modules.json Requires: discover_modules to have been run first bitbake <recipe> -c generate_modules Generate go-mod-git.inc and go-mod-cache.inc from modules.json Requires: extract_modules to have been run first bitbake <recipe> -c discover_and_generate Run all three steps: discover -> extract -> generate This is the "do everything" convenience task bitbake <recipe> -c show_upgrade_commands Show copy-pasteable command lines without running anything bitbake <recipe> -c clean_discovery Remove the persistent discovery cache Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-05-25 08:48:52 +0000