summaryrefslogtreecommitdiffstats
path: root/classes
Commit message (Collapse)AuthorAgeFilesLines
* container-cross-install: add CONTAINER_SERVICE_FILE supportBruce Ashfield2026-02-062-1/+236
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for custom systemd service files (Docker) or Quadlet container files (Podman) instead of auto-generated ones for container autostart. For containers requiring specific startup configuration (ports, volumes, capabilities, dependencies), users can now provide custom service files using the CONTAINER_SERVICE_FILE varflag: CONTAINER_SERVICE_FILE[container-name] = "${UNPACKDIR}/myservice.service" For BUNDLED_CONTAINERS in image recipes: SRC_URI += "file://myapp.service" BUNDLED_CONTAINERS = "myapp-container:docker:autostart" CONTAINER_SERVICE_FILE[myapp-container] = "${UNPACKDIR}/myapp.service" For container-bundle packages: SRC_URI = "file://myapp.service" CONTAINER_BUNDLES = "myapp-container:autostart" CONTAINER_SERVICE_FILE[myapp-container] = "${UNPACKDIR}/myapp.service" Implementation: - container-cross-install.bbclass: Add get_container_service_file_map() to build varflag map, install_custom_service() for BUNDLED_CONTAINERS, and install_custom_service_from_bundle() for bundle packages - container-bundle.bbclass: Install custom service files to ${datadir}/container-bundles/${runtime}/services/ Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add host layer type and delta-only copyingBruce Ashfield2026-02-052-14/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add two enhancements to multi-layer OCI image support: 1. Delta-only copying for directories/files layers: - directories and files layers now only copy content that doesn't already exist in the bundle rootfs from earlier layers - Prevents duplication when a directories layer references paths that were already populated by a packages layer - Logs show "delta: N copied, M skipped" for visibility 2. New 'host' layer type for build machine content: - Copies files from the build machine filesystem (outside Yocto) - Format: name:host:source_path:dest_path - Multiple pairs: name:host:src1:dst1+src2:dst2 - Emits warning at parse time about reproducibility impact - Fatal error if source path doesn't exist - Use case: deployment-specific config, certificates, keys that cannot be packaged in recipes Example: OCI_LAYERS = "\ base:packages:busybox \ app:directories:/opt/myapp \ certs:host:/etc/ssl/certs/ca.crt:/etc/ssl/certs/ca.crt \ " Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer-tarball: build all architectures via single bitbake commandBruce Ashfield2026-02-051-0/+5
| | | | | | | | | | | | | | | | | | | | Previously, building vcontainer-tarball required multiple bitbake invocations or complex command lines to build both x86_64 and aarch64 blobs. This was a usability issue. Changes: - mcdepends now triggers builds for BOTH architectures automatically - VCONTAINER_ARCHITECTURES defaults to "x86_64 aarch64" (was auto-detect) - Add informational banner at parse time showing what will be built - Fix duplicate sanity check messages when multiconfig is active Usage is now simply: bitbake vcontainer-tarball To build only one architecture, set in local.conf: VCONTAINER_ARCHITECTURES = "x86_64" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer: add sanity checks and auto-enable virtfs for QEMUBruce Ashfield2026-01-231-2/+4
| | | | | | | | | | | | | | | | | Fix virtio-9p (virtfs) support for container-cross-install batch imports which provides ~50x speedup over base64-over-serial. The issue was that native recipes don't see target DISTRO_FEATURES, so qemu-system-native wasn't getting virtfs enabled. Fix by: - layer.conf: Propagate virtualization to DISTRO_FEATURES_NATIVE when vcontainer or virtualization is in target DISTRO_FEATURES - qemu-system-native: Check DISTRO_FEATURES_NATIVE for virtfs enable - container-cross-install: Prepend native sysroot to PATH so vrunner finds the QEMU with virtfs support Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer: add sanity checks and auto-enable virtfs for QEMUBruce Ashfield2026-01-211-0/+36
| | | | | | | | | | | Add sanity check that warns when vcontainer distro feature is enabled but BBMULTICONFIG is missing the required vruntime-* multiconfigs. Add qemu-system-native bbappend to auto-enable virtfs (virtio-9p) when vcontainer or virtualization distro feature is set. This is required for the fast batch-import path in container-cross-install. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer: add multi-arch OCI supportBruce Ashfield2026-01-211-0/+243
| | | | | | | | | | | | | | | | | | | | | Add functions to detect and handle multi-architecture OCI Image Index format with automatic platform selection during import. Also add oci-multiarch.bbclass for build-time multi-arch OCI creation. Runtime support (vcontainer-common.sh): - is_oci_image_index() - detect multi-arch OCI images - get_oci_platforms() - list available platforms - select_platform_manifest() - select manifest for target architecture - extract_platform_oci() - extract single platform to new OCI dir - normalize_arch_to_oci/from_oci() - architecture name mapping - Update vimport to auto-select platform from multi-arch images Build-time support (oci-multiarch.bbclass): - Create OCI Image Index from multiconfig builds - Collect images from vruntime-aarch64, vruntime-x86-64 - Combine blobs and create unified manifest list Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-cross-install: fix image naming and default runtimeBruce Ashfield2026-01-211-17/+56
| | | | | | | | | | | | Fix extract_container_info() to properly handle multi-part container names and add automatic runtime detection based on CONTAINER_PROFILE. - Fix multi-part name parsing (app-container-multilayer-latest-oci now correctly becomes app-container-multilayer:latest) - Add CONTAINER_DEFAULT_RUNTIME from CONTAINER_PROFILE - Add CONTAINER_IMPORT_TIMEOUT_BASE/PER for dynamic timeout scaling Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add layer caching for multi-layer OCI buildsBruce Ashfield2026-01-212-2/+201
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add layer caching to speed up multi-layer OCI image rebuilds. When enabled, pre-installed package layers are cached to disk and restored on subsequent builds, avoiding repeated package installation. New variables: - OCI_LAYER_CACHE: Enable/disable caching (default "1") - OCI_LAYER_CACHE_DIR: Cache location (default ${TOPDIR}/oci-layer-cache/${MACHINE}) Cache key is computed from: - Layer name and type - Sorted package list - Package versions from PKGDATA_DIR - MACHINE and TUNE_PKGARCH Cache automatically invalidates when: - Package versions change - Layer definition changes - Architecture changes Benefits: - First build: ~10-30s per layer (cache miss, packages installed) - Subsequent builds: ~1s per layer (cache hit, files copied) - Shared across recipes with identical layer definitions Build log shows cache status: NOTE: OCI Cache HIT: Layer 'base' (be88c180f651416b) NOTE: OCI: Pre-installed packages for 3 layers (cache: 3 hits, 0 misses) Also adds comprehensive pytest suite for multi-layer OCI functionality including tests for 1/2/3 layer modes and cache behavior. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add multi-layer OCI image support with OCI_LAYERSBruce Ashfield2026-01-212-23/+365
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for creating multi-layer OCI images with explicit layer definitions via OCI_LAYERS variable. This enables fine-grained control over container layer composition. New variables: - OCI_LAYER_MODE: Set to "multi" for explicit layer definitions - OCI_LAYERS: Define layers as "name:type:content" entries - packages: Install specific packages in a layer - directories: Copy directories from IMAGE_ROOTFS - files: Copy specific files from IMAGE_ROOTFS Package installation uses Yocto's package manager classes (RpmPM, OpkgPM) for consistency with do_rootfs, rather than calling dnf/opkg directly. Example usage: OCI_LAYER_MODE = "multi" OCI_LAYERS = "\ base:packages:base-files+base-passwd+netbase \ shell:packages:busybox \ app:packages:curl \ " This creates a 3-layer OCI image with discrete base, shell, and app layers that can be shared and cached independently. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-bundle: add CONTAINER_BUNDLE_DEPLOY for base layer useBruce Ashfield2026-01-211-0/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | Add CONTAINER_BUNDLE_DEPLOY variable to enable dual-use of container-bundle: 1. Target packages (existing): Creates installable packages for target container storage (Docker/Podman) 2. Base layer source (new): When CONTAINER_BUNDLE_DEPLOY = "1", also deploys the fetched OCI image to DEPLOY_DIR_IMAGE for use as a base layer via OCI_BASE_IMAGE This enables fetching external images (docker.io, quay.io) and using them as base layers for Yocto-built container images. Example usage: # recipes-containers/oci-base-images/alpine-oci-base_3.19.bb inherit container-bundle CONTAINER_BUNDLES = "docker.io/library/alpine:3.19" CONTAINER_DIGESTS[docker.io_library_alpine_3.19] = "sha256:..." CONTAINER_BUNDLE_DEPLOY = "1" # Then in your app container recipe: OCI_BASE_IMAGE = "alpine-oci-base" IMAGE_INSTALL = "myapp" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add multi-layer OCI support and CMD defaultBruce Ashfield2026-01-212-15/+395
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for multi-layer OCI images, enabling base + app layer builds: Multi-layer support: - Add OCI_BASE_IMAGE variable to specify base layer (recipe name or path) - Add OCI_BASE_IMAGE_TAG for selecting base image tag (default: latest) - Resolve base image type (recipe/path/remote) at parse time - Copy base OCI layout before adding new layer via umoci repack - Fix merged-usr whiteout ordering issue for non-merged-usr base images (replaces problematic whiteouts with filtered entries to avoid Docker pull failures when layering merged-usr on traditional layout) CMD/ENTRYPOINT behavior change: - Add OCI_IMAGE_CMD variable (default: "/bin/sh") - Change OCI_IMAGE_ENTRYPOINT default to empty string - This makes `docker run image /bin/sh` work as expected (like Docker Hub images) - OCI_IMAGE_ENTRYPOINT_ARGS still works for legacy compatibility - Fix shlex.split() for proper shell quoting in CMD/ENTRYPOINT values The multi-layer feature requires umoci backend (default). The sloci backend only supports single-layer images and will error if OCI_BASE_IMAGE is set. Example usage: OCI_BASE_IMAGE = "container-base" IMAGE_INSTALL = "myapp" OCI_IMAGE_CMD = "/usr/bin/myapp" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: add build-time metadata labels for traceabilityBruce Ashfield2026-01-212-1/+81
| | | | | | | | | | | | | | | | | | | | | | | Automatically embed source and build information into OCI images using standard OCI annotations (opencontainers.org image-spec): - org.opencontainers.image.revision: git commit SHA - org.opencontainers.image.ref.name: git branch name - org.opencontainers.image.created: ISO 8601 build timestamp - org.opencontainers.image.version: PV (if meaningful) New variables: - OCI_IMAGE_REVISION: explicit SHA override (auto-detects from TOPDIR) - OCI_IMAGE_BRANCH: explicit branch override (auto-detects from TOPDIR) - OCI_IMAGE_BUILD_DATE: explicit timestamp override (auto-generated) - OCI_IMAGE_APP_RECIPE: hook for future cross-recipe extraction Set any variable to "none" to disable that specific label. This enables 1:1 traceability between container images and source code, following industry best practices for CI/CD and release management. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-registry: add industry-standard tag strategiesBruce Ashfield2026-01-211-4/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | Add comprehensive tag support for registry push operations: Tag strategies (CONTAINER_REGISTRY_TAG_STRATEGY): - sha/git: short git commit hash for traceability - branch: git branch name (sanitized) for dev workflows - semver: nested SemVer tags (1.2.3 -> 1.2.3, 1.2, 1) - timestamp: YYYYMMDD-HHMMSS format - version: single version tag from PV - latest: the "latest" tag - arch: append architecture suffix Helper script enhancements: - push --tag <tag>: explicit tags (repeatable) - push --strategy <strategies>: override tag strategy - push --version <ver>: version for semver strategy - Baked-in defaults from bitbake variables - Environment variable overrides supported This aligns with industry practices: - Git SHA for CI/CD traceability - SemVer nested tags for release management - Branch tags for feature development Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-registry: add local OCI registry infrastructureBruce Ashfield2026-01-211-0/+203
| | | | | | | | | | | Add container registry support for Yocto container workflows: - container-registry.bbclass with helper functions - container-registry-index.bb generates helper script with baked paths - docker-registry-config.bb for Docker daemon on targets - container-oci-registry-config.bb for Podman/Skopeo/Buildah targets - IMAGE_FEATURES container-registry for easy target configuration Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* vcontainer: add auto-start daemon with idle timeoutBruce Ashfield2026-01-211-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | Add automatic daemon startup and idle timeout cleanup for vdkr/vpdmn: - vmemres daemon auto-starts on first command (no manual start needed) - Daemon auto-stops after idle timeout (default: 30 minutes) - --no-daemon flag for ephemeral mode (single-shot QEMU) - New config keys: idle-timeout, auto-daemon Changes: - vcontainer-init-common.sh: Parse idle_timeout from cmdline, add read -t timeout to daemon loop for auto-shutdown - vrunner.sh: Add --idle-timeout option, pass to kernel cmdline - vcontainer-common.sh: Auto-start logic in run_runtime_command(), --no-daemon flag, config defaults - container-cross-install.bbclass: Add --no-daemon for explicit ephemeral mode during Yocto builds Configuration: vdkr vconfig idle-timeout 3600 # 1 hour timeout vdkr vconfig auto-daemon false # Disable auto-start Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci: fix symlink loop on image rebuildBruce Ashfield2026-01-212-6/+9
| | | | | | | | | | | | | Use ln -sfn instead of ln -sf to prevent creating symlinks inside existing symlink target directories. Without -n, if the convenience symlink (e.g., container-base-latest-oci) already exists and points to a directory, ln -sf creates the new symlink inside that directory rather than replacing it, resulting in a self-referencing symlink. This manifests as "Too many levels of symbolic links" errors when container-bundle tries to copy OCI images after a rebuild. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* docs: fix dead references to vdkr-native and obsolete test classesBruce Ashfield2026-01-211-1/+1
| | | | | | | | | Update references to reflect the current architecture: - Change vdkr-native/vpdmn-native to vcontainer-native in comments - Remove TestContainerCrossTools and TestContainerCrossInitramfs from README - Fix build command: vdkr-native → vcontainer-tarball Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* classes: factor out shared functions to container-common.bbclassBruce Ashfield2026-01-213-66/+58
| | | | | | | | | | | | | | Create container-common.bbclass with shared Python functions used by both container-bundle.bbclass and container-cross-install.bbclass: - get_vruntime_multiconfig(d): Maps TARGET_ARCH to multiconfig name - get_vruntime_machine(d): Maps TARGET_ARCH to MACHINE for deploy path - get_blob_arch(d): Maps TARGET_ARCH to blob directory name This removes ~55 lines of duplicated code and ensures consistency between the two bbclass files. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-cross-install: add bbclass for bundling containers into imagesBruce Ashfield2026-01-211-0/+911
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This class enables bundling pre-built OCI containers into Yocto images at build time. It uses vdkr/vpdmn to process containers via QEMU, producing properly formatted storage that can be merged into the target rootfs. Two mechanisms for bundling containers: 1. BUNDLED_CONTAINERS variable (direct specification): - Set in local.conf or image recipe - Format: "name:runtime[:autostart][:external]" - Dependencies auto-generated at parse time - Example: BUNDLED_CONTAINERS = "container-base:docker:autostart" 2. container-bundle packages (package-based): - Recipes inherit container-bundle.bbclass to create packages - Installing packages via IMAGE_INSTALL triggers processing - merge_installed_bundles() scans installed OCI directories - Runs vrunner once in batch-import mode for efficiency Automatic dependency generation for BUNDLED_CONTAINERS: - Parses entries at recipe parse time via python __anonymous() - Derives recipe name from OCI dir names (strips -latest-oci/-oci suffix) - Generates do_rootfs[depends] on recipe:do_image_complete - Use :external tag to skip dependency for third-party blobs - Single "bitbake container-image-host" builds containers + image Usage in image recipe: inherit container-cross-install Features: - Automatic detection of target architecture - Support for both Docker and Podman runtimes - Batch import mode for efficiency with multiple containers - Autostart support via systemd services (Docker) or Quadlet (Podman) - Integration with vrunner for QEMU-based processing - Backwards compatible with legacy -latest-oci naming convention The class processes containers during do_rootfs and merges the resulting storage into /var/lib/docker or /var/lib/containers. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* container-bundle: add package-based container bundling supportBruce Ashfield2026-01-211-0/+441
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This class creates installable packages that bundle pre-processed container images. When installed via IMAGE_INSTALL, containers are automatically merged into the target image's container storage. Component relationships for bundling a local container: 1. Application Recipe (builds the software) recipes-demo/myapp/myapp_1.0.bb - Compiles application binaries - Creates installable package (myapp) 2. Container Image Recipe (creates OCI image containing the app) recipes-demo/images/myapp-container.bb - inherit image image-oci - IMAGE_INSTALL = "myapp" - Produces: ${DEPLOY_DIR_IMAGE}/myapp-container-latest-oci/ 3. Bundle Recipe (packages container images for deployment) recipes-demo/bundles/my-bundle_1.0.bb - inherit container-bundle - CONTAINER_BUNDLES = "myapp-container:autostart" - Creates installable package with OCI data Flow: application recipe -> container image recipe -> bundle recipe -> IMAGE_INSTALL in host image -> container deployed on target Usage: inherit container-bundle CONTAINER_BUNDLES = "myapp-container:autostart redis-container" CONTAINER_BUNDLES format: source[:autostart-policy] - source: Container IMAGE recipe name or remote registry URL - autostart-policy: Optional (autostart, always, unless-stopped, on-failure) Features: - Auto-generates dependencies on container image recipes (do_image_complete) - Supports remote containers via skopeo (requires CONTAINER_DIGESTS) - Runtime auto-detected from CONTAINER_PROFILE (docker/podman) - Produces OCI directories and metadata for container-cross-install Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-fetcher: fix shallow clone handling, duplicates, and discovery workflowBruce Ashfield2026-01-041-2/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | oe-go-mod-fetcher.py: - Remove BB_GIT_SHALLOW_EXTRA_REFS generation - refs must be present in ALL repositories which isn't the case for module dependencies. Instead, use tag= parameter in individual SRC_URI entries. - Add tag=<tagname> to SRC_URI when ref is a tag, allowing BitBake's shallow clone to include the necessary tag (with BB_GIT_SHALLOW=1) - Remove premature _ref_points_to_commit() check that was clearing ref_hints before repos were fetched, preventing tag= from being added - Fix pseudo-version verification: only use shallow fetch for actual tags (refs/tags/...), not branch refs. Pseudo-versions with branch refs (refs/heads/...) now correctly use unshallow path to reach historical commits that aren't fetchable with depth=1 oe-go-mod-fetcher-hybrid.py: - Fix duplicate SRC_URI entries when multiple modules share the same git repo/commit (e.g., errdefs and errdefs/pkg). Track added vcs_hashes to skip duplicates. - Add --discovery-cache option to calculate module sizes from discovery cache .zip files, enabling size recommendations during discover_and_generate go-mod-discovery.bbclass: - Add automatic hybrid mode recommendations after generate_modules, showing module sizes and suggested --git prefixes for conversion - Add GO_MOD_DISCOVERY_SKIP_VERIFY variable to skip commit verification on retries (useful after fixing verification issues) - Pass --discovery-cache to hybrid script for accurate size calculations Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod: introduce hybrid modeBruce Ashfield2025-12-081-0/+224
| | | | | | | | | | | | | | | | | | | | | | Introduce the ability to have hybrid gomod:// and git:// repositories. This allows SRCREV bumping when fixing bugs, and using the git archiver for some, all or none of the modules in a go mod project. Example: k3s Hybrid Conversion 1. Ensure VCS mode works first bitbake k3s 2. Get recommendations bitbake k3s -c go_mod_recommend 3. Convert with recommended prefixes (keep containerd, k8s.io as git://) python3 ./meta-virtualization/scripts/oe-go-mod-fetcher-hybrid.py \ --recipedir ./meta-virtualization/recipes-containers/k3s/ \ --git "github.com/containerd,k8s.io,sigs.k8s.io,github.com/rancher" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod: make extracted files writeableBruce Ashfield2025-12-081-0/+13
| | | | | | | | Make extracted files writable so BitBake can clean them later Go's module cache is read-only by design, but this breaks rm -rf and our subsquent task runs will fail. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-vcs: declare cache defaultsBruce Ashfield2025-12-081-2/+9
| | | | | | | | | | There are variables which are common to all recipes, declare defaults to make recipe integration simpler. Also, when creating the cache the messages are normal about synthesis, etc, so reduce them to bb.note. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-vcs: missing go.mod is not a warningBruce Ashfield2025-12-081-1/+2
| | | | Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-discovery: support multi-patckage buildBruce Ashfield2025-12-081-1/+5
| | | | | | | When building multiple packages (./... or multiple targets), go build requires the output to be a directory. Create the directory and use it. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-discovery: move patching after discovery buildBruce Ashfield2025-12-081-2/+5
| | | | | | | | | | | | When updating a SRCREV and then running discovery, we expect that some patches will fail. We don't want that to block discovery as these patches are not normally ADDING dependencies that would be found. So we move discovery before patch and patch failures can be dealt with on a full build. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* discover: manage go.* filesBruce Ashfield2025-12-082-0/+27
| | | | | | | | | | | | | | The discovery repository can be in an incosistent state after a build. By restoring the go.* files, we ensure a stable start point. Add automatic removal of go.sum files from git-fetched dependencies in vcs_cache during do_create_module_cache. This prevents checksum mismatch errors caused by stale go.sum files in dependencies having different checksums than the git-sourced modules. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* go-mod-discovery: split into more tasksBruce Ashfield2025-12-081-214/+307
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To make it easier to run individual components of the go module discovery and generation process, we add invidual tasks as well as a combined task: AVAILABLE TASKS: bitbake <recipe> -c discover_modules Build project and download modules from proxy.golang.org This populates the discovery cache but does NOT extract or generate bitbake <recipe> -c extract_modules Extract module metadata from discovery cache to modules.json Requires: discover_modules to have been run first bitbake <recipe> -c generate_modules Generate go-mod-git.inc and go-mod-cache.inc from modules.json Requires: extract_modules to have been run first bitbake <recipe> -c discover_and_generate Run all three steps: discover -> extract -> generate This is the "do everything" convenience task bitbake <recipe> -c show_upgrade_commands Show copy-pasteable command lines without running anything bitbake <recipe> -c clean_discovery Remove the persistent discovery cache Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* classes: add go-mod-vcs and go-mod-discovery for Go module buildsBruce Ashfield2025-12-082-0/+1548
| | | | | | | | | | | | | | | | | | | | | | Add two new bbclass files that enable building Go applications using git-based module resolution instead of network proxy fetches: go-mod-vcs.bbclass: - Provides do_create_module_cache task to build GOMODCACHE from git sources - Implements pure Python h1: hash calculation with go-dirhash-native fallback - Creates properly structured module zip files and hash files - Handles module path transformations and case encoding go-mod-discovery.bbclass: - Runs module discovery using the oe-go-mod-fetcher tool - Generates go-mod-git.inc and go-mod-cache.inc files - Supports bootstrap mode for initial recipe conversion Together these classes enable fully offline, reproducible Go builds by fetching module sources via git and constructing the module cache during the build rather than relying on network access to module proxies. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci-unmoci: preserve file modesBruce Ashfield2025-01-171-1/+1
| | | | | | | | | | | | | | | | | Based on the following patch: Subject: [meta-virtualization][PATCH] image-oci-umoci: preserve file modes in rootfs From: "Florian Wickert via lists.yoctoproject.org" <FlorianWickert=gmail.com@lists.yoctoproject.org> cp does not preserve file modes by default. This will break sudo (among others) which needs the suid flag to do its work. This patch adds the --preserve=mode flag to the cp call to fix this. Signed-off-by: Florian Wickert <FlorianWickert@gmail.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* xen: enable networking and guest image bundlingBruce Ashfield2024-11-151-0/+26
| | | | | | | | | | | | | | | | | | The xen host image reference needed signifant work to be functional for launching and testing Xen domu guests. Here we add additional tools to the host image, and allow it to automatically bundle guests if the configuration is enabled. We also add systemd networking configuration to create a xenbr0 which offeres connectivity to the entire reference system. See the recipes and the README for details on testing and bundling. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* qemuboot: add note about virt_networking bbclassBruce Ashfield2024-11-151-0/+3
| | | | Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* classes: add depreciated warning bbclassBruce Ashfield2024-03-151-0/+3
| | | | | | | Adding a simple bbclass that when inherited appends do_compile to emit a warning that the recipe will be removed in the future. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* classes/image-oci: Map image architecture correctlyJoshua Watt2023-11-061-1/+1
| | | | | | | | | | | OCI requires that the architecture [1] be a valid GOARCH [2]. To correctly perform this mapping, use the go library code from OE core. [1]: https://github.com/opencontainers/image-spec/blob/main/config.md#properties [2]: https://go.dev/doc/install/source#environment Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* classes/image-oci-umoci: Replace ":" in image file nameJoshua Watt2023-11-061-3/+7
| | | | | | | | | | | | | The OCI_IMAGE_TAG variable can contain a ":" if the user wants to give the image a name and a tag, as in: OCI_IMAGE_TAG = "${IMAGE_BASENAME}:latest" However, while this is valid for tag name, the ":" is illegal in OCI image file names so replace it with "_" when naming the symlinks Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* classes/image-oci-umoci: Allow labels to have spacesJoshua Watt2023-11-061-2/+2
| | | | | | | | | | | Quote the label argument passed to umoci to allow it to correctly handle spaces. Note that this requires that OCI_IMAGE_LABELS have the argument quoted with single quotes, as in: OCI_IMAGE_LABELS = "org.opencontainers.image.description='${SUMMARY}'" Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* classes/image-oci-umoci: Allow environment variables to have spacesJoshua Watt2023-11-061-2/+2
| | | | | | | | | | | Quote the environment variable argument passed to umoci to allow it to correctly handle spaces. Note that this requires that OCI_IMAGE_ENV_VARS have the argument quoted with single quotes, as in: OCI_IMAGE_ENV_VARS = "FOO='bar bat'" Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* umoci: allow encoding of multiple entrypoint or cmd argumentsBruce Ashfield2023-11-061-2/+3
| | | | | | | | | | | | umoci can encode both entrypoint and cmd lists into the oci image. This is done by using the --config.entrypoint and --config.cmd arguments multiple times. We can split our OCI_* variables on space and support multiple arguments in either command and keep compatibility with existing users. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* conf: make container recipes parseable when virtualization is not setBruce Ashfield2023-03-201-1/+5
| | | | | | | | | | | | | The container stack flexibilty features set defaults (like other parts of the layer) when 'virtualization' is in the distro features. That reqirement means that the recipes fail parsing and QA checks when the distro feature isn't enabled. The defaults are currently safe for a virtualization enabled and disabled configuration, so we include them in either case. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* conf: introduce container configuration valuesBruce Ashfield2023-03-081-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | From the configuration file itself: These variables represent groupings of functionality in the CNCF landscape. In particular, they are areas where there is a choice between more than one implementation or an area where abstraction is beneficial. The contents of the variables are are runtime components that recipes may use for RDEPENDS. Build dependencies are not typically flexible, so do not currently have DEPENDS equivalents for the components (i.e. DEPENDS on runc versus crun). Distro features such as kubernetes or other container stacks can be used to set different defaults for these variables. Note: these are "global" values, since they represent choices. If more than of a grouping is required on target, then the variable can be appended or set to multiple values. That being said, Recipes should generally agree on the values, hence the global namespace. Recipe specific choices can still be done, but they risk conflicting on target or causing runtime issues / errors. ## CNCF "components" # engines: docker-ce/docker-moby, virtual-containerd, cri-o, podman VIRTUAL-RUNTIME_container_engine ??= "podman" # runtime: runc, crun, runv, runx VIRTUAL-RUNTIME_container_runtime ??= "virtual-runc" # networking: cni, netavark VIRTUAL-RUNTIME_container_networking ??= "cni" # dns: cni, aardvark-dns VIRTUAL-RUNTIME_container_dns ??= "cni" # orchestration: k8s, k3s VIRTUAL-RUNTIME_container_orchestration ??= "k3s" ## Kubernetes terminology "components" VIRTUAL-RUNTIME_cri ??= "containerd" VIRTUAL-RUNTIME_cni ??= "cni" Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* containers: introduce container-host classBruce Ashfield2023-02-171-0/+15
| | | | | | | | | | | | | | | | Introducing a small (at the moment) class that represents configuration and processing required to prepare a target image to be a container host. A recipe that requires container configuration should inherit this class, and the container-host-config package will be added as a RDEPENDS, and install common configuration files. In the future, additional functionality or dependencies will be added here to synchronize the configuration of multiple container host packages. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* oci-image: create two different tar outputsBruce Ashfield2023-02-102-2/+28
| | | | | | | | | | | | | | | | | | | | | | | The original tar output of the oci image followed the sloci-image convention of putting the oci contents into a subdirectory in the tarball. This allows it to work directly with skopeo, etc, but it isn't the format that tools like podman expect in an oci-image tarball. We move the original format to have "-dir" in the name, and let the more simply named one be the oci-image format as expcted by various 3rd party tools 1) image_name.tar: compatible with oci tar format, blobs and rootfs are at the top level. Can load directly from something like podman 2) image_name-dir.tar: original format from meta-virt, is just a tar'd up oci image directory (compatible with skopeo :dir format) We also fix a bug in the sloci-image backend, where the sloci tar was removing the raw oci image directory leaving a dangling symlink. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci-umoci: add convenience symlink for oci image tarBruce Ashfield2022-12-071-0/+2
| | | | | | | | We have been symlinking the -oci image directory for a while, but we can also create a shorted convenience symlink for the tar'd version of that directory. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci-sloci: add convenience symlinks in deployBruce Ashfield2022-12-071-0/+7
| | | | | | | | To sync the sloci OCI image backend with the umoci variant, create shortened convenience symlinks for the image and image.tar in the deploy directory. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci-umoci: add convenience symlink to latest imageBruce Ashfield2022-09-121-0/+2
| | | | | | | | Symlink the date-time based image name, to just the imagename-oci. This makes scripting use of the deployed container much simpler. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci-umoci: add parameter for stop signalVasileios Anagnostopoulos2022-07-152-0/+5
| | | | | | | | Add an optional parameter that allows to override the stop signal that is used. Signed-off-by: Vasileios Anagnostopoulos <vasileios.anagnostopoulos@siemens.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* image-oci-umoci: properly handle tags other than latestPascal Bach2022-07-151-17/+17
| | | | | Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* xen: Disable highmem on qemuarmMichal Orzel2022-06-201-2/+4
| | | | | | | | | | | | | | | | | | By default, highmem option is enabled for machine types later than virt-2.12. This allows qemu to place devices and RAM in physical address space above 32-bits. This can cause issues as according to the documentation Xen supports up to 12GiB of physical address space. Recently the issue was observed using runqemu, that was causing Linux running on top of Xen to stall when trying to access ECAM space placed by qemu at 256GiB mark. Even though this issue is most probably related to QEMU and not Xen (the investigation showed that it can map ECAM space correctly), it is best to avoid such issues by disabling highmem on qemuarm. Signed-off-by: Michal Orzel <michal.orzel@arm.com> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com> Reviewed-by: Christopher Clark <christopher.w.clark@gmail.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
* Revert "xen: Disable PCI on qemuarm with Xen"Michal Orzel2022-06-201-22/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit fb0a7b97db4f419b82309f98a4944ae3330d1e2e. Justification: Commit fb0a7b97db4f removed the PCI node from the qemu device tree due to the issue when trying to access PCI configuration space that was causing Linux running on top of Xen to end up in an infinite loop. The investigation showed that the problem occurs due to qemu placing ECAM space at 256GiB mark. Even though Xen officially supports on arm32 up to 12GiB of physical address space, it is able to map it correctly for dom0. However, when Linux tries to access the ECAM space and the stage2 translation fault occurs, HPFAR register contains incorrect IPA. We can say it is incorrect because using hardware AT instruction or software lookup, we can obtain the correct IPA from the same VA. This can suggest that the problem is related to QEMU fault handling. Instead of removing the PCI node the follow-up commit will disable highmem option in qemu which will cause placing ECAM space in the 32-bit space. Signed-off-by: Michal Orzel <michal.orzel@arm.com> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com> Reviewed-by: Christopher Clark <christopher.w.clark@gmail.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-06-28 21:42:51 +0000