linux/meta-virtualization.git/classes/go-mod-discovery.bbclass, branch master

oe-go-mod-fetcher: add license scanning for Go module dependencies

2026-04-28T18:49:19+00:00

Add --scan-licenses to oe-go-mod-fetcher.py which scans Go module zips for license files and generates go-mod-licenses.inc with LICENSE and LIC_FILES_CHKSUM entries matching OE-core's go-mod-update-modules format. License detection uses OE-core's glob patterns and MD5 + crunched MD5 matching against known SPDX licenses. The hash database resolves from: 1. --common-license-dir (explicit path) 2. Auto-detected poky tree common-licenses 3. Bundled scripts/data/license-hashes.csv (offline fallback) New files: - scripts/generate-license-hashes.py: regenerate bundled CSV - scripts/data/license-hashes.csv: pre-computed hash DB (704 entries) bbclass changes: - go-mod-discovery: pass --scan-licenses during do_generate_modules - GO_MOD_DISCOVERY_SKIP_LICENSES variable to bypass scanning - do_update_license_hashes task to refresh bundled CSV Co-Authored-By: Claude Opus 4.6 (1M context)

go-mod-discovery/oe-go-mod-fetcher: add GO_MOD_VCS_EXCLUDE for deleted repos

2026-03-19T15:56:06+00:00

Some upstream Go module repositories get deleted from GitHub (e.g., github.com/vtolstov/go-ioctl). While the Go module proxy still serves cached archives, VCS mode cannot git clone a deleted repo. This causes both do_fetch failures and generator verification failures. Add GO_MOD_VCS_EXCLUDE recipe variable (space-separated module path prefixes) and corresponding --exclude-module CLI flag. Excluded modules are filtered out before verification and SRC_URI generation. Recipes must provide a gomod:// SRC_URI entry for excluded modules as fallback. Usage in recipe: SRC_URI += "gomod://example.com/deleted-repo;version=v1.0.0;sha256sum=..." GO_MOD_VCS_EXCLUDE = "example.com/deleted-repo" Signed-off-by: Bruce Ashfield

go-mod-fetcher: fix shallow clone handling, duplicates, and discovery workflow

2026-01-04T16:40:15+00:00

oe-go-mod-fetcher.py: - Remove BB_GIT_SHALLOW_EXTRA_REFS generation - refs must be present in ALL repositories which isn't the case for module dependencies. Instead, use tag= parameter in individual SRC_URI entries. - Add tag= to SRC_URI when ref is a tag, allowing BitBake's shallow clone to include the necessary tag (with BB_GIT_SHALLOW=1) - Remove premature _ref_points_to_commit() check that was clearing ref_hints before repos were fetched, preventing tag= from being added - Fix pseudo-version verification: only use shallow fetch for actual tags (refs/tags/...), not branch refs. Pseudo-versions with branch refs (refs/heads/...) now correctly use unshallow path to reach historical commits that aren't fetchable with depth=1 oe-go-mod-fetcher-hybrid.py: - Fix duplicate SRC_URI entries when multiple modules share the same git repo/commit (e.g., errdefs and errdefs/pkg). Track added vcs_hashes to skip duplicates. - Add --discovery-cache option to calculate module sizes from discovery cache .zip files, enabling size recommendations during discover_and_generate go-mod-discovery.bbclass: - Add automatic hybrid mode recommendations after generate_modules, showing module sizes and suggested --git prefixes for conversion - Add GO_MOD_DISCOVERY_SKIP_VERIFY variable to skip commit verification on retries (useful after fixing verification issues) - Pass --discovery-cache to hybrid script for accurate size calculations Signed-off-by: Bruce Ashfield

go-mod-discovery: support multi-patckage build

2025-12-09T01:57:44+00:00

When building multiple packages (./... or multiple targets), go build requires the output to be a directory. Create the directory and use it. Signed-off-by: Bruce Ashfield

go-mod-discovery: move patching after discovery build

2025-12-09T01:57:44+00:00

When updating a SRCREV and then running discovery, we expect that some patches will fail. We don't want that to block discovery as these patches are not normally ADDING dependencies that would be found. So we move discovery before patch and patch failures can be dealt with on a full build. Signed-off-by: Bruce Ashfield

discover: manage go.* files

2025-12-09T01:57:44+00:00

The discovery repository can be in an incosistent state after a build. By restoring the go.* files, we ensure a stable start point. Add automatic removal of go.sum files from git-fetched dependencies in vcs_cache during do_create_module_cache. This prevents checksum mismatch errors caused by stale go.sum files in dependencies having different checksums than the git-sourced modules. Signed-off-by: Bruce Ashfield

go-mod-discovery: split into more tasks

2025-12-09T01:57:44+00:00

To make it easier to run individual components of the go module discovery and generation process, we add invidual tasks as well as a combined task: AVAILABLE TASKS: bitbake -c discover_modules Build project and download modules from proxy.golang.org This populates the discovery cache but does NOT extract or generate bitbake -c extract_modules Extract module metadata from discovery cache to modules.json Requires: discover_modules to have been run first bitbake -c generate_modules Generate go-mod-git.inc and go-mod-cache.inc from modules.json Requires: extract_modules to have been run first bitbake -c discover_and_generate Run all three steps: discover -> extract -> generate This is the "do everything" convenience task bitbake -c show_upgrade_commands Show copy-pasteable command lines without running anything bitbake -c clean_discovery Remove the persistent discovery cache Signed-off-by: Bruce Ashfield

classes: add go-mod-vcs and go-mod-discovery for Go module builds

2025-12-09T01:57:44+00:00

Add two new bbclass files that enable building Go applications using git-based module resolution instead of network proxy fetches: go-mod-vcs.bbclass: - Provides do_create_module_cache task to build GOMODCACHE from git sources - Implements pure Python h1: hash calculation with go-dirhash-native fallback - Creates properly structured module zip files and hash files - Handles module path transformations and case encoding go-mod-discovery.bbclass: - Runs module discovery using the oe-go-mod-fetcher tool - Generates go-mod-git.inc and go-mod-cache.inc files - Supports bootstrap mode for initial recipe conversion Together these classes enable fully offline, reproducible Go builds by fetching module sources via git and constructing the module cache during the build rather than relying on network access to module proxies. Signed-off-by: Bruce Ashfield