diff options
| author | Bruce Ashfield <bruce.ashfield@gmail.com> | 2026-06-01 15:47:50 +0000 |
|---|---|---|
| committer | Bruce Ashfield <bruce.ashfield@gmail.com> | 2026-06-01 20:01:42 +0000 |
| commit | aa529792ad56263480fd506883d96a67f074433a (patch) | |
| tree | 29028b93c0ddd21e90a751b5b158e40b659cd557 /scripts | |
| parent | c7dfb08a3e6c3b00a80dcd34b5b017e5d6c6b41e (diff) | |
| download | meta-virtualization-aa529792ad56263480fd506883d96a67f074433a.tar.gz | |
ceph: convert from gitsm:// to explicit git:// submodules
The bitbake gitsm:// fetcher recursively reads .gitmodules at the
recipe's SRCREV and tries to fetch each submodule from its declared
upstream URL at the pinned SHA. Ceph regularly pins submodule commits
that upstream subsequently loses (force-pushes, branch reaps, fork
sync drops), so do_fetch dies with errors like:
ERROR: Fetcher failure: Unable to find revision
c196241ae89b1aa4f62efeb849a937c011b3a926 in branch even from
upstream
for src/isa-l. Hit on the v20.3.0-tip bump where ceph's main updated
isa-l (commit a46428a1fd2 "isa-l: Update isa-l submodule from 2.30.0
to 2.32.0", March 2026) to point at c196241a — a commit that doesn't
exist in github.com/ceph/isa-l's published refs. The old recipe's
pin bee5180a is also not in upstream; previous builds got away with
it because the commit happened to be in the local DL_DIR clone from
some earlier (and now-untraceable) fetch.
Exploding gitsm:// into explicit git:// SRC_URI entries — one per
submodule — gives the recipe direct control over each component's
SRCREV and URL. When a pin rots, fix it here without waiting for
upstream to restore the commit. The pattern follows the upx recipe:
SRCREV_FORMAT names the canonical SRCREV, and per-component
SRCREV_<name> values match the name= parameter on each SRC_URI entry.
Submodule names are short-form sanitized last-path-elements (e.g.
"src/erasure-code/jerasure/gf-complete" -> "gf_complete").
This commit also resolves the build-time fallout from the
conversion. Each item below was hit on a successive build pass:
- isa-l URL switched from ceph/isa-l to intel/isa-l. The orphaned
SRCREV c196241a is exactly the v2.32.0 tag in intel's upstream;
ceph's main updated its submodule pin to that release without
syncing the ceph/isa-l fork. Repointing at intel/isa-l (where
the SHA actually lives) is the fix; the SRCREV stays unchanged.
- jerasure, gf-complete, googletest, and libkmip switched from
branch=<X> (as declared in .gitmodules) to nobranch=1. The
.gitmodules `branch =` field is a tracking hint for
`git submodule update --remote`, not a validation: the pinned
SHAs frequently sit on a tag or different branch than what
.gitmodules names. bitbake's strict branch=X mode requires the
SHA be reachable from that ref and bails otherwise. nobranch=1
tells bitbake "trust the rev, don't validate refs", which is the
right mode for SHA-pinned submodules.
- Added s3select's two nested submodules (rapidjson, csvparser).
gitsm fetched these recursively; the explicit form needs them
listed. Without them, configure fails with:
Imported target "RapidJSON::RapidJSON" includes non-existent
path ".../src/s3select/rapidjson/include"
- liburing added to DEPENDS and WITH_LIBURING flipped from OFF to
ON (with WITH_SYSTEM_LIBURING=ON to use meta-oe's package
instead of ceph's bundled build). The cephfs-tool target added
on the v20.3.0-tip line unconditionally links uring::uring with
no WITH_LIBURING guard, so disabling liburing leaves an undefined
CMake target. Using the system package is cleaner than carrying
a patch to gate that link line.
- python3-pip-native added to DEPENDS. Ceph's
src/pybind/rados/cmake_install.cmake invokes
`python3 -m pip install --use-pep517 --no-build-isolation
--no-deps --ignore-installed <local-source-path>` to build and
install the cython rados/rgw/rbd/cephfs Python bindings. The
invocation is fully hermetic (the trailing arg is a directory,
not a PyPI package name; --no-build-isolation and --no-deps
block any network access), but pip itself isn't part of the
bare python3-native sysroot, so it has to be pulled in
explicitly.
- Added 0001-cmake-Distutils-include-CMAKE_C_COMPILER_ARG1-in-PY_-
*.patch. cmake/modules/Distutils.cmake's
distutils_install_cython_module function builds PY_CC from only
${CMAKE_C_COMPILER}, dropping ${CMAKE_C_COMPILER_ARG1}. When
OECMAKE_C_COMPILER is set to "<gcc-bin> --sysroot=<path>" (the
standard OE cross-compile pattern), CMake splits it into
COMPILER + COMPILER_ARG1, and the --sysroot ends up in ARG1.
Distutils.cmake then emits a `set(ENV{CC} "<bare-gcc>")` line in
the generated cmake_install.cmake, and the pip-driven setuptools
cython build invokes the target cross-gcc with no sysroot —
failing with `fatal error: assert.h: No such file or directory`
when Python.h tries to include the C standard library. The
patch mirrors what distutils_add_cython_module (the build-time
counterpart in the same file) already does: appends
${c_compiler_arg1} to the PY_CC / PY_LDSHARED constructions.
- SYSTEMD_SERVICE entry ceph-exporter.service updated to
ceph-exporter@.service. Upstream renamed the unit on the
v20.3.0-tip line (the `.in` template moved from
ceph-exporter.service.in -> ceph-exporter@.service.in to make it
a systemd template unit). The other 20 services in the list
were cross-checked against the installed unit set and match.
- do_install:append shebang sed extended to include
${D}${sbindir}/ceph-node-proxy. This new tool (added on the
v20.3.0-tip line) is pip-installed as a console script, which
bakes the absolute path of python3-native into its shebang line.
Without the rewrite, do_package_qa fails with both a
[buildpaths] (TMPDIR reference in shipped binary) and a
[file-rdeps] (the recipe sysroot's python3-native path can't be
resolved at install time on the target) error.
- Added a find/rm for direct_url.json and dependency_links.txt
under ${D}. These are PEP 610 / setuptools metadata files that
pip writes inside each installed package's .dist-info or
.egg-info directory, recording the file:// URL the package was
installed from. With our local in-tree source layout that URL
is the WORKDIR build path, which trips do_package_qa
[buildpaths]. The files are pure introspection metadata
consumed only by `pip show --verbose` and similar tools; baking
the build host's path into the target image serves no purpose,
so they're removed at install time.
SRCREVs below are from `git submodule status` at the ceph v20.3.0-tip
SRCREV 6830c0390c2. Branches in .gitmodules are not always reliable
indicators of where the pinned SHA lives (see jerasure/gf-complete
notes above); favor nobranch=1 on uprev unless a follow-up build
confirms branch=X is correct.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Diffstat (limited to 'scripts')
0 files changed, 0 insertions, 0 deletions
