[nu-demo] hero_voice's ort-sys 2.0.0-rc.12 requires ONNX Runtime 1.24+ but our canonical install is 1.23.2 (what hero_embedder uses) #173
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Symptom
After resolving the libclang-dev + cmake prerequisites (home#169, home#172),
cargo build -p hero_voice_serverstill fails:Verbose debug shows ort-sys falls into a
doing full static linking since no single-file library was foundpath after failing to match against the installed shared libs.Root cause
hero_embedderusesort = "2.0.0-rc.11"which works with ONNX Runtime 1.23.x (per ort-rs matrix).hero_voice's whisper-rs (and/or silero-vad) pullsort-sys = "2.0.0-rc.12"which its Cargo manifest declaresdescription = "Unsafe Rust bindings for ONNX Runtime 1.24"— i.e. it expects 1.24, not 1.23.Installing ONNX 1.23.2 system-wide (what we did for the embedder) doesn't satisfy ort-sys-rc.12. Installing 1.24 system-wide would in turn break the embedder (ort-rc.11 wants 1.23).
The underlying conflict: two sibling Hero services pin different ort-sys RC versions that target different ONNX binary versions.
Fixes (ordered by reversibility)
1. Pin both services to the same ort-sys RC
Preferred. Either:
ort = "2.0.0-rc.12"(and ship ONNX 1.24) — needs embedder retest.2. Ship TWO ONNX installs side-by-side
less pretty:
/usr/local/onnxruntime-1.23/liband/usr/local/onnxruntime-1.24/lib, setORT_LIB_LOCATIONper-service in the hero_proc action env. hero_embedder points at 1.23, hero_voice at 1.24. Works but doubles install footprint.3. Let ort-sys download its own ONNX blob
ort-sys-rc.12 has a
download-binariesfeature (enabled viaCARGO_FEATURE_DOWNLOAD_BINARIES=1in the verbose log). Theoretically it'll fetch 1.24 binaries at build time. On herodemo this path also failed — likely because the cargo sandbox / network context at build time doesn't have unrestricted outbound, or the download URL is broken.4. Drop whisper-rs, use a remote STT service
hero_voice's transcription could route through hero_aibroker's
/v1/audio/transcriptionsendpoint (Groq Whisper) instead of local whisper.cpp. Eliminates the ONNX 1.24 dependency entirely. Adds a network hop but removes ~100 MB of C++ compile + local models.Demo impact
hero_voice is not critical for the Hero OS core demo (AI assistant + books + office + business all work without it). Voice is a nice-to-have. For herodemo we're shipping without it and flagging clearly.
Demo workaround (herodemo 2026-04-24)
None applied — hero_voice binary missing, service inactive, voice island shows "Waiting for hero_voice" in dock.
Related
Signed-off-by: mik-tf
Demo hotfix applied 2026-04-24
Both ONNX runtimes coexist on herodemo:
/usr/local/onnxruntime/→ 1.23.2 (used byhero_embedderd)/usr/local/onnxruntime-1.24/→ 1.24.4 (used byhero_voice)hero_voice_serverandhero_voice_uiaction env patched with:ORT_PREFER_DYNAMIC_LINK=1is the key flag — without it, ort-sys rc.12 attempts static linking and errors out even with the correctORT_LIB_LOCATION. Per-process LD_LIBRARY_PATH ensures embedder and voice each load their own version.Verified: hero_voice running, both
rpc.sockandui.sockcreated, UI loads at https://herodemo.gent01.grid.tf/hero_voice/ui/.Prod-level fix needed
The double-install is workable but not ideal:
ortcrate version across hero_voice, hero_embedder, and any future ONNX consumers. Choose either 2.0.0-rc.11 (pin embedder + voice both there) or rc.12 (bump embedder to 1.24-compatible model loader). Prefer rc.12 since it's newer and has bugfixes.ORT_PREFER_DYNAMIC_LINK=1requirement in hero_skills' service_voice.nu installer + add it to the canonical action env spec.Deploy-side resolved:
install_onnxruntimenow installs ONNX 1.23.2 + 1.24.4 side-by-side, and service_voice/service_embedder env vars route each binary to the right runtime. Full ort crate unification across hero_embedder + hero_voice (single ort version) is still pending — keeping open.Part of:
lhumina_code/hero_skills@7c823d1(PR lhumina_code/hero_skills#126).Tracker: #185.
Resolved
Landed across three repos on
development(squash commits):hero_voicefc89cec — pinort = "=2.0.0-rc.11"withload-dynamic+ndarrayfeatures in workspace.dependencies; consume as direct dep incrates/hero_voice/Cargo.tomlto anchor the resolution. Cargo.lock downgraded to rc.11.hero_skills0a561f9 —install_onnxruntimetrims to a single ONNX 1.23.2 download;service_voice.nuORT_LIB_LOCATION→/usr/local/onnxruntime/lib(same as service_embedder).hero_demo9b5bc6e — runbook §2.3 single-ONNX line; §13 drops the home#173 row.Real root cause
The issue framed it as "two services pin different ort-sys RC versions" — true symptom, but
whisper-rs(which the issue named) does not pullortat all. It useswhisper-rs-sys0.15 → bindgen + cmake on whisper.cpp directly, no ONNX dependency. The actualortconsumer in hero_voice iskokoro-micro1.0.0 (TTS).Kokoro's manifest declares
ort = "2.0.0-rc.11"— same RC hero_embedder pins. The drift to rc.12 was purely cargo's caret-semver auto-upgrade. A single exact-pin inhero_voice/Cargo.tomlworkspace plus a direct consume from one crate is enough to anchor the resolution to rc.11. No fork of whisper-rs / silero-vad needed.Verification on herodemo (2026-04-26)
cargo build -p hero_voice_server(release, 6m04s on herodemo)service_voice start --resetregisters actionsORT_LIB_LOCATION=/usr/local/onnxruntime/lib(1.23.2, same as embedder)hero_proc service list | grep hero_voicecurl /health(rpc + ui sockets)GET /tts/voicesPOST /tts{"text":"Hello world","voice":"af_heart","speed":1.0}hero_embedderstill running on the same ONNX installEffort vs. estimate
Runbook §13 estimated multi-day cross-repo work assuming whisper-rs forking would be needed. Actual: ~1 hour from plan to live verification. The misattribution of
ortto whisper-rs was the false complexity multiplier.Signed-off-by: mik-tf
We should have 1.24 for embedder instead of going down to 1.23
home#173 follow-up — flipped direction: keep ONNX 1.24 across embedder + voice
Kristof's call on the previous comment ("We should have 1.24 for embedder instead of going down to 1.23") landed. Reverting the temporary unification down to rc.11/ONNX 1.23 and instead pulling hero_voice up to rc.12/ONNX 1.24, so both services share the modern stack.
Why the rc.12 bump looked scary at first
rc.12 introduced two source-level breaks for callers:
api-XXcargo feature is now mandatory. rc.12's EP modules (e.g.vitis.rs) referenceOrtApifields gated behind#[cfg(feature = "api-18")]and above. Without an explicitapi-XXfeature, cargo errors withE0609("no field … on type &OrtApi"). Fix: add"api-24"(matches ort's owndefault = [..., "api-24"]) to ort's feature list.ort::Error<T>is no longerSend + Syncfor FFI-bound inner types likeNonNull<OrtSessionOptions>. The?operator can't auto-convert these intoanyhow::Error(which requiresSend + Sync). Fix: stringify at the boundary using the canonical anyhow idiom —.map_err(|e| anyhow::anyhow!("{e}"))?.Neither is a bug — both are defensible upstream design choices (explicit API-version opt-in; not lying about thread-safety of FFI handles). The migration cost on our side was 2 source sites in
hero_embedder_lib. No code on the hero_voice side needed any change beyond the version pin (whisper-rs and silero-vad already target rc.12 natively; kokoro-micro 1.0.0's caret-pinnedort = "2.0.0-rc.11"resolves up to rc.12 fine — semver allows pre-release escalation within2.0.0).Changes
Three repos, all on
development_mikand pushed:f24619bort = "=2.0.0-rc.12", comment block explaining the rc.11→rc.12 escalation across whisper-rs + silero-vad + kokoro-microe3e4d68ort = "=2.0.0-rc.12"+"api-24"feature;.map_err(|e| anyhow::anyhow!("{e}"))?migration inembedder.rsandreranker.rs62af07ainstall_onnxruntime1.23.2 → 1.24.4 with VERSION_NUMBER detection so existing 1.23.2 installs auto-upgrade instead of silently skippingFiles touched on the embedder side (the only non-trivial fix)
crates/hero_embedder_lib/src/ml/embedder.rs:43-48— Session::builder chain now uses.map_err(\|e\| anyhow::anyhow!("{e}"))?for the two builder steps that returnedort::Error<NonNull<OrtSessionOptions>>.crates/hero_embedder_lib/src/ml/reranker.rs:29-34— Same migration, same chain.Both sites have a comment block referencing this issue so future readers don't go re-investigating the same thing.
Verification
cargo fmt --check✓,cargo clippy --workspace --all-targets -- -D warnings✓,cargo checkon_lib+_server+_embedderd✓.nuparse-checked.Outstanding
development_mikbranches →development(per workflow gate).hero_skillsand confirm bothhero_embedderandhero_voicestart cleanly against the same/usr/local/onnxruntime/(1.24.4) so the original sharing motivation is preserved.Keeping this issue open until the merges land + herodemo is verified on the new shared 1.24 stack.
Signed-off-by: mik-tf
home#173 follow-up — landed on development across all three repos
Squash-merged 2026-04-27:
8bd278baab92214ec144dAll three branches verified end-to-end on herodemo before merging:
restarts: 0)install_onnxruntimeversion-detection branch fired correctly on herodemo's existing 1.23.2 install:Replacing ONNX Runtime unknown → 1.24.4Follow-ups surfaced during the herodemo verification — tracked separately, not blocking this issue:
install_onnxruntimeitself:^sudoassumption breaks on hosts where driver has no sudo — needs a$env.USER == "root"guard. Will land in a separate hero_skills patch.Closing this issue once #197 (the cosmetic kokoro audio item that was flagged by the original commenter) is at least triaged.
Signed-off-by: mik-tf
Meta close-out
The rc.12 / ONNX 1.24 follow-up surfaced 4 separate issues during the herodemo verification — all 4 now have fixes landed on
development:lhumina_code/kokoro-microwith style-indexing fix; hero_voice pinned to fork; upstream PR https://github.com/DavidValin/kokoro-micro/pull/2ensure_shell_initin hero_skillsinstall_coreis the root-cause fix (.profile → .bashrc bridge); broker re-seeded on herodemoenv_service.rsnow uses/hero_osis/ui/config/env; needs hero_os WASM rebuild + hero_demo redeployClosing #173 itself — the original ort/ONNX unification is done, the modern stack is alive on herodemo, and the cascade of follow-ups is documented above. #199 stays open until the next hero_demo redeploy verifies the WASM fix in-browser.
Signed-off-by: mik-tf