Summary

Cargo.toml's target-specific fastembed dependency for Linux/Windows enables both ort-download-binaries-rustls-tls and ort-load-dynamic together:

[target.'cfg(any(target_os = "windows", target_os = "linux"))'.dependencies.fastembed]
version = "5"
features = [
    "hf-hub-rustls-tls",
    "ort-download-binaries-rustls-tls",
    "ort-load-dynamic",
]

ort-sys's build script treats load-dynamic (which forwards to ort-sys/disable-linking) as an early-return condition — when it's set, the build script skips the download step entirely, on the theory that the consuming application will supply libonnxruntime.so itself at runtime (via ORT_DYLIB_PATH or next to the executable). So ort-download-binaries-rustls-tls is silently a no-op whenever ort-load-dynamic is also enabled — the two features are mutually exclusive in practice, not additive.

Since llm-kernel doesn't separately ship/install libonnxruntime.so, any consumer using embedding-fastembed on Linux gets a binary that can't find the ONNX Runtime shared library at runtime.

Impact — this manifests as a silent hang, not an error

Worse than a missing-library error: calling EmbeddingProvider::embed() (via LazyFastembedProvider) on a fresh Linux deployment doesn't fail cleanly — it deadlocks. Confirmed via strace//proc/PID/wchan in production: the calling thread blocks forever on futex_wait_queue at 0% CPU, no syscalls, no log output.

Root cause, best I can tell: LazyFastembedProvider (src/embedding/lazy.rs) defers loading to a background std::thread and has the caller Condvar::wait() for ModelState::Ready. If that background thread panics (e.g. ort::init() panicking because the dylib load failed) before it can transition the shared state and notify, the waiting caller thread blocks indefinitely — the panic is silently swallowed rather than surfacing as ModelState::Failed(..).

Reproduction

• Build for aarch64-unknown-linux-gnu (or -musl) with embedding-fastembed via cargo-zigbuild on a non-Linux host.
• Deploy just the resulting executable (no accompanying .so).
• Call .embed() for the first time → hangs forever instead of erroring.

What I verified as a workaround

Building a throwaway crate with ort = { features = ["download-binaries"] } without load-dynamic correctly downloads and statically links libonnxruntime.a at build time (cached under ~/Library/Caches/ort.pyke.io/dfbin/<target>/.../libonnxruntime.a on macOS) — fully self-contained binary, no runtime dylib resolution needed at all. This is the behavior we actually want.

Suggested fixes (either would help; the second is more important)

1. Drop ort-load-dynamic from the Linux/Windows fastembed feature set (or make it a separate, opt-in llm-kernel feature) so ort-download-binaries-rustls-tls actually does its job and statically links.
2. Regardless of (1): make LazyFastembedProvider's background loader panic-safe — wrap it so a panic during load transitions state to Failed(..) and notifies the Condvar (e.g. via a drop guard), so callers get a clear error instead of an infinite hang. This protects against any future ort/fastembed init failure mode, not just this one.

Happy to verify against a real deploy once a fix lands, like last time.