Skip to content

feat(channels): ship librefang-sidecar-telegram binary in release tarballs#5937

Merged
houko merged 2 commits into
mainfrom
feat/ship-rust-sidecar-binaries
May 31, 2026
Merged

feat(channels): ship librefang-sidecar-telegram binary in release tarballs#5937
houko merged 2 commits into
mainfrom
feat/ship-rust-sidecar-binaries

Conversation

@houko

@houko houko commented May 31, 2026

Copy link
Copy Markdown
Contributor

Closes #5936.

The Rust Telegram sidecar (#5831) shipped only as source, so using it meant installing a Rust toolchain, cloning the repo, building librefang-sidecar-telegram, and pointing a [[sidecar_channels]] block at the resulting path.
This bundles the binary inside the existing release tarballs so librefang update lands it at ~/.librefang/bin/ and the daemon auto-resolves it — making the migration off the Python sidecar a one-step config change.

Part 1 — Release build (bundle inside existing tarballs)

Both workflows change in lockstep (release-cli.yml is the manual mirror of release.yml's CLI jobs).
For each channel-capable native target the in-scope jobs are cli_mac (x86_64 + aarch64), cli_linux (x86_64-gnu + aarch64-gnu[cross]), cli_musl (x86_64 + aarch64, both cross), cli_windows (x86_64 + aarch64).
Per job:

  • A new "Build Telegram sidecar" step builds the sidecar for the same target with --manifest-path sdk/rust/librefang-sidecar-telegram/Cargo.toml (using cross where that job already uses cross — i.e. all of cli_musl, and the aarch64 leg of cli_linux).
  • The Package step copies the sidecar binary (from its separate-workspace output sdk/rust/librefang-sidecar-telegram/target/<target>/release/) next to librefang and adds it to the same archive.
    macOS ad-hoc codesigns it like librefang; cli_musl verifies it is statically linked; Windows adds librefang-sidecar-telegram.exe to the Compress-Archive path list.

No new release assets are introduced — the binary ships inside the existing per-target librefang-<target>.tar.gz / .zip, so EXPECTED_PLATFORMS=12 and the cosign SHA256SUMS manifest are unchanged.

Deliberately skipped: the cli_android job and all mini variants (Android and mini carry no channels).

Part 2 — Install scripts (extract the bundled binary)

  • install.sh (POSIX sh): after installing librefang, if librefang-sidecar-telegram is present in the extracted tarball, chmod +x it (and on macOS strip quarantine + ad-hoc codesign). Absent ⇒ skipped silently (backward compatible with older tarballs).
  • install.ps1: mirror — if librefang-sidecar-telegram.exe is present, copy it to $InstallDir; absent ⇒ skipped.

Part 3 — Daemon auto-resolution

resolve_sidecar_command(command, home_dir) in crates/librefang-channels/src/sidecar.rs, called once in SidecarAdapter::new (where home_dir is already available, so the resolved command flows into both the adapter field and the spawn SpawnCtx):

  • Eligible only when command is empty or the bare stem librefang-sidecar-telegram (no path component).
  • Resolution order, first hit wins: the daemon's own executable directory (current_exe().parent()), then <home_dir>/bin/, then fall through to the original command (PATH lookup — historical behavior).
  • An absolute / relative path, or any other program (python3, uv, ./librefang-sidecar-telegram, …), passes through unchanged so explicit operator intent always wins.

Three focused unit tests cover: resolves to <home>/bin when the bundled binary is present (bare stem and empty command); falls through unchanged when no bundled binary exists; leaves explicit commands untouched even when a bundled binary is on disk.

Part 4 — Docs + CHANGELOG

  • docs/architecture/rust-telegram-sidecar.md and docs/src/app/configuration/channels/page.mdx now state the binary ships in release tarballs, lands in ~/.librefang/bin/ via librefang update, and is auto-resolved — manual build no longer required. Added an "Auto-resolution" section documenting the search order.
  • CHANGELOG.md entry added under the existing [Unreleased] ### Added.

Verification

Run via the repo's Dockerfile.rust-dev image (no native cargo on the host; Linux container, scoped commands only):

  • cargo test -p librefang-channels --lib482 passed, 0 failed (includes the 3 new resolve_sidecar_command_* tests, confirmed passing individually).
  • cargo build --release --manifest-path sdk/rust/librefang-sidecar-telegram/Cargo.toml → compiles clean, binary produced (~8 MB).
  • cargo clippy -p librefang-channels --all-targets -- -D warnings → clean.
  • cargo fmt -p librefang-channels -- --check → clean.
  • sh -n web/public/install.sh → syntax OK (shellcheck not available on the host).
  • release.yml / release-cli.yml validated as YAML.

Not locally verifiable — needs CI on a real tag

The release pipeline itself cannot be exercised locally (no tag, no GitHub Actions runners, and cargo build for the kernel is forbidden in-session).
The workflow edits — the cross-compiled sidecar build steps, the cross --manifest-path invocation for the separate workspace, the macOS codesign / musl static-link / Windows zip packaging, and the assertion that asset names and the EXPECTED_PLATFORMS=12 / cosign manifest are unchanged — must be validated by CI on a real tag (or a release-cli manual dispatch).
The sidecar build was proven to compile for the container's Linux target only; per-target cross builds (aarch64, musl, Windows, macOS) are CI's responsibility.

Update — PR-time CI guard (post-review)

Added a PR-time guard so the sidecar's compilability and the cross-compile risk are no longer deferred to CI-on-tag.

  • ci.yml: new changes output sidecar_rust (set when sdk/rust/librefang-sidecar{,-telegram}/ or the release workflows change), feeding a new rust-telegram-sidecar job modeled on the existing skills-wasm-sdk job.
    The job runs cargo fmt --check + clippy -D warnings + test + build --release on the sidecar workspace, then cargo install cross + cross build --release --target aarch64-unknown-linux-musl --manifest-path sdk/rust/librefang-sidecar-telegram/Cargo.toml.
    That cross step really runs cross on this PR, proving up front whether cross can mount and resolve the out-of-workspace ../librefang-sidecar path dependency — the exact risk the original PR could only flag as "needs CI-on-tag validation".
    rust-cache lists both sdk/rust/librefang-sidecar-telegram and the sibling sdk/rust/librefang-sidecar.
  • release.yml + release-cli.yml: the four native sidecar-building jobs (mac / linux / musl / windows) now cache the sidecar's separate target/ by listing . and sdk/rust/librefang-sidecar-telegram under rust-cache workspaces: (mini / android untouched).

If the cross step fails on this PR's CI, that is the real answer we couldn't get locally — it would mean the sidecar workspace needs a Cross.toml / mount tweak so cross can see ../librefang-sidecar (a follow-up).

Local verification (Docker dev image) of the native rust-telegram-sidecar step: cargo fmt --check clean, clippy --all-targets -- -D warnings clean, cargo test 41 passed, cargo build --release OK.
All three workflow YAMLs pass yaml.safe_load and actionlint.
The cross-musl step is intentionally CI-only.

The Rust Telegram sidecar (#5831) shipped only as source, so operators had
to install a Rust toolchain and compile it before pointing a
[[sidecar_channels]] block at the result. Bundle the binary in the release
tarballs and auto-resolve it so the migration off the Python sidecar is a
one-step config change.

Release pipeline: for each channel-capable native target (macOS x86_64 /
aarch64, linux-gnu x86_64 / aarch64, musl x86_64 / aarch64, Windows x86_64 /
aarch64) release.yml and its manual mirror release-cli.yml compile the sidecar
from its separate cargo workspace and bundle the binary inside the existing
per-target archive next to librefang. No new release assets, so the cosign
SHA256SUMS manifest and EXPECTED_PLATFORMS=12 are unchanged. macOS ad-hoc
codesigns the sidecar; musl verifies it is statically linked; Windows adds
the .exe to the Compress-Archive path list. Android and the mini variants
are deliberately skipped (neither carries channels).

Install scripts: install.sh and install.ps1 install the bundled binary when
present and stay silent on older tarballs that lack it (backward compatible).

Daemon: resolve_sidecar_command in librefang-channels resolves an empty or
bare-stem command to the daemon's own executable directory, then
~/.librefang/bin/, then PATH. An absolute / relative path or any other
program (python3 -m ...) is treated as explicit operator intent and passed
through unchanged.

Docs + CHANGELOG updated.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying librefang with  Cloudflare Pages  Cloudflare Pages

Latest commit: 26da2e7
Status: ✅  Deploy successful!
Preview URL: https://92f8987d.librefang.pages.dev
Branch Preview URL: https://feat-ship-rust-sidecar-binar.librefang.pages.dev

View logs

@houko

houko commented May 31, 2026

Copy link
Copy Markdown
Contributor Author

⚠️ Highest-priority CI-on-tag risk: cross + sibling path dependency

The release-pipeline changes cannot be exercised by this PR's CI (release.yml only triggers on tag push). The single thing to validate before/at the next release:

librefang-sidecar-telegram is its own cargo workspace at sdk/rust/librefang-sidecar-telegram/, and it path-depends on a sibling separate workspace: librefang-sidecar = { path = "../librefang-sidecar" } (i.e. sdk/rust/librefang-sidecar/, which is outside the telegram workspace root).

For the cross targets (aarch64-unknown-linux-gnu, x86_64/aarch64-unknown-linux-musl), cross build --manifest-path sdk/rust/librefang-sidecar-telegram/Cargo.toml runs in a container that mounts the workspace root. Whether cross also mounts the sibling ../librefang-sidecar path dep is version-dependent; if it doesn't, the cross builds fail with an unresolved path dependency. The repo-root Cross.toml does not apply here (cross resolves config relative to the built workspace, not the repo root).

Native targets (mac x86_64/aarch64, linux-gnu x86_64, windows x86_64/aarch64) are not affected — the sidecar was confirmed to compile on the host Linux target.

De-risk without cutting a release

Trigger release-cli.yml via workflow_dispatch on an existing tag (its stated purpose) — it mirrors these exact build steps and will surface a cross failure on the sibling path dep without publishing a new release.

Fallback if cross fails on the path dep

Give sdk/rust/librefang-sidecar-telegram/ its own Cross.toml that mounts the parent (sdk/rust/), or set CROSS_CONTAINER_OPTS to mount the sibling, or vendor librefang-sidecar into the telegram workspace for the cross builds.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

@github-actions github-actions Bot added area/docs Documentation and guides area/channels Messaging channel adapters area/ci CI/CD and build tooling size/L 250-999 lines changed ready-for-review PR is ready for maintainer review labels May 31, 2026
The librefang-sidecar-telegram binary the release pipeline ships is its own
cargo workspace, not a member of the kernel workspace, so the regular CI
RUST lane never compiles it — a break would only surface when cutting a
tag. This PR makes it release-blocking, so move the verification forward to
every PR.

ci.yml: route changes under sdk/rust/librefang-sidecar{,-telegram}/ and the
release workflows to a new rust-telegram-sidecar job (modeled on the
existing skills-wasm-sdk job). It fmt/clippy/test/builds the sidecar
natively and then runs the cross build for aarch64-unknown-linux-musl that
the release pipeline depends on, proving at PR time that cross can resolve
the out-of-workspace ../librefang-sidecar path dependency — the one risk
that previously could only be validated on a real tag.

release.yml + release-cli.yml: the four native jobs that build the sidecar
(mac / linux / musl / windows) now list both . and
sdk/rust/librefang-sidecar-telegram under Swatinem/rust-cache workspaces so
the sidecar's separate target/ is cached instead of recompiled from scratch
each run. The mini and android jobs are untouched (they do not build the
sidecar).
@houko
houko merged commit f772ddf into main May 31, 2026
35 checks passed
@houko
houko deleted the feat/ship-rust-sidecar-binaries branch May 31, 2026 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/channels Messaging channel adapters area/ci CI/CD and build tooling area/docs Documentation and guides ready-for-review PR is ready for maintainer review size/L 250-999 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ship librefang-sidecar-telegram binary in release assets

1 participant