feat(channels)!: migrate gotify from in-process adapter to sidecar#5263
Merged
Conversation
BREAKING: the `[channels.gotify]` config block is no longer recognised. Gotify now runs as a `[[sidecar_channels]]` adapter (`python3 -m librefang.sidecar.adapters.gotify`, channel_type = "gotify"); the new Python adapter ships in `librefang-sdk` with full parity for inbound WebSocket subscribe + outbound REST publish. Same pattern as the ntfy (#5224) and telegram (#5241) migrations. Added sidecar: - `sdk/python/librefang/sidecar/adapters/gotify.py` — preserves the Rust adapter's runtime semantics. WebSocket `/stream` subscribe with the client token in the query string; JSON frames decoded to `(id, message, title, priority, appid)`; empty `message` skipped; `/`-prefixed text → `Command`, otherwise `Text`; sender derived from `title` (fallback `app-{appid}`); metadata carries title / priority / app_id; reconnect with exponential backoff (1s → 60s). REST `/message` publish posts `{title:"LibreFang", message, priority:5}` with the app token in the query string; chunked at 65535 chars suffixing `(i/N)` to the title. Optional `GOTIFY_ACCOUNT_ID` env var surfaces as the multi-bot routing key. - The WebSocket client is a hand-rolled RFC 6455 reader on `socket` + `ssl` — no third-party WS dependency, keeping the SDK zero-dep (matches ntfy / telegram). Server frames are read unmasked; client→server frames (only pong / close) carry a fresh 4-byte mask per frame as the spec requires. Fragmented messages are reassembled before yielding. - `sdk/python/tests/test_gotify_adapter.py` — 22 unit tests: URL/scheme normalization, required-env enforcement, JSON frame parsing including empty-message skip and bad-JSON, command vs text routing, sender fallback to app-id, message chunking with numbered titles, HTTP 500 / HTTPError surfacing, and a loopback-server end-to-end test that performs a real RFC 6455 handshake and asserts both text frames are yielded as decoded strings. Removed in-process gotify (full cascade): - `librefang-channels`: `src/gotify.rs` deleted (649 lines), `lib.rs` mod, `Cargo.toml` `channel-gotify` feature + `all-channels` membership, `channels-allowlist.txt` entry (so `cargo xtask channel-policy` permanently blocks any in-process gotify reintroduction). Stale comment references in `discord.rs` / `slack.rs` / `teams.rs` ("the gotify slice (PR #4447)") generalised to "the wiremock pattern" — the pattern is still in use, just no longer rooted in a gotify example. - `librefang-types`: `GotifyConfig` struct + Default + the `channels.gotify` field + default, validation hook. - `librefang-api`: `channel-gotify` feature + membership in `all-channels` / `all-channels-no-email`; channel_bridge `use` + spawn block + `check_channel` arm + `find_channel_info` arm + default-empty test assertion; `routes/channels.rs` ChannelMeta + the 4 match arms (`gotify_some` / `serde_to_value` / `len` / `ser`); `routes/config.rs` `ch!(gotify)`. - `librefang-kernel`: `channel_sender` `for_each_channel_field!` macro entry + name list. - `librefang-cli`: TUI `ChannelDef`, with the Notifications-section header generalised since gotify joins ntfy off-tree. - `docs/architecture/sidecar-channels.md`: update the long-tail backlog note to record gotify joining ntfy + telegram. The separate gotify push-notification provider (`push_provider = "gotify"`, used by device pairing under `crates/librefang-kernel/ src/pairing.rs`) is **unaffected** — different feature, deliberately left intact. Operator action required: # config.toml: remove [channels.gotify] server_url = "..." app_token_env = "GOTIFY_APP_TOKEN" client_token_env = "GOTIFY_CLIENT_TOKEN" # config.toml: add [[sidecar_channels]] name = "gotify" command = "python3" args = ["-m", "librefang.sidecar.adapters.gotify"] channel_type = "gotify" [sidecar_channels.env] GOTIFY_SERVER_URL = "https://gotify.example.com" GOTIFY_APP_TOKEN = "A..." # secrets.env via secrets system GOTIFY_CLIENT_TOKEN = "C..." # secrets.env via secrets system Verification: - `cd sdk/python && pytest`: 97 tests pass (22 new gotify, 75 pre-existing ntfy/telegram/runtime/protocol all green). - Loopback WebSocket handshake test exercises real RFC 6455 Sec-WebSocket-Accept computation and frame parsing. - `cargo check / clippy / test` deferred to CI — the host's cargo is a Docker wrapper with `rust:latest` not cached locally, and `cargo test --workspace` is forbidden by repo policy. - `kernel_config_schema.golden.json` left with 12 residual gotify refs — the matching test is `#[ignore]`'d on `main` (golden has drifted across earlier migrations); regen is a follow-up via `cargo test -p librefang-api --test config_schema_golden -- --ignored regenerate_golden --nocapture`.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Deploying librefang-docs with
|
| Latest commit: |
3cfb186
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://2e34008c.librefang-docs-web.pages.dev |
| Branch Preview URL: | https://feat-gotify-sidecar-migratio.librefang-docs-web.pages.dev |
Self-review follow-up to the gotify sidecar migration. The main commit deleted the in-process `GotifyConfig` / `GotifyAdapter` but left several docs pages still showing the old `[channels.gotify]` block and pointing at the deleted Rust adapter. Telegram's #5241 follow-up landed the same kind of redirect — mirror that pattern. - `docs/.../configuration/page.mdx` (en + zh): replace the old `[channels.gotify]` field reference with a redirect-style `[[sidecar_channels]]` example, mirroring the existing Telegram redirect. Also drop the Gotify row from the "validator-checked env vars" table — the validation hook was removed with `GotifyConfig`. - `docs/.../configuration/channels/page.mdx` (en + zh): same redirect. - `docs/.../integrations/channels/integrations/page.mdx` (en + zh): the "## Gotify" section was already stale before this PR — it used hypothetical `GOTIFY_URL` / `GOTIFY_TOKEN` env vars and a `[channels.gotify]` block whose `url_env` / `token_env` keys never matched the actual `GotifyConfig` shape. Rewritten to point at the sidecar with the correct `GOTIFY_SERVER_URL` / `GOTIFY_APP_TOKEN` / `GOTIFY_CLIENT_TOKEN` env var names. - `docs/.../security/integrity/page.mdx`, `docs/.../architecture/security/page.mdx` (en + zh): drop the `GotifyAdapter` row from the "Zeroizing<String> usage in in-process channel adapters" table — the struct is gone with this PR; the env vars themselves are now read by the Python sidecar process, which inherits the daemon's environment without needing in-Rust zeroizing. - `examples/custom-channel/README.md`: drop the `gotify` row from the "study these existing adapters" comparison table — `gotify.rs` no longer exists. The env-var tables under "Required Environment Variables" at `configuration/page.mdx` and `configuration/security/page.mdx` are deliberately kept — the env vars themselves are still in use, just read by the sidecar process rather than the deleted in-process adapter. No code changes; docs-only follow-up to #5263.
Round-2 self-review follow-up. Cap inbound WebSocket frame payloads at 1 MiB (MAX_FRAME_PAYLOAD). RFC 6455 allows up to 2^63 bytes via the 64-bit extended-length form; the previous unbounded implementation would faithfully try to read any length the server announced. A hostile or buggy gotify server could send a 64-bit length and force the sidecar to spin in `_recv_exact` allocating multi-exabyte buffers. The cap is well over Gotify's largest realistic payload (a 65 535-char message body wrapped in a JSON envelope sits under 100 KiB), so legitimate traffic is unaffected. Cap breach raises a RuntimeError, which the producer loop catches and treats as a transport error → reconnect with the existing exponential backoff. Three new tests close earlier coverage gaps: - `test_websocket_reader_responds_to_server_ping_with_pong`: drives the reader against a loopback server that sends `text → ping → text → close`. Asserts both text frames are yielded AND that the client sent back exactly one masked pong carrying the same payload as the ping, plus the close echo. Exercises the previously-untested control-frame branches. - `test_websocket_reader_rejects_oversized_frame`: server announces a 1 GiB 64-bit-length text frame; client must raise RuntimeError matching "exceeds cap" instead of trying to allocate. Locks in the new MAX_FRAME_PAYLOAD guard. - `test_account_id_surfaced_via_ready_event` / `test_no_account_id_when_unset`: assert that `GOTIFY_ACCOUNT_ID` reaches LibreFang via the base SidecarAdapter.ready_event() multi-bot routing key (and that an empty env var emits no `account_id` field) — closes the gap noted in the first review. Verification: `cd sdk/python && pytest` — 101 tests pass (4 new gotify + 22 prior gotify + 75 unchanged ntfy/telegram/runtime).
8 tasks
houko
added a commit
that referenced
this pull request
May 19, 2026
…5264) * feat(channels)!: migrate mastodon from in-process adapter to sidecar BREAKING: the `[channels.mastodon]` config block is no longer recognised. Mastodon now runs as a `[[sidecar_channels]]` adapter (`python3 -m librefang.sidecar.adapters.mastodon`, channel_type = "mastodon"); the new Python adapter ships in `librefang-sdk` with full parity for SSE primary + REST polling fallback inbound and form-encoded REST outbound. Same pattern as the ntfy (#5224), telegram (#5241), and gotify (#5263) migrations. Added sidecar: - `sdk/python/librefang/sidecar/adapters/mastodon.py` — preserves the Rust adapter's runtime semantics. SSE subscribe to `{instance}/api/v1/streaming/user` with Bearer auth; filter `event: notification` payloads to `type == "mention"`; strip HTML from `status.content` (`<br>` / `</p>` / `</div>` / `</li>` insert newlines, entities decoded via stdlib `html.unescape`); `/cmd args` → Command, else Text; sender from `display_name` fallback `username`; metadata carries status_id / notification_id / acct / visibility / in_reply_to_id; `thread_id` = in_reply_to_id; `verify_credentials` at startup discovers the bot's own account id so self-mentions are skipped; REST polling fallback when SSE fails (newest-first ordering, high-water-mark before iteration); exponential-backoff reconnect (1s → 60s). - Outbound: POST `/api/v1/statuses` form-encoded with `status`, `visibility="unlisted"`, optional `in_reply_to_id`. Chunked at `MAX_MESSAGE_LEN` (default 500); chained chunks thread under each prior response id so a long reply stays one conversational thread. Replies to inbound mentions use `cmd.thread_id` (in_reply_to_id). `suppress_error_responses = True` — Mastodon posts are public. - New env-var knobs beyond the in-process equivalents: `MASTODON_VISIBILITY` (public/unlisted/private/direct, default unlisted), `MASTODON_MAX_MESSAGE_LEN` (default 500, raise for longer-toot instances), optional `MASTODON_ACCOUNT_ID` for multi-bot routing surfaced via `ready_event`. - `sdk/python/tests/test_mastodon_adapter.py` — 32 unit tests: URL/scheme normalization, required-env enforcement, visibility allowlist, max_len validation, account_id surfacing, HTML stripper edge cases (plain passthrough, basic tags, block-close newlines, entity decoding, typical mention anchor shape), notification parsing (full shape, thread_id from in_reply_to, non-mention skip, self-mention skip, empty text skip, slash command, display_name fallback), message chunking, REST publish (bearer auth + form + visibility, chunked thread chaining, reply to inbound thread, HTTPError surfaced, 5xx surfaced, custom visibility). Removed in-process mastodon (full cascade): - `librefang-channels`: `src/mastodon.rs` deleted (850 lines), `lib.rs` mod, `Cargo.toml` `channel-mastodon` feature + `all-channels` membership, `channels-allowlist.txt` entry (so `cargo xtask channel-policy` permanently blocks any in-process mastodon reintroduction). - `librefang-types`: `MastodonConfig` struct + Default + the `channels.mastodon` field + default, validation hook. - `librefang-api`: `channel-mastodon` feature + membership in `all-channels` / `all-channels-no-email`; channel_bridge `use` + spawn block + `check_channel` arm + `find_channel_info` arm + default-empty test assertion; `routes/channels.rs` ChannelMeta + 4 match arms (`mastodon_some` / `serde_to_value` / `len` / `ser`); `routes/config.rs` `ch!(mastodon)`. Added Mastodon to SIDECAR_CATALOG so the dashboard Channels page surfaces a configurable discovery row. - `librefang-kernel`: `channel_sender` `for_each_channel_field!` macro entry + name list. - `librefang-cli`: TUI `ChannelDef`. - `docs/src/app/*`: redirect both English and Chinese config / channels / integrations pages from `[channels.mastodon]` to `[[sidecar_channels]]`. Operator action required: # config.toml: remove [channels.mastodon] instance_url = "..." access_token_env = "MASTODON_ACCESS_TOKEN" # config.toml: add [[sidecar_channels]] name = "mastodon" command = "python3" args = ["-m", "librefang.sidecar.adapters.mastodon"] channel_type = "mastodon" [sidecar_channels.env] MASTODON_INSTANCE_URL = "https://mastodon.social" # secrets.env via secrets system: # MASTODON_ACCESS_TOKEN=... Verification: - `cd sdk/python && pytest`: 133 tests pass (32 new mastodon, 26 prior gotify, 75 unchanged ntfy/telegram/runtime/protocol). - `cargo check / clippy / test` deferred to CI — the host's cargo is a Docker wrapper with `rust:latest` not cached locally, and workspace-wide `cargo test` is forbidden by repo policy. - `kernel_config_schema.golden.json`: the matching test is `#[ignore]`'d on main (the fixture has drifted across earlier migrations); regen is a follow-up via `cargo test -p librefang-api --test config_schema_golden -- --ignored regenerate_golden`. * fix(sidecar/mastodon): call _verify_credentials before SSE/poll loop Self-review follow-up. The Rust adapter called `validate()` at the start of `ChannelAdapter::start()` to (a) validate the OAuth token and (b) discover the bot's own account id used by the self-mention guard in `parse_mastodon_notification`. My port defined `_verify_credentials` but never actually called it from `_producer_blocking`, so `self.own_account_id` stayed `None` and the guard: if self.own_account_id and account_id == self.own_account_id: return None silently short-circuited — a self-mention from the bot's own toots would be delivered back to the agent as an inbound event, potentially looping if the agent's reply itself triggers another mention. Same parity bug the in-process adapter explicitly defended against. Fix: `_producer_blocking` now runs verify with exponential backoff (1s → 60s) before entering the SSE-with-polling-fallback loop. The loop refuses to emit any event until verify has succeeded at least once. Transient verify failures (instance returning 5xx, network flake) retry instead of taking the sidecar down; persistent auth failures keep retrying — the supervisor can kill the subprocess if the operator wants a hard fail. Three new tests close coverage gaps the original PR missed: - `test_verify_credentials_sets_own_account_id`: monkeypatch urlopen, assert the request hits `/api/v1/accounts/verify_credentials` with the right Bearer header AND that the response `id` is stored on `own_account_id`. Locks in the discovery contract. - `test_verify_credentials_raises_on_bad_token`: a 401 response surfaces the HTTPError; `own_account_id` stays None so the self-mention guard does not match anything by accident. - `test_self_mention_skipped_only_after_verify`: the failing case — without verify, a self-mention IS delivered (proving the latent bug shape); after `own_account_id` is set the same payload is filtered. Catches a future regression where someone removes the verify call again. Verification: `cd sdk/python && pytest` — 35 mastodon tests pass (+3 new), full SDK suite 136 passed.
houko
pushed a commit
that referenced
this pull request
May 19, 2026
The Quality CI job on PR #5253 failed at `cargo xtask fmt` against 8 files under `crates/librefang-api/src/routes/`, `crates/librefang-api/tests/`, and `crates/librefang-channels/src/`, all merged in from the sidecar work on main (#5249/#5250/#5252/#5263). None of these files are touched by this PR's logic — the drift sits on main and any later dashboard-only PR rebased onto current main would hit the same gate. Mechanical rustfmt application: - `routes/channels.rs`: alphabetize `use super::sidecar_describe` before `use super::skills`; collapse the `secrets_env_keys` and `upsert_secret(...).map_err(...)` chains to rustfmt's preferred break style. - `routes/secrets_env.rs`: collapse the `parent.join(format!(...))` to a single line that fits in 100 cols. - `routes/sidecar_describe.rs` & `channels/src/sidecar.rs`: collapse two-arg fn signatures (`describe_sidecar`, `build_spawn_env`, `SidecarAdapter::new`) onto one line. - `tests/channels_routes_test.rs`: wrap the inline `vec![X { ... }]` literal across multiple lines so each field reads at indentation. - `tests/sidecar_describe_test.rs`: collapse the `describe_sidecar(...)` call to one line. - `channels/src/sidecar.rs`: split the long `got.get(...).map(|s| s.as_str())` chain across two lines per rustfmt's chain_width heuristic. Verification: - Each diff applied byte-for-byte from the CI log `gh api repos/.../actions/jobs/76681052747/logs`. cargo not available locally; relying on CI to confirm. - No behavioural change. Refs #5199, #5253
This was referenced May 19, 2026
houko
added a commit
that referenced
this pull request
May 20, 2026
… to live connection (#5199) (#5253) * fix(dashboard/chat): smooth auto-pin of agentId-only sessions; explicit "Unpinned" label (#5199) PR #5211 added URL auto-pinning after the first message of an unpinned `?agentId=` chat, but reused `handleBackendNewSession` which is wired for `/new` semantics — it bumps `sessionVersion` to force a full reload. The cascade was visible: data.session_id arrives → handleBackendNewSession → navigate(?sessionId=...) → useChatMessages.sessionId changes → load effect re-runs with sessionVersion bumped → deleteCachedChatMessages + setMessages([]) → fetchQuery(agentQueries.session) refetch + flicker. B fix: split the callback. New `handleAutoPinSession`: - navigates with `replace: true` (no Back-button trail through bare `?agentId=` URLs) - does NOT bump sessionVersion (the messages we just rendered belong to this session; bumping would wipe them) - invalidates only the per-agent sessions list so a freshly-created session surfaces in the dropdown Before navigating, the WS response handler seeds the chat-message cache under the new `(agentId, sessionId)` key from the just-applied response patch. The post-navigation load effect then takes a cache hit on sessionVersion=0 and renders instantly — no flicker, no refetch. The response patch is reified as a pure mapper so the cache snapshot matches what `setMessages` enqueues (messagesRef hasn't picked up the update yet at handler time). Auto-pin is also gated on `sendAgentId === currentAgentRef.current` so a late WS response for an agent the user has already navigated away from doesn't rewrite the URL off the agent they're now viewing. C fix: replace the generic "Session" placeholder in the dropdown label with an explicit "Unpinned" hint (new i18n keys en/zh) when `activeSessionId` is undefined. PR #5211 already prevents the misleading highlight from appearing on a row, but the label was ambiguous — "Unpinned" surfaces the ambiguity directly so users know messages are riding the canonical pointer rather than a chosen session. Backend issue #5198 (canonical pointer context loss) is intentionally out of scope here. Closes #5199 * fix(dashboard/chat): guard auto-pin against user mid-flight session swap; test dropdown label contract (#5199) Review follow-ups to PR #5253: - Tighten the WS-response auto-pin gate to also require `currentSessionRef.current === null`. The existing gate only checked `sendAgentId === currentAgentRef.current`, so if the user manually pinned a session by clicking another row in the dropdown between send and response, an already-queued WS frame on the about-to-detach listener could overwrite the user's deliberate pin. `ws.close()` is async — a frame queued in the browser can still reach the listener before close takes effect — so the WS swap-out alone is not enough to drop in-flight responses. Skipping the auto-pin in that case preserves the worse-case behaviour from before #5253 (URL stays unpinned for that one turn) instead of regressing into "URL silently rewrote off the session I just selected". - Extract the dropdown label-picking logic into a pure `pickSessionDropdownLabel` helper alongside `deriveDropdownActiveSessionId`. The label branch was inline JSX in a 2.5k-line file; with the helper extracted the contract (active row label > short id prefix > null) is unit-testable and a future refactor that drops the short-id fallback or the unpinned guard will be caught by the tests rather than landing silently. - 6 new vitest cases for `pickSessionDropdownLabel` covering the unpinned case (returns null so the caller renders the localized "Unpinned" hint), label hit, short-id fallback when the session hasn't surfaced in the list yet, defensive empty-id handling, and non-truncation of human labels. Verification: - pnpm typecheck: clean - pnpm test --run src/lib/sessionSelector.test.ts: 15/15 pass - pnpm test --run: 649/662 pass; the 13 failures are pre-existing in `ModelsPage.test.tsx` on main and untouched by this change (same baseline noted in PR #5253's body). - pnpm build: clean (ChatPage chunk 83.40 kB / 22.31 kB gz, unchanged from prior commit). - node scripts/i18n-parity.mjs: clean. Refs #5199 * style(channels): apply cargo fmt to sidecar + channel routes The Quality CI job on PR #5253 failed at `cargo xtask fmt` against 8 files under `crates/librefang-api/src/routes/`, `crates/librefang-api/tests/`, and `crates/librefang-channels/src/`, all merged in from the sidecar work on main (#5249/#5250/#5252/#5263). None of these files are touched by this PR's logic — the drift sits on main and any later dashboard-only PR rebased onto current main would hit the same gate. Mechanical rustfmt application: - `routes/channels.rs`: alphabetize `use super::sidecar_describe` before `use super::skills`; collapse the `secrets_env_keys` and `upsert_secret(...).map_err(...)` chains to rustfmt's preferred break style. - `routes/secrets_env.rs`: collapse the `parent.join(format!(...))` to a single line that fits in 100 cols. - `routes/sidecar_describe.rs` & `channels/src/sidecar.rs`: collapse two-arg fn signatures (`describe_sidecar`, `build_spawn_env`, `SidecarAdapter::new`) onto one line. - `tests/channels_routes_test.rs`: wrap the inline `vec![X { ... }]` literal across multiple lines so each field reads at indentation. - `tests/sidecar_describe_test.rs`: collapse the `describe_sidecar(...)` call to one line. - `channels/src/sidecar.rs`: split the long `got.get(...).map(|s| s.as_str())` chain across two lines per rustfmt's chain_width heuristic. Verification: - Each diff applied byte-for-byte from the CI log `gh api repos/.../actions/jobs/76681052747/logs`. cargo not available locally; relying on CI to confirm. - No behavioural change. Refs #5199, #5253 * fix(api/chat): pin HTTP fallback turns too — close Codex P2 on #5253 PR #5253 added URL auto-pinning for unpinned `?agentId=` chats, but only on the WS `response` event path. When `wsConnected` is false on first send, or the WS drops and `sendViaHttp` handles the turn, the HTTP fallback updates the assistant message but never calls `onAutoPinSession` — and the wire shape had no `session_id` body field anyway. So a bare `?agentId=` chat that took the HTTP path stayed unpinned even though the kernel persisted to a concrete session, leaving the user bookmarkable into a different canonical session after a daemon restart. That is the exact regression #5253 was meant to close, just for the second transport. Backend (preferred over a follow-up GET because the WS path already includes the field — adding it here is the natural symmetry, not a new surface): - `MessageResponse` gains `pub session_id: Option<String>` with `#[serde(default, skip_serializing_if = "Option::is_none")]`. Field is omitted entirely on the wire when None — matches the WS handler which only emits `session_id` when `explicit_session.is_none()`. - `routes/agents.rs::send_message` populates `session_id` from the post-turn registry entry only when `session_id_override.is_none()` — exact mirror of the WS branch in `ws.rs:1363`. When the caller explicitly pinned a session, the field is omitted so we never imply a server-side auto-resolution that did not happen. - `routes/skills.rs::hand_send_message` passes `session_id: None` (hands do not surface an auto-pinnable session id via this body; #5199 is dashboard-chat-only). - Two new serde-level tests in `types.rs::tests` lock the wire contract: `session_id` is omitted (not `null`) when None, and serializes as a bare string when present. A regression that flips the field to `null` would silently break the client's `typeof === "string"` check. Frontend: - New shared gate `shouldAutoPinResolvedSession` in `src/lib/sessionSelector.ts` — pure type-predicate function that both transports call. Five guards: same agent, no manual pin in flight, URL was unpinned, `resolvedSessionId` is a non-empty string. Extracted so the WS and HTTP paths cannot drift apart in a future refactor. - `ChatPage.tsx::sendViaHttp` now reifies its response-patch mapper (same pattern as the WS handler) and calls `shouldAutoPinResolvedSession` + `onAutoPinSession` exactly like the WS branch. Same cache-seed-before-navigate trick to avoid the post-nav flicker described in #5199-B. - WS handler refactored to call the same gate — round-1 had this as an inline 4-guard block; folding both paths through the helper preserves the round-1 invariant verbatim (verified against the invariants partner listed: deep-link, second-message-on-unpinned, back-button, reconnect — see the helper's docstring). - `AgentMessageResponse` TS type gains optional `session_id?: string`. - 8 new vitest cases for `shouldAutoPinResolvedSession` covering: happy path, agent-swap mid-flight (Codex P1), manual pin mid-flight (round-1), already-pinned request, omitted/`null`/empty `session_id`, non-string `session_id` (defensive), and the type predicate narrowing contract. Generated artifacts: - `openapi.json` updated by hand to add the `session_id` property in alphabetical position (utoipa's stable ordering). The OpenAPI Drift CI job will overwrite from the struct via `cargo xtask codegen --openapi`; if my edit differs from the regen the auto-commit branch handles it transparently for internal PRs. - `xtask/baselines/openapi.sha256` updated to match the hand-edited spec (`shasum -a 256 openapi.json`). - SDKs not regenerated locally — rustfmt unavailable, and the JS/Go/ Python generators don't model the response shape (no diff). CI's `python3 scripts/codegen-sdks.py` step regenerates and auto-commits if drift remains. Verification: - `pnpm typecheck`: clean. - `pnpm test --run src/lib/sessionSelector.test.ts`: 24/24 pass (was 15/15 after round-1, +9 for the new gate). - `pnpm test --run`: 658/671 pass; the 13 failures are the pre-existing `ModelsPage.test.tsx` baseline noted in #5253's body and unrelated to this change. - `pnpm build`: clean. ChatPage chunk 83.65 kB / 22.36 kB gz (was 83.40 kB / 22.31 kB — +0.25 kB for the new gate + shared HTTP fallback code path). - cargo not available locally; CI is the gate for `cargo check --workspace --lib`, `cargo clippy --workspace --all-targets -- -D warnings`, and the two new types.rs serde tests. Refs #5199, #5253 --------- Co-authored-by: vip <[email protected]>
This was referenced May 20, 2026
houko
added a commit
that referenced
this pull request
May 20, 2026
…5297) Replaces the 535-line in-process librefang-channels::twitch adapter with the Python sidecar librefang.sidecar.adapters.twitch. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281. Behaviour parity: - OAuth PASS / NICK handshake (oauth: prefix auto-added when absent) - JOIN #channel for each configured channel - PRIVMSG -> ChannelMessage with case-insensitive self-skip - /cmd / !cmd -> Content::Command, else Content::Text - PING -> PONG keepalive - MAX_MESSAGE_LEN = 500 chunking - Exponential reconnect backoff (1s -> 60s) - TWITCH_ACCOUNT_ID multi-bot routing Three improvements over the Rust adapter (file/line evidence): 1. TLS by default. The Rust adapter hard-coded plaintext port 6667 (TWITCH_IRC_PORT in crates/librefang-channels/src/twitch.rs:24) and sent the OAuth token in cleartext on every connect. The sidecar defaults to irc.chat.twitch.tv:6697 with ssl.create_default_context(). Plaintext is reachable only via TWITCH_PLAINTEXT=1 (tests/mock listeners). Credential-leak-on-wire fixed automatically on upgrade. 2. Per-message reply threading via IRCv3 tags. The Rust adapter never requested any capability and discarded any @-tag block (parse_privmsg in twitch.rs:100 only handled the bare ":nick!u@h PRIVMSG #ch :text" shape), so chunked replies arrived as a flat sequence with no link back. The sidecar issues CAP REQ :twitch.tv/tags twitch.tv/commands, parses @id, surfaces it as thread_id, and attaches @reply-parent-msg-id=<id> on outbound PRIVMSG. 3. Token-bucket send rate-limiter. Twitch drops the bot above 20 msgs / 30 s (100 / 30 s if mod). The Rust adapter shipped zero throttling (send() in twitch.rs:312 wrote PRIVMSG frames straight to the socket). Sidecar gates every outbound chunk; tunable via TWITCH_RATE_LIMIT_MSGS / TWITCH_RATE_LIMIT_SECS. Cascade removal: crates/librefang-channels/src/twitch.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, channel-twitch feature in librefang-channels + librefang-api (incl. all-channels / all-channels-no-email membership); TwitchConfig + ChannelsConfig.twitch field + Default; validation env-var hook; channel_bridge import + spawn block + find_channel_info! arm + check_channel! arm + default- empty test assertion; routes/channels.rs ChannelMeta + 4 match arms; routes/config.rs ch!(twitch); kernel channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef; types/config/mod.rs unit tests referencing TwitchConfig; docs (en + zh): configuration / channels / integrations / architecture / communication pages -- [channels.twitch] blocks replaced with [[sidecar_channels]] redirect; architecture-row moves twitch into the sidecar list. New env-var knobs: TWITCH_NICK, TWITCH_CHANNELS (comma-separated, no '#'), optional TWITCH_ACCOUNT_ID, optional TWITCH_RATE_LIMIT_MSGS / TWITCH_RATE_LIMIT_SECS, optional TWITCH_PLAINTEXT / TWITCH_HOST / TWITCH_PORT (test escape hatches). Operator action required: an existing [channels.twitch] block is no longer recognised -- re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.twitch with env vars TWITCH_NICK, TWITCH_CHANNELS (in [sidecar_channels.env]) and TWITCH_OAUTH_TOKEN (~/.librefang/secrets.env). Verification: - cargo check --workspace --lib: clean - cargo clippy -p librefang-types -p librefang-channels -p librefang-kernel -p librefang-api -p librefang-cli --all-targets -- -D warnings: clean - cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib: 843 passed - cargo test -p librefang-api --lib: 686 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cd sdk/python && python -m pytest tests/test_twitch_adapter.py: 68 passed
7 tasks
houko
added a commit
that referenced
this pull request
May 20, 2026
Replaces the 640-line in-process librefang-channels::nextcloud adapter with the Python sidecar librefang.sidecar.adapters.nextcloud. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298. Behaviour parity: - GET /ocs/v2.php/cloud/user startup credential probe - Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default - Empty allowlist auto-discovers via apps/spreed/api/v4/room - Same Bearer + OCS-APIRequest: true headers - Slash-command (/cmd args) routing - Multi-bot account_id metadata injection - 32000-char message chunking (MAX_MESSAGE_LEN) - Per-room transport-error isolation - 304-as-no-op handling of Talk's long-poll-expired response - suppress_error_responses (Talk rooms are multi-participant) Three improvements over the Rust adapter (file/line evidence in CHANGELOG): 1. Outbound threading via replyTo is now actually wired. The Rust api_send_message (nextcloud.rs:130-160) posted just {"message": ...} — Talk's replyTo form parameter was never sent and chunked/threaded replies always landed at the room root. The sidecar surfaces the inbound id (or parentMessage.id when inside a thread) as thread_id, and on_send posts replyTo so replies thread correctly. Mirrors reddit/bluesky/mastodon/ rocketchat. 2. Self-skip on (actorType, actorId) == ("users", own_user_id) rather than actorId alone. The Rust adapter compared on actorId (nextcloud.rs:338) without inspecting actorType, so a Talk guest/federated_users actor whose id happened to match the bot's user id silently spoofed self-skip. Parallels the rocketchat #5298 fix. 3. Dedupe set on id, bounded SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000. The Rust adapter only relied on the server-side lastKnownMessageId cursor (nextcloud.rs:347-354); under retry/re-poll boundaries Talk can resend the same id. Same policy as reddit/rocketchat. Cascade removal: crates/librefang-channels/src/nextcloud.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud feature in librefang-channels + librefang-api (incl. all-channels / all-channels-no-email membership); NextcloudConfig + ChannelsConfig.nextcloud field + Default; validation env-var hook; channel_bridge import + spawn block + find_channel_info! arm + check_channel! arm + default-empty test assertion; routes/channels.rs ChannelMeta + 4 match arms; routes/config.rs ch!(nextcloud); kernel channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef; docs (en + zh): configuration / channels / integrations / architecture / communication / security pages — [channels.nextcloud] blocks replaced with [[sidecar_channels]] redirect; architecture-row moves Nextcloud Talk into the sidecar list. routes/channels.rs SIDECAR_CATALOG gains a nextcloud entry. Golden fixture regenerated. New env-var knobs: NEXTCLOUD_SERVER_URL (replaces server_url), optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty = auto-discover), optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing, optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1). Operator action required: an existing [channels.nextcloud] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.nextcloud with env var NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN (in ~/.librefang/secrets.env). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib: 842 passed - cargo test -p librefang-api --lib: 686 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cd sdk/python && pytest tests/test_nextcloud_adapter.py: 58 passed
houko
added a commit
that referenced
this pull request
May 20, 2026
…#5301) Replaces the 640-line in-process librefang-channels::nextcloud adapter with the Python sidecar librefang.sidecar.adapters.nextcloud. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298. Behaviour parity: - GET /ocs/v2.php/cloud/user startup credential probe - Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default - Empty allowlist auto-discovers via apps/spreed/api/v4/room - Same Bearer + OCS-APIRequest: true headers - Slash-command (/cmd args) routing - Multi-bot account_id metadata injection - 32000-char message chunking (MAX_MESSAGE_LEN) - Per-room transport-error isolation - 304-as-no-op handling of Talk's long-poll-expired response - suppress_error_responses (Talk rooms are multi-participant) Three improvements over the Rust adapter (file/line evidence in CHANGELOG): 1. Outbound threading via replyTo is now actually wired. The Rust api_send_message (nextcloud.rs:130-160) posted just {"message": ...} — Talk's replyTo form parameter was never sent and chunked/threaded replies always landed at the room root. The sidecar surfaces the inbound id (or parentMessage.id when inside a thread) as thread_id, and on_send posts replyTo so replies thread correctly. Mirrors reddit/bluesky/mastodon/ rocketchat. 2. Self-skip on (actorType, actorId) == ("users", own_user_id) rather than actorId alone. The Rust adapter compared on actorId (nextcloud.rs:338) without inspecting actorType, so a Talk guest/federated_users actor whose id happened to match the bot's user id silently spoofed self-skip. Parallels the rocketchat #5298 fix. 3. Dedupe set on id, bounded SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000. The Rust adapter only relied on the server-side lastKnownMessageId cursor (nextcloud.rs:347-354); under retry/re-poll boundaries Talk can resend the same id. Same policy as reddit/rocketchat. Cascade removal: crates/librefang-channels/src/nextcloud.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud feature in librefang-channels + librefang-api (incl. all-channels / all-channels-no-email membership); NextcloudConfig + ChannelsConfig.nextcloud field + Default; validation env-var hook; channel_bridge import + spawn block + find_channel_info! arm + check_channel! arm + default-empty test assertion; routes/channels.rs ChannelMeta + 4 match arms; routes/config.rs ch!(nextcloud); kernel channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef; docs (en + zh): configuration / channels / integrations / architecture / communication / security pages — [channels.nextcloud] blocks replaced with [[sidecar_channels]] redirect; architecture-row moves Nextcloud Talk into the sidecar list. routes/channels.rs SIDECAR_CATALOG gains a nextcloud entry. Golden fixture regenerated. New env-var knobs: NEXTCLOUD_SERVER_URL (replaces server_url), optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty = auto-discover), optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing, optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1). Operator action required: an existing [channels.nextcloud] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.nextcloud with env var NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN (in ~/.librefang/secrets.env). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib: 842 passed - cargo test -p librefang-api --lib: 686 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cd sdk/python && pytest tests/test_nextcloud_adapter.py: 58 passed
houko
added a commit
that referenced
this pull request
May 20, 2026
) * feat(channels)!: migrate webex from in-process adapter to sidecar Replaces the 645-line in-process librefang-channels::webex adapter (Cisco Mercury WebSocket gateway for activity events + GET /messages/<id> REST follow-up + POST /messages publish) with the Python sidecar librefang.sidecar.adapters.webex. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299, nextcloud #5301, slack #5302. Behaviour parity: - GET /people/me startup credential probe (bot id + display name) - Hard-coded Mercury WSS URL with Bearer auth on upgrade request (no device-registration handshake — the Rust adapter never did one either) - data.activity envelope parsing; verb=="post" filter; actor-id self-skip - GET /messages/<id> REST follow-up for the full message body - Room filter (empty allowlist = all rooms the bot is in) - Slash-command (/cmd args) → Command routing - roomType == "group" → is_group mapping - Multi-bot account_id metadata injection - 7439-char message chunking (WEBEX_MSG_LIMIT parity with the Rust MAX_MESSAGE_LEN) - Exponential reconnect backoff (1s → 60s) - ChannelType::Custom("webex") preserved as channel_type = "webex" on the sidecar entry — existing routing / channel_role_mapping keys that reference webex continue to resolve. Four improvements over the Rust adapter (file/line evidence in CHANGELOG): 1. parentId outbound threading wired. The Rust api_send_message (webex.rs:171-201) posted just {"roomId", "text"} — Webex's parentId field was never sent and chunked replies always landed at the room root regardless of inbound context; the inbound side dropped the message id entirely (thread_id: None at webex.rs:438). The sidecar surfaces the inbound id (or inbound parentId when inside a thread) as thread_id, and on_send posts parentId populated so replies thread correctly. Mirrors reddit / rocketchat / nextcloud / mastodon / bluesky. 2. 429 Retry-After honoured on both GET /messages/<id> fetch and POST /messages send. The Rust adapter had no 429 handling at either call site, so a server-side rate-limit either lost the inbound fetch or returned Err and dropped the outbound. The sidecar parses Retry-After (default 30 s fallback, floor 1 s, cap MAX_BACKOFF_SECS), sleeps, and retries once before logging-and-continuing on the second 429. Same shape as #5303. 3. Mercury activity-id dedupe. Mercury can re-deliver an activity.object.id on reconnect (Rust adapter had no dedupe, see unconditional emit at webex.rs:459). Bounded local set on activity.object.id with SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000. 4. Explicit HTTP timeouts on every urlopen call (urllib has no default timeout; a hung Webex API would hang the producer thread forever). Cascade removal: - src/webex.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, cargo features in librefang-channels + librefang-api (incl. membership in all-channels / all-channels-no-email) removed - WebexConfig struct + Default impl + ChannelsConfig.webex field + matching Default entry removed; validation hook removed - channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion removed - routes/channels.rs: ChannelMeta entry + 4 match arms removed; SIDECAR_CATALOG webex entry added - routes/config.rs ch!(webex) call removed - kernel channel_sender for_each_channel_field! macro entry + EXPECTED name-list entry removed - CLI: TUI ChannelDef row removed, migrated-to-sidecar comment grown - README / AGENTS for librefang-channels updated to list webex among the migrated-to-sidecar adapters - Docs (EN + ZH): configuration/page.mdx, configuration/channels/page.mdx, integrations/channels/enterprise/page.mdx, configuration/security/page.mdx, architecture/page.mdx — [channels.webex] blocks replaced with [[sidecar_channels]] redirect blocks (fence language `bash` per Shiki); architecture wave table moves webex out of Wave 3 in-process and into the sidecar list New env-var knobs (read from [sidecar_channels.env]): - WEBEX_ALLOWED_ROOMS (comma-separated room IDs, empty = allow all) - WEBEX_ACCOUNT_ID (optional, multi-bot routing key) Secret in ~/.librefang/secrets.env: WEBEX_BOT_TOKEN. Operator action required: an existing [channels.webex] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.webex with WEBEX_BOT_TOKEN (in ~/.librefang/secrets.env) and any of the optional knobs above (in [sidecar_channels.env]). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib: 420 + 838 + 1087 passed - cargo test -p librefang-api --lib: 685 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cd sdk/python && pytest tests/test_webex_adapter.py: 78 passed * fix(channels/webex): apply review nits — display name, suppress flag, dead code, redundant filter Self-review pass on top of the migration commit; addresses 4 nits from the pre-merge review without changing the migration's core behaviour: 1. **personDisplayName drives user_name** (`webex.py:281-291, 326-330`). The migration commit preserved the Rust adapter's `webex.rs:431` behaviour of using `personEmail` unconditionally, which leaks emails into bot logs and dashboard UI. Now `parse_webex_message` prefers `personDisplayName` from the `/messages/<id>` body and falls back to `personEmail` when the field is absent (older Webex orgs / some service-account principals) or empty; `personEmail` stays in metadata for routing / audit. 3 new tests (`test_parse_prefers_person_display_name_for_user_name`, `…_falls_back_to_email_when_display_name_missing`, `…_falls_back_to_email_when_display_name_empty`). 2. **Explicit `suppress_error_responses = False`** on `WebexAdapter` with a precedent comment naming the discord / slack / twitch chat-room default vs. the mastodon / bluesky / reddit / nextcloud public-broadcast default. The migration left this implicit; an explicit class-level attr matches how `twitch.py:312` documents the choice so the next reader doesn't have to reconstruct the reasoning. 3. **`register_webhook` dead-code drop documented in CHANGELOG**. The Rust adapter carried a never-wired `register_webhook` helper at `webex.rs:137-168` (marked `#[allow(dead_code)]`) for an HTTP-webhook delivery alternative the channel-bridge never enabled; the sidecar drops it. The CHANGELOG now names this explicitly and points operators who want webhook delivery at the canonical generic `[[sidecar_channels]]` running `librefang.sidecar.adapters.webhook`. 4. **Skip redundant room filter inside `parse_webex_message`** when called from `_handle_envelope` (`webex.py:867-878`). `_handle_envelope` already filtered against `self.allowed_rooms` at line 855 before paying for the REST fetch; passing `allowed_rooms=[]` into the parser drops one redundant list-membership check per inbound message. The parser keeps the parameter for direct callers (tests; future code paths). Verification (run inside the worktree): * `cd sdk/python && python3 -m pytest tests/test_webex_adapter.py -q` → **81 passed** (was 78; +3 personDisplayName tests). No existing test modified. * `cargo check` / `clippy` not re-run — this commit is Python + CHANGELOG only, no Rust touched. Diff: +73 / -5 across `CHANGELOG.md`, `sdk/python/librefang/sidecar/adapters/webex.py`, `sdk/python/tests/test_webex_adapter.py`.
houko
added a commit
that referenced
this pull request
May 20, 2026
* feat(channels)!: migrate line from in-process adapter to sidecar Replaces the 881-line in-process librefang-channels::line adapter (axum webhook route at /channels/line/webhook on the shared API server, X-Line-Signature HMAC-SHA256 verification, POST /v2/bot/message/push for outbound) with the Python sidecar librefang.sidecar.adapters.line. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299, nextcloud #5301, slack #5302, webex #5309. Behaviour parity: - GET /v2/bot/info startup credential probe (bot display name) - HTTP webhook server (BaseHTTPRequestHandler over ThreadingTCPServer) on LINE_WEBHOOK_PORT (default 9090) + LINE_WEBHOOK_PATH (default /webhook) - X-Line-Signature HMAC-SHA256 verification over the raw wire bytes (constant-time compare, line.rs:229-250 parity, incl. the wire-bytes-vs-JSON-roundtrip regression) - Only message events with type==text (other event types and non-text message types dropped, line.rs:256-273) - Source-type -> reply_to mapping (user -> userId, group -> groupId, room -> roomId; group/room -> is_group=true, line.rs:280-290) - Slash-command routing (/cmd args -> Command) - Metadata preservation: user_id / reply_to / reply_token / source_type - Multi-bot account_id injection via LINE_ACCOUNT_ID - 5000-char chunking (LINE_MSG_LIMIT parity with MAX_MESSAGE_LEN at line.rs:39) - Image-branch wire shape (originalContentUrl + previewImageUrl both set to the caller URL, caption sent as follow-up text push, line.rs:464-490) - ChannelType::Custom("line") preserved as channel_type = "line" on the sidecar entry so existing routing / channel_role_mapping keys continue to resolve Three improvements over the Rust adapter: 1. 429 Retry-After honoured on outbound. The Rust api_push_message (line.rs:148-184) had no 429 handling; throttled push returned Err and the chunked reply dropped on the floor. Sidecar parses Retry-After (default 30 s fallback, floor 1 s, cap MAX_BACKOFF_SECS), sleeps, retries once, then logs-and-continues on the second 429. Same shape as #5303. 2. Inbound dedupe on message.id. LINE redelivers webhook events when the operator's endpoint fails; the Rust handler at line.rs:413-427 emitted every event unconditionally, causing duplicate agent invocations on transient failures. Bounded local set on message.id with SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000 (matches reddit / rocketchat / nextcloud / webex policy). 3. Explicit HTTP timeouts on every urlopen call (urllib has no default timeout; a hung LINE API would otherwise hang the worker thread forever). Cascade removal: - src/line.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, cargo features in librefang-channels + librefang-api (incl. membership in all-channels / all-channels-no-email) removed - LineConfig struct + Default impl + ChannelsConfig.line field + matching Default entry removed; validation hook removed - channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion removed - routes/channels.rs: ChannelMeta entry + 5 match arms removed (is_some / serialize / len / ser / is_channel_configured); webhook_route_suffix allowlist entry removed (sidecar runs its own webhook listener, not on the shared API port); SIDECAR_CATALOG line entry added - routes/config.rs ch!(line) call removed - kernel channel_sender for_each_channel_field! entry + EXPECTED name-list entry removed - CLI: TUI ChannelDef row removed, migrated-to-sidecar comment grown - README / AGENTS for librefang-channels updated; the mandatory- HMAC paragraph now lists Teams + DingTalk only (LINE moves to the sidecar but keeps the same X-Line-Signature contract inside the new process) - Docs (EN + ZH): configuration/page.mdx, configuration/channels/page.mdx, configuration/security/page.mdx, integrations/channels/integrations/page.mdx - [channels.line] blocks replaced with [[sidecar_channels]] redirect blocks; security env-var table flags both LINE_CHANNEL_SECRET and LINE_CHANNEL_ACCESS_TOKEN as sidecar-only; validation table drops the LINE row New env-var knobs (read from [sidecar_channels.env]): - LINE_WEBHOOK_PORT (default 9090) - LINE_WEBHOOK_PATH (default /webhook) - LINE_BIND_HOST (default 0.0.0.0) - LINE_ACCOUNT_ID (optional, multi-bot routing key) Secrets in ~/.librefang/secrets.env: LINE_CHANNEL_SECRET + LINE_CHANNEL_ACCESS_TOKEN. Operator action required (substantive): the sidecar runs its OWN HTTP webhook server - it is no longer mounted on the LibreFang API port - so the webhook URL registered at the LINE Developers Console must be updated to point at the sidecar host (typical pattern: an HTTPS reverse proxy in front of the sidecar's LINE_WEBHOOK_PORT). An existing [channels.line] block is no longer recognised - re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.line with LINE_CHANNEL_SECRET + LINE_CHANNEL_ACCESS_TOKEN (in ~/.librefang/secrets.env) and any of the optional knobs above (in [sidecar_channels.env]). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo fmt --check: clean - cargo test --lib -p librefang-types -p librefang-channels -p librefang-kernel: 420 + 838 + 1087 passed - cargo test --lib -p librefang-api: 685 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cargo xtask channel-policy: passed - cd sdk/python && pytest tests/test_line_adapter.py: 68 passed * docs(channels): correct line sidecar HTTP server class references The module docstring claimed `ThreadingHTTPServer` but the implementation uses `socketserver.ThreadingTCPServer` (intentional: skips `HTTPServer.server_bind()`'s `socket.getfqdn()` DNS lookup on startup). Align the three doc references with the actual class and spell out why we don't use `ThreadingHTTPServer` so the next reader doesn't "fix" it back.
houko
added a commit
that referenced
this pull request
May 20, 2026
Replaces the 758-line in-process `librefang-channels::qq` adapter (token mint + WebSocket gateway + HELLO/IDENTIFY/READY handshake + heartbeat + DISPATCH for 4 event types + REST send with markdown stripping) with an out-of-process Python sidecar at `sdk/python/librefang/sidecar/adapters/qq.py`. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299, nextcloud #5301, slack #5302, webex #5309, line #5312, zulip #5310, mattermost #5315, signal #5317. Behaviour parity (every claim cited against the pre-migration tree): - Token fetch: POST bots.qq.com/app/getAppAccessToken (qq.rs:542-557). - Gateway: GET /gateway with Bearer auth (qq.rs:559-573). - WS handshake: HELLO(op=10) → IDENTIFY(op=2) with token = "QQBot <access_token>", intents bitmask, shard=[0,1] → READY dispatch (qq.rs:413-433). - Heartbeat: op=1 frame at the server-supplied interval, carrying last seen `s` from dispatch frames (qq.rs:359-368). - 4 dispatch event types: MESSAGE_CREATE / AT_MESSAGE_CREATE → /channels/{channel_id}/messages, DIRECT_MESSAGE_CREATE → /dms/{guild_id}/messages, GROUP_AT_MESSAGE_CREATE → /v2/groups/{group_openid}/messages, C2C_MESSAGE_CREATE → /v2/users/{user_openid}/messages (qq.rs:194-224). - Bot-mention `/` strip + slash-command routing (qq.rs:227). - User allowlist + multi-bot account_id metadata (#5003, qq.rs:400-405). - Outbound markdown stripping pipeline preserves order: think tags → code blocks → inline code → bold → italic → headings → table separators → links → blockquotes → HRs → triple-newline collapse (qq.rs:137-180). - 2000-char chunking (qq.rs:26), 1–60 s exponential reconnect backoff (qq.rs:282). Improvements over the Rust adapter ================================== 1. **Reply context actually round-trips.** The Rust `parse_dispatch_event` (qq.rs:182-246) computed `reply_endpoint` and `msg_id` but the dispatch loop bound them to `_endpoint` / `_msg_id` (qq.rs:399) and dropped them on the floor; `send` (qq.rs:497-498) then expected `user.platform_id` to be encoded as `"<endpoint>|<msg_id>"` and silently no-op'd when the delimiter wasn't there. The Rust adapter therefore failed every real outbound — only the synthetic wiremock tests at qq.rs:686-712 exercised the working shape. The sidecar surfaces the reply endpoint as `channel_id` and the QQ `msg_id` as `thread_id` on the inbound event so the bridge round-trips them through to `on_send`, which posts to `{api_base}{channel_id}` with the correct passive-reply `msg_id`. 2. **Inbound dedupe on `msg.id`.** Rust emitted every parsed event unconditionally (qq.rs:399-410); WS reconnect could replay. Bounded local set, SEEN_MAX=10000 / EVICT=5000. 3. **429 Retry-After honoured on every REST path.** Token, gateway, and outbound send all sleep + retry once; second 429 logs-and-continues (matches webex / line / mattermost / signal #5303). 4. **Explicit 15s urlopen timeouts on every REST call.** Rust pre-configured reqwest's 30s default at qq.rs:71; sidecar passes `timeout=SEND_TIMEOUT_SECS` (15 s) so a misbehaving endpoint trips an explicit error. Cascade removal ================ - `src/qq.rs` deleted; `lib.rs` mod, `channels-allowlist.txt` entry, both `Cargo.toml` features removed. - `QqConfig` struct + Default + `ChannelsConfig.qq` field gone. - `channel_bridge.rs` import / spawn block / `check_channel!` / find_channel_info!(qq) (none existed) removed. - `routes/channels.rs` `ChannelMeta` entry + 4 match arms (`is_some` / serialize / `len` / `ser`) removed; SIDECAR_CATALOG `qq` entry added. - `routes/config.rs` `ch!(qq)` removed. - `channel_sender.rs` `for_each_channel_field!` macro entry + `EXPECTED` name-list entry removed. - CLI / TUI had no QQ wizard arm to remove. - librefang-migrate had no QQ block to handle (QQ wasn't in the OpenClaw schema). - Docs EN + ZH (6 files): `[channels.qq]` blocks replaced with `[[sidecar_channels]]` redirects; integrations tutorial rewritten for the sidecar. New env-var knobs (`[sidecar_channels.env]`) ============================================= - `QQ_APP_ID` (required) - `QQ_ALLOWED_USERS` (optional, CSV) - `QQ_ACCOUNT_ID` (optional) - `QQ_INTENTS` (optional, decimal/hex bitmask override) Secret (`~/.librefang/secrets.env`) ==================================== - `QQ_APP_SECRET` (required) Operator action required ======================== An existing `[channels.qq]` block is no longer recognised — re-declare as `[[sidecar_channels]]` running `python3 -m librefang.sidecar.adapters.qq`. See `sdk/python/librefang/sidecar/adapters/qq.py` header for the exact config. `ChannelType::Custom("qq")` is preserved via `channel_type = "qq"` on the sidecar entry so existing routing and `channel_role_mapping` keys continue to resolve. Verification ============ - `cargo check -p librefang-types -p librefang-channels -p librefang-api -p librefang-kernel -p librefang-cli --lib` — clean - `cargo xtask channel-policy` — passed (`qq` removed from allowlist) - `pytest sdk/python/tests/test_qq_adapter.py -q` — **77 passed** (covers env parsing, markdown stripping, parse_qq_event for all 4 event types + edge cases, _mark_seen LRU eviction, _fetch_token + _fetch_gateway happy path / 429 / non-200 / missing field, _post_message chunking + 429 retry-once + persistent-429 fail-open + 5xx fail-open, on_send wiring with markdown stripping at the outbound boundary, WS gateway flow via mock _WebSocketClient — HELLO → IDENTIFY token+intents+ shard, DISPATCH emission, msg.id dedupe, RECONNECT/INVALID_SESSION, heartbeat after interval, --describe schema, capabilities contract). Co-authored-by: Evan <[email protected]>
This was referenced May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
BREAKING: removes the in-process Rust gotify channel adapter and replaces it with a Python sidecar shipped in
librefang-sdk. Same pattern as the ntfy (#5224) and telegram (#5241) migrations — third entry in the long-tail sidecar-first backlog called out in `docs/architecture/sidecar-channels.md`.The separate gotify push-notification provider (`push_provider = "gotify"` for device pairing under `crates/librefang-kernel/src/pairing.rs`) is unaffected — different feature, deliberately left intact.
Operator action required
```toml
config.toml: remove
[channels.gotify]
server_url = "..."
app_token_env = "GOTIFY_APP_TOKEN"
client_token_env = "GOTIFY_CLIENT_TOKEN"
config.toml: add
[[sidecar_channels]]
name = "gotify"
command = "python3"
args = ["-m", "librefang.sidecar.adapters.gotify"]
channel_type = "gotify"
[sidecar_channels.env]
GOTIFY_SERVER_URL = "https://gotify.example.com\"
secrets.env (auto-written via the dashboard save flow):
GOTIFY_APP_TOKEN=...
GOTIFY_CLIENT_TOKEN=...
```
Test plan