feat(channels)!: migrate twitch from in-process adapter to sidecar#5297
Conversation
Replaces the 535-line in-process librefang-channels::twitch adapter with the Python sidecar librefang.sidecar.adapters.twitch. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281. Behaviour parity: - OAuth PASS / NICK handshake (oauth: prefix auto-added when absent) - JOIN #channel for each configured channel - PRIVMSG -> ChannelMessage with case-insensitive self-skip - /cmd / !cmd -> Content::Command, else Content::Text - PING -> PONG keepalive - MAX_MESSAGE_LEN = 500 chunking - Exponential reconnect backoff (1s -> 60s) - TWITCH_ACCOUNT_ID multi-bot routing Three improvements over the Rust adapter (file/line evidence): 1. TLS by default. The Rust adapter hard-coded plaintext port 6667 (TWITCH_IRC_PORT in crates/librefang-channels/src/twitch.rs:24) and sent the OAuth token in cleartext on every connect. The sidecar defaults to irc.chat.twitch.tv:6697 with ssl.create_default_context(). Plaintext is reachable only via TWITCH_PLAINTEXT=1 (tests/mock listeners). Credential-leak-on-wire fixed automatically on upgrade. 2. Per-message reply threading via IRCv3 tags. The Rust adapter never requested any capability and discarded any @-tag block (parse_privmsg in twitch.rs:100 only handled the bare ":nick!u@h PRIVMSG #ch :text" shape), so chunked replies arrived as a flat sequence with no link back. The sidecar issues CAP REQ :twitch.tv/tags twitch.tv/commands, parses @id, surfaces it as thread_id, and attaches @reply-parent-msg-id=<id> on outbound PRIVMSG. 3. Token-bucket send rate-limiter. Twitch drops the bot above 20 msgs / 30 s (100 / 30 s if mod). The Rust adapter shipped zero throttling (send() in twitch.rs:312 wrote PRIVMSG frames straight to the socket). Sidecar gates every outbound chunk; tunable via TWITCH_RATE_LIMIT_MSGS / TWITCH_RATE_LIMIT_SECS. Cascade removal: crates/librefang-channels/src/twitch.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, channel-twitch feature in librefang-channels + librefang-api (incl. all-channels / all-channels-no-email membership); TwitchConfig + ChannelsConfig.twitch field + Default; validation env-var hook; channel_bridge import + spawn block + find_channel_info! arm + check_channel! arm + default- empty test assertion; routes/channels.rs ChannelMeta + 4 match arms; routes/config.rs ch!(twitch); kernel channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef; types/config/mod.rs unit tests referencing TwitchConfig; docs (en + zh): configuration / channels / integrations / architecture / communication pages -- [channels.twitch] blocks replaced with [[sidecar_channels]] redirect; architecture-row moves twitch into the sidecar list. New env-var knobs: TWITCH_NICK, TWITCH_CHANNELS (comma-separated, no '#'), optional TWITCH_ACCOUNT_ID, optional TWITCH_RATE_LIMIT_MSGS / TWITCH_RATE_LIMIT_SECS, optional TWITCH_PLAINTEXT / TWITCH_HOST / TWITCH_PORT (test escape hatches). Operator action required: an existing [channels.twitch] block is no longer recognised -- re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.twitch with env vars TWITCH_NICK, TWITCH_CHANNELS (in [sidecar_channels.env]) and TWITCH_OAUTH_TOKEN (~/.librefang/secrets.env). Verification: - cargo check --workspace --lib: clean - cargo clippy -p librefang-types -p librefang-channels -p librefang-kernel -p librefang-api -p librefang-cli --all-targets -- -D warnings: clean - cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib: 843 passed - cargo test -p librefang-api --lib: 686 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cd sdk/python && python -m pytest tests/test_twitch_adapter.py: 68 passed
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cb8192a443
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| window_secs = 1.0 | ||
| self.capacity = capacity | ||
| self.window = float(window_secs) | ||
| self.tokens = float(capacity) |
There was a problem hiding this comment.
Enforce Twitch per-channel send pacing
Initialize-and-drain token-bucket logic allows an immediate burst of capacity messages (20 by default) because tokens starts full, so chunked replies or back-to-back sends can flush multiple PRIVMSG frames in under a second. Twitch’s documented IRC limits include 1 message/second per channel for non-mod/VIP users, so this implementation still triggers silent drops (or temporary chat suppression) in normal usage despite the new limiter. Please gate sends with an additional per-channel 1 Hz limiter (or equivalent pacing) instead of only a global bucket.
Useful? React with 👍 / 👎.
Deploying librefang-docs with
|
| Latest commit: |
cb8192a
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://a6d71461.librefang-docs-web.pages.dev |
| Branch Preview URL: | https://feat-channels-twitch-sidecar.librefang-docs-web.pages.dev |
Replaces the 640-line in-process librefang-channels::nextcloud adapter with the Python sidecar librefang.sidecar.adapters.nextcloud. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298. Behaviour parity: - GET /ocs/v2.php/cloud/user startup credential probe - Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default - Empty allowlist auto-discovers via apps/spreed/api/v4/room - Same Bearer + OCS-APIRequest: true headers - Slash-command (/cmd args) routing - Multi-bot account_id metadata injection - 32000-char message chunking (MAX_MESSAGE_LEN) - Per-room transport-error isolation - 304-as-no-op handling of Talk's long-poll-expired response - suppress_error_responses (Talk rooms are multi-participant) Three improvements over the Rust adapter (file/line evidence in CHANGELOG): 1. Outbound threading via replyTo is now actually wired. The Rust api_send_message (nextcloud.rs:130-160) posted just {"message": ...} — Talk's replyTo form parameter was never sent and chunked/threaded replies always landed at the room root. The sidecar surfaces the inbound id (or parentMessage.id when inside a thread) as thread_id, and on_send posts replyTo so replies thread correctly. Mirrors reddit/bluesky/mastodon/ rocketchat. 2. Self-skip on (actorType, actorId) == ("users", own_user_id) rather than actorId alone. The Rust adapter compared on actorId (nextcloud.rs:338) without inspecting actorType, so a Talk guest/federated_users actor whose id happened to match the bot's user id silently spoofed self-skip. Parallels the rocketchat #5298 fix. 3. Dedupe set on id, bounded SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000. The Rust adapter only relied on the server-side lastKnownMessageId cursor (nextcloud.rs:347-354); under retry/re-poll boundaries Talk can resend the same id. Same policy as reddit/rocketchat. Cascade removal: crates/librefang-channels/src/nextcloud.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud feature in librefang-channels + librefang-api (incl. all-channels / all-channels-no-email membership); NextcloudConfig + ChannelsConfig.nextcloud field + Default; validation env-var hook; channel_bridge import + spawn block + find_channel_info! arm + check_channel! arm + default-empty test assertion; routes/channels.rs ChannelMeta + 4 match arms; routes/config.rs ch!(nextcloud); kernel channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef; docs (en + zh): configuration / channels / integrations / architecture / communication / security pages — [channels.nextcloud] blocks replaced with [[sidecar_channels]] redirect; architecture-row moves Nextcloud Talk into the sidecar list. routes/channels.rs SIDECAR_CATALOG gains a nextcloud entry. Golden fixture regenerated. New env-var knobs: NEXTCLOUD_SERVER_URL (replaces server_url), optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty = auto-discover), optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing, optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1). Operator action required: an existing [channels.nextcloud] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.nextcloud with env var NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN (in ~/.librefang/secrets.env). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib: 842 passed - cargo test -p librefang-api --lib: 686 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cd sdk/python && pytest tests/test_nextcloud_adapter.py: 58 passed
…#5301) Replaces the 640-line in-process librefang-channels::nextcloud adapter with the Python sidecar librefang.sidecar.adapters.nextcloud. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298. Behaviour parity: - GET /ocs/v2.php/cloud/user startup credential probe - Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default - Empty allowlist auto-discovers via apps/spreed/api/v4/room - Same Bearer + OCS-APIRequest: true headers - Slash-command (/cmd args) routing - Multi-bot account_id metadata injection - 32000-char message chunking (MAX_MESSAGE_LEN) - Per-room transport-error isolation - 304-as-no-op handling of Talk's long-poll-expired response - suppress_error_responses (Talk rooms are multi-participant) Three improvements over the Rust adapter (file/line evidence in CHANGELOG): 1. Outbound threading via replyTo is now actually wired. The Rust api_send_message (nextcloud.rs:130-160) posted just {"message": ...} — Talk's replyTo form parameter was never sent and chunked/threaded replies always landed at the room root. The sidecar surfaces the inbound id (or parentMessage.id when inside a thread) as thread_id, and on_send posts replyTo so replies thread correctly. Mirrors reddit/bluesky/mastodon/ rocketchat. 2. Self-skip on (actorType, actorId) == ("users", own_user_id) rather than actorId alone. The Rust adapter compared on actorId (nextcloud.rs:338) without inspecting actorType, so a Talk guest/federated_users actor whose id happened to match the bot's user id silently spoofed self-skip. Parallels the rocketchat #5298 fix. 3. Dedupe set on id, bounded SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000. The Rust adapter only relied on the server-side lastKnownMessageId cursor (nextcloud.rs:347-354); under retry/re-poll boundaries Talk can resend the same id. Same policy as reddit/rocketchat. Cascade removal: crates/librefang-channels/src/nextcloud.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud feature in librefang-channels + librefang-api (incl. all-channels / all-channels-no-email membership); NextcloudConfig + ChannelsConfig.nextcloud field + Default; validation env-var hook; channel_bridge import + spawn block + find_channel_info! arm + check_channel! arm + default-empty test assertion; routes/channels.rs ChannelMeta + 4 match arms; routes/config.rs ch!(nextcloud); kernel channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef; docs (en + zh): configuration / channels / integrations / architecture / communication / security pages — [channels.nextcloud] blocks replaced with [[sidecar_channels]] redirect; architecture-row moves Nextcloud Talk into the sidecar list. routes/channels.rs SIDECAR_CATALOG gains a nextcloud entry. Golden fixture regenerated. New env-var knobs: NEXTCLOUD_SERVER_URL (replaces server_url), optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty = auto-discover), optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing, optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1). Operator action required: an existing [channels.nextcloud] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.nextcloud with env var NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN (in ~/.librefang/secrets.env). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib: 842 passed - cargo test -p librefang-api --lib: 686 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cd sdk/python && pytest tests/test_nextcloud_adapter.py: 58 passed
) * feat(channels)!: migrate webex from in-process adapter to sidecar Replaces the 645-line in-process librefang-channels::webex adapter (Cisco Mercury WebSocket gateway for activity events + GET /messages/<id> REST follow-up + POST /messages publish) with the Python sidecar librefang.sidecar.adapters.webex. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299, nextcloud #5301, slack #5302. Behaviour parity: - GET /people/me startup credential probe (bot id + display name) - Hard-coded Mercury WSS URL with Bearer auth on upgrade request (no device-registration handshake — the Rust adapter never did one either) - data.activity envelope parsing; verb=="post" filter; actor-id self-skip - GET /messages/<id> REST follow-up for the full message body - Room filter (empty allowlist = all rooms the bot is in) - Slash-command (/cmd args) → Command routing - roomType == "group" → is_group mapping - Multi-bot account_id metadata injection - 7439-char message chunking (WEBEX_MSG_LIMIT parity with the Rust MAX_MESSAGE_LEN) - Exponential reconnect backoff (1s → 60s) - ChannelType::Custom("webex") preserved as channel_type = "webex" on the sidecar entry — existing routing / channel_role_mapping keys that reference webex continue to resolve. Four improvements over the Rust adapter (file/line evidence in CHANGELOG): 1. parentId outbound threading wired. The Rust api_send_message (webex.rs:171-201) posted just {"roomId", "text"} — Webex's parentId field was never sent and chunked replies always landed at the room root regardless of inbound context; the inbound side dropped the message id entirely (thread_id: None at webex.rs:438). The sidecar surfaces the inbound id (or inbound parentId when inside a thread) as thread_id, and on_send posts parentId populated so replies thread correctly. Mirrors reddit / rocketchat / nextcloud / mastodon / bluesky. 2. 429 Retry-After honoured on both GET /messages/<id> fetch and POST /messages send. The Rust adapter had no 429 handling at either call site, so a server-side rate-limit either lost the inbound fetch or returned Err and dropped the outbound. The sidecar parses Retry-After (default 30 s fallback, floor 1 s, cap MAX_BACKOFF_SECS), sleeps, and retries once before logging-and-continuing on the second 429. Same shape as #5303. 3. Mercury activity-id dedupe. Mercury can re-deliver an activity.object.id on reconnect (Rust adapter had no dedupe, see unconditional emit at webex.rs:459). Bounded local set on activity.object.id with SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000. 4. Explicit HTTP timeouts on every urlopen call (urllib has no default timeout; a hung Webex API would hang the producer thread forever). Cascade removal: - src/webex.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, cargo features in librefang-channels + librefang-api (incl. membership in all-channels / all-channels-no-email) removed - WebexConfig struct + Default impl + ChannelsConfig.webex field + matching Default entry removed; validation hook removed - channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion removed - routes/channels.rs: ChannelMeta entry + 4 match arms removed; SIDECAR_CATALOG webex entry added - routes/config.rs ch!(webex) call removed - kernel channel_sender for_each_channel_field! macro entry + EXPECTED name-list entry removed - CLI: TUI ChannelDef row removed, migrated-to-sidecar comment grown - README / AGENTS for librefang-channels updated to list webex among the migrated-to-sidecar adapters - Docs (EN + ZH): configuration/page.mdx, configuration/channels/page.mdx, integrations/channels/enterprise/page.mdx, configuration/security/page.mdx, architecture/page.mdx — [channels.webex] blocks replaced with [[sidecar_channels]] redirect blocks (fence language `bash` per Shiki); architecture wave table moves webex out of Wave 3 in-process and into the sidecar list New env-var knobs (read from [sidecar_channels.env]): - WEBEX_ALLOWED_ROOMS (comma-separated room IDs, empty = allow all) - WEBEX_ACCOUNT_ID (optional, multi-bot routing key) Secret in ~/.librefang/secrets.env: WEBEX_BOT_TOKEN. Operator action required: an existing [channels.webex] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.webex with WEBEX_BOT_TOKEN (in ~/.librefang/secrets.env) and any of the optional knobs above (in [sidecar_channels.env]). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib: 420 + 838 + 1087 passed - cargo test -p librefang-api --lib: 685 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cd sdk/python && pytest tests/test_webex_adapter.py: 78 passed * fix(channels/webex): apply review nits — display name, suppress flag, dead code, redundant filter Self-review pass on top of the migration commit; addresses 4 nits from the pre-merge review without changing the migration's core behaviour: 1. **personDisplayName drives user_name** (`webex.py:281-291, 326-330`). The migration commit preserved the Rust adapter's `webex.rs:431` behaviour of using `personEmail` unconditionally, which leaks emails into bot logs and dashboard UI. Now `parse_webex_message` prefers `personDisplayName` from the `/messages/<id>` body and falls back to `personEmail` when the field is absent (older Webex orgs / some service-account principals) or empty; `personEmail` stays in metadata for routing / audit. 3 new tests (`test_parse_prefers_person_display_name_for_user_name`, `…_falls_back_to_email_when_display_name_missing`, `…_falls_back_to_email_when_display_name_empty`). 2. **Explicit `suppress_error_responses = False`** on `WebexAdapter` with a precedent comment naming the discord / slack / twitch chat-room default vs. the mastodon / bluesky / reddit / nextcloud public-broadcast default. The migration left this implicit; an explicit class-level attr matches how `twitch.py:312` documents the choice so the next reader doesn't have to reconstruct the reasoning. 3. **`register_webhook` dead-code drop documented in CHANGELOG**. The Rust adapter carried a never-wired `register_webhook` helper at `webex.rs:137-168` (marked `#[allow(dead_code)]`) for an HTTP-webhook delivery alternative the channel-bridge never enabled; the sidecar drops it. The CHANGELOG now names this explicitly and points operators who want webhook delivery at the canonical generic `[[sidecar_channels]]` running `librefang.sidecar.adapters.webhook`. 4. **Skip redundant room filter inside `parse_webex_message`** when called from `_handle_envelope` (`webex.py:867-878`). `_handle_envelope` already filtered against `self.allowed_rooms` at line 855 before paying for the REST fetch; passing `allowed_rooms=[]` into the parser drops one redundant list-membership check per inbound message. The parser keeps the parameter for direct callers (tests; future code paths). Verification (run inside the worktree): * `cd sdk/python && python3 -m pytest tests/test_webex_adapter.py -q` → **81 passed** (was 78; +3 personDisplayName tests). No existing test modified. * `cargo check` / `clippy` not re-run — this commit is Python + CHANGELOG only, no Rust touched. Diff: +73 / -5 across `CHANGELOG.md`, `sdk/python/librefang/sidecar/adapters/webex.py`, `sdk/python/tests/test_webex_adapter.py`.
* feat(channels)!: migrate line from in-process adapter to sidecar Replaces the 881-line in-process librefang-channels::line adapter (axum webhook route at /channels/line/webhook on the shared API server, X-Line-Signature HMAC-SHA256 verification, POST /v2/bot/message/push for outbound) with the Python sidecar librefang.sidecar.adapters.line. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299, nextcloud #5301, slack #5302, webex #5309. Behaviour parity: - GET /v2/bot/info startup credential probe (bot display name) - HTTP webhook server (BaseHTTPRequestHandler over ThreadingTCPServer) on LINE_WEBHOOK_PORT (default 9090) + LINE_WEBHOOK_PATH (default /webhook) - X-Line-Signature HMAC-SHA256 verification over the raw wire bytes (constant-time compare, line.rs:229-250 parity, incl. the wire-bytes-vs-JSON-roundtrip regression) - Only message events with type==text (other event types and non-text message types dropped, line.rs:256-273) - Source-type -> reply_to mapping (user -> userId, group -> groupId, room -> roomId; group/room -> is_group=true, line.rs:280-290) - Slash-command routing (/cmd args -> Command) - Metadata preservation: user_id / reply_to / reply_token / source_type - Multi-bot account_id injection via LINE_ACCOUNT_ID - 5000-char chunking (LINE_MSG_LIMIT parity with MAX_MESSAGE_LEN at line.rs:39) - Image-branch wire shape (originalContentUrl + previewImageUrl both set to the caller URL, caption sent as follow-up text push, line.rs:464-490) - ChannelType::Custom("line") preserved as channel_type = "line" on the sidecar entry so existing routing / channel_role_mapping keys continue to resolve Three improvements over the Rust adapter: 1. 429 Retry-After honoured on outbound. The Rust api_push_message (line.rs:148-184) had no 429 handling; throttled push returned Err and the chunked reply dropped on the floor. Sidecar parses Retry-After (default 30 s fallback, floor 1 s, cap MAX_BACKOFF_SECS), sleeps, retries once, then logs-and-continues on the second 429. Same shape as #5303. 2. Inbound dedupe on message.id. LINE redelivers webhook events when the operator's endpoint fails; the Rust handler at line.rs:413-427 emitted every event unconditionally, causing duplicate agent invocations on transient failures. Bounded local set on message.id with SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000 (matches reddit / rocketchat / nextcloud / webex policy). 3. Explicit HTTP timeouts on every urlopen call (urllib has no default timeout; a hung LINE API would otherwise hang the worker thread forever). Cascade removal: - src/line.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, cargo features in librefang-channels + librefang-api (incl. membership in all-channels / all-channels-no-email) removed - LineConfig struct + Default impl + ChannelsConfig.line field + matching Default entry removed; validation hook removed - channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion removed - routes/channels.rs: ChannelMeta entry + 5 match arms removed (is_some / serialize / len / ser / is_channel_configured); webhook_route_suffix allowlist entry removed (sidecar runs its own webhook listener, not on the shared API port); SIDECAR_CATALOG line entry added - routes/config.rs ch!(line) call removed - kernel channel_sender for_each_channel_field! entry + EXPECTED name-list entry removed - CLI: TUI ChannelDef row removed, migrated-to-sidecar comment grown - README / AGENTS for librefang-channels updated; the mandatory- HMAC paragraph now lists Teams + DingTalk only (LINE moves to the sidecar but keeps the same X-Line-Signature contract inside the new process) - Docs (EN + ZH): configuration/page.mdx, configuration/channels/page.mdx, configuration/security/page.mdx, integrations/channels/integrations/page.mdx - [channels.line] blocks replaced with [[sidecar_channels]] redirect blocks; security env-var table flags both LINE_CHANNEL_SECRET and LINE_CHANNEL_ACCESS_TOKEN as sidecar-only; validation table drops the LINE row New env-var knobs (read from [sidecar_channels.env]): - LINE_WEBHOOK_PORT (default 9090) - LINE_WEBHOOK_PATH (default /webhook) - LINE_BIND_HOST (default 0.0.0.0) - LINE_ACCOUNT_ID (optional, multi-bot routing key) Secrets in ~/.librefang/secrets.env: LINE_CHANNEL_SECRET + LINE_CHANNEL_ACCESS_TOKEN. Operator action required (substantive): the sidecar runs its OWN HTTP webhook server - it is no longer mounted on the LibreFang API port - so the webhook URL registered at the LINE Developers Console must be updated to point at the sidecar host (typical pattern: an HTTPS reverse proxy in front of the sidecar's LINE_WEBHOOK_PORT). An existing [channels.line] block is no longer recognised - re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.line with LINE_CHANNEL_SECRET + LINE_CHANNEL_ACCESS_TOKEN (in ~/.librefang/secrets.env) and any of the optional knobs above (in [sidecar_channels.env]). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo fmt --check: clean - cargo test --lib -p librefang-types -p librefang-channels -p librefang-kernel: 420 + 838 + 1087 passed - cargo test --lib -p librefang-api: 685 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cargo xtask channel-policy: passed - cd sdk/python && pytest tests/test_line_adapter.py: 68 passed * docs(channels): correct line sidecar HTTP server class references The module docstring claimed `ThreadingHTTPServer` but the implementation uses `socketserver.ThreadingTCPServer` (intentional: skips `HTTPServer.server_bind()`'s `socket.getfqdn()` DNS lookup on startup). Align the three doc references with the actual class and spell out why we don't use `ThreadingHTTPServer` so the next reader doesn't "fix" it back.
Replaces the 758-line in-process `librefang-channels::qq` adapter (token mint + WebSocket gateway + HELLO/IDENTIFY/READY handshake + heartbeat + DISPATCH for 4 event types + REST send with markdown stripping) with an out-of-process Python sidecar at `sdk/python/librefang/sidecar/adapters/qq.py`. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299, nextcloud #5301, slack #5302, webex #5309, line #5312, zulip #5310, mattermost #5315, signal #5317. Behaviour parity (every claim cited against the pre-migration tree): - Token fetch: POST bots.qq.com/app/getAppAccessToken (qq.rs:542-557). - Gateway: GET /gateway with Bearer auth (qq.rs:559-573). - WS handshake: HELLO(op=10) → IDENTIFY(op=2) with token = "QQBot <access_token>", intents bitmask, shard=[0,1] → READY dispatch (qq.rs:413-433). - Heartbeat: op=1 frame at the server-supplied interval, carrying last seen `s` from dispatch frames (qq.rs:359-368). - 4 dispatch event types: MESSAGE_CREATE / AT_MESSAGE_CREATE → /channels/{channel_id}/messages, DIRECT_MESSAGE_CREATE → /dms/{guild_id}/messages, GROUP_AT_MESSAGE_CREATE → /v2/groups/{group_openid}/messages, C2C_MESSAGE_CREATE → /v2/users/{user_openid}/messages (qq.rs:194-224). - Bot-mention `/` strip + slash-command routing (qq.rs:227). - User allowlist + multi-bot account_id metadata (#5003, qq.rs:400-405). - Outbound markdown stripping pipeline preserves order: think tags → code blocks → inline code → bold → italic → headings → table separators → links → blockquotes → HRs → triple-newline collapse (qq.rs:137-180). - 2000-char chunking (qq.rs:26), 1–60 s exponential reconnect backoff (qq.rs:282). Improvements over the Rust adapter ================================== 1. **Reply context actually round-trips.** The Rust `parse_dispatch_event` (qq.rs:182-246) computed `reply_endpoint` and `msg_id` but the dispatch loop bound them to `_endpoint` / `_msg_id` (qq.rs:399) and dropped them on the floor; `send` (qq.rs:497-498) then expected `user.platform_id` to be encoded as `"<endpoint>|<msg_id>"` and silently no-op'd when the delimiter wasn't there. The Rust adapter therefore failed every real outbound — only the synthetic wiremock tests at qq.rs:686-712 exercised the working shape. The sidecar surfaces the reply endpoint as `channel_id` and the QQ `msg_id` as `thread_id` on the inbound event so the bridge round-trips them through to `on_send`, which posts to `{api_base}{channel_id}` with the correct passive-reply `msg_id`. 2. **Inbound dedupe on `msg.id`.** Rust emitted every parsed event unconditionally (qq.rs:399-410); WS reconnect could replay. Bounded local set, SEEN_MAX=10000 / EVICT=5000. 3. **429 Retry-After honoured on every REST path.** Token, gateway, and outbound send all sleep + retry once; second 429 logs-and-continues (matches webex / line / mattermost / signal #5303). 4. **Explicit 15s urlopen timeouts on every REST call.** Rust pre-configured reqwest's 30s default at qq.rs:71; sidecar passes `timeout=SEND_TIMEOUT_SECS` (15 s) so a misbehaving endpoint trips an explicit error. Cascade removal ================ - `src/qq.rs` deleted; `lib.rs` mod, `channels-allowlist.txt` entry, both `Cargo.toml` features removed. - `QqConfig` struct + Default + `ChannelsConfig.qq` field gone. - `channel_bridge.rs` import / spawn block / `check_channel!` / find_channel_info!(qq) (none existed) removed. - `routes/channels.rs` `ChannelMeta` entry + 4 match arms (`is_some` / serialize / `len` / `ser`) removed; SIDECAR_CATALOG `qq` entry added. - `routes/config.rs` `ch!(qq)` removed. - `channel_sender.rs` `for_each_channel_field!` macro entry + `EXPECTED` name-list entry removed. - CLI / TUI had no QQ wizard arm to remove. - librefang-migrate had no QQ block to handle (QQ wasn't in the OpenClaw schema). - Docs EN + ZH (6 files): `[channels.qq]` blocks replaced with `[[sidecar_channels]]` redirects; integrations tutorial rewritten for the sidecar. New env-var knobs (`[sidecar_channels.env]`) ============================================= - `QQ_APP_ID` (required) - `QQ_ALLOWED_USERS` (optional, CSV) - `QQ_ACCOUNT_ID` (optional) - `QQ_INTENTS` (optional, decimal/hex bitmask override) Secret (`~/.librefang/secrets.env`) ==================================== - `QQ_APP_SECRET` (required) Operator action required ======================== An existing `[channels.qq]` block is no longer recognised — re-declare as `[[sidecar_channels]]` running `python3 -m librefang.sidecar.adapters.qq`. See `sdk/python/librefang/sidecar/adapters/qq.py` header for the exact config. `ChannelType::Custom("qq")` is preserved via `channel_type = "qq"` on the sidecar entry so existing routing and `channel_role_mapping` keys continue to resolve. Verification ============ - `cargo check -p librefang-types -p librefang-channels -p librefang-api -p librefang-kernel -p librefang-cli --lib` — clean - `cargo xtask channel-policy` — passed (`qq` removed from allowlist) - `pytest sdk/python/tests/test_qq_adapter.py -q` — **77 passed** (covers env parsing, markdown stripping, parse_qq_event for all 4 event types + edge cases, _mark_seen LRU eviction, _fetch_token + _fetch_gateway happy path / 429 / non-200 / missing field, _post_message chunking + 429 retry-once + persistent-429 fail-open + 5xx fail-open, on_send wiring with markdown stripping at the outbound boundary, WS gateway flow via mock _WebSocketClient — HELLO → IDENTIFY token+intents+ shard, DISPATCH emission, msg.id dedupe, RECONNECT/INVALID_SESSION, heartbeat after interval, --describe schema, capabilities contract). Co-authored-by: Evan <[email protected]>
Summary
Replaces the 535-line in-process
librefang-channels::twitchadapter (plaintext IRC onirc.chat.twitch.tv:6667) with the Python sidecarlibrefang.sidecar.adapters.twitch. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281.Behaviour parity
PASS/NICKhandshake (oauth:prefix auto-added when absent)JOIN #channelfor each configured channel/cmd/!cmd→Content::Command, elseContent::TextMAX_MESSAGE_LEN = 500chunkingTWITCH_ACCOUNT_IDmulti-bot routing (twitch:<account_id>routing key)Improvements over the Rust adapter (with file/line evidence)
6667(TWITCH_IRC_PORTatcrates/librefang-channels/src/twitch.rs:24in the deleted file) and sent the OAuth token in cleartext on every connect — a credential-leak-on-wire that operators get fixed automatically on upgrade. The sidecar defaults toirc.chat.twitch.tv:6697and wraps the socket withssl.create_default_context(). Plaintext is reachable only viaTWITCH_PLAINTEXT=1for local mock listeners (the test suite uses it; production must not).parse_privmsgatcrates/librefang-channels/src/twitch.rs:100only handled the bare":nick!u@h PRIVMSG #ch :text"shape and discarded any leading@-tag block; the adapter never requested IRCv3 capabilities, so chunked replies arrived as a flat sequence with no link back to the source. The sidecar issuesCAP REQ :twitch.tv/tags twitch.tv/commandsimmediately after socket-open, parses the@…tag block on every PRIVMSG, surfaces@idasthread_id, and attaches@reply-parent-msg-id=<id>on outbound PRIVMSG so Twitch renders threaded replies. Matches the bluesky feat(channels)!: migrate bluesky from in-process adapter to sidecar #5277 pattern.send()atcrates/librefang-channels/src/twitch.rs:312wrote PRIVMSG frames straight to the socket with zero throttling — a chatty agent would be silently dropped. The sidecar gates every outbound chunk through an in-process token bucket; defaults20 / 30 s, override viaTWITCH_RATE_LIMIT_MSGS/TWITCH_RATE_LIMIT_SECS.Cascade removal
crates/librefang-channels/src/twitch.rsdeletedlib.rsmod decl,channels-allowlist.txtentrychannel-twitchfeature inlibrefang-channels+librefang-api(incl.all-channels/all-channels-no-emailmembership)TwitchConfigstruct +Default,ChannelsConfig.twitchfield + Default, validation env-var hookchannel_bridge.rs: import, spawn block,find_channel_info!arm,check_channel!arm, default-empty test assertionroutes/channels.rs:ChannelMetaentry + 4 match arms (is_some/ serialize /len/ser); newSIDECAR_CATALOGentry pointing atpython3 -m librefang.sidecar.adapters.twitchroutes/config.rs:ch!(twitch)callchannel_senderfor_each_channel_field!macro entry +EXPECTEDname-listChannelDefcrates/librefang-types/src/config/mod.rs: unit tests referencingTwitchConfigconfiguration/page.mdx,configuration/channels/page.mdx,integrations/channels/social/page.mdx,integrations/api/communication/page.mdx,architecture/page.mdx—[channels.twitch]blocks replaced with[[sidecar_channels]]redirects; architecture row moves twitch into the sidecar listCHANGELOG.md### Changedbullet under[Unreleased]New env-var knobs & operator action required
New:
TWITCH_NICK,TWITCH_CHANNELS(comma-separated, no#), optionalTWITCH_ACCOUNT_ID, optionalTWITCH_RATE_LIMIT_MSGS/TWITCH_RATE_LIMIT_SECS, optionalTWITCH_PLAINTEXT/TWITCH_HOST/TWITCH_PORT(test escape hatches).Operator action required: an existing
[channels.twitch]block is no longer recognised — re-declare as a[[sidecar_channels]]runningpython3 -m librefang.sidecar.adapters.twitchwith env varsTWITCH_NICK,TWITCH_CHANNELS(in[sidecar_channels.env]) andTWITCH_OAUTH_TOKEN(in~/.librefang/secrets.env). Theoauth:prefix is auto-added if you leave it off.Verification
Commands actually run inside the worktree (
/Volumes/Lexar/Workspace/oss/librefang/wt-twitch-sidecar):Test plan
CLAUDE.md)