Skip to content

feat(channels)!: migrate twitch from in-process adapter to sidecar#5297

Merged
houko merged 1 commit into
mainfrom
feat/channels-twitch-sidecar
May 20, 2026
Merged

feat(channels)!: migrate twitch from in-process adapter to sidecar#5297
houko merged 1 commit into
mainfrom
feat/channels-twitch-sidecar

Conversation

@houko

@houko houko commented May 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the 535-line in-process librefang-channels::twitch adapter (plaintext IRC on irc.chat.twitch.tv:6667) with the Python sidecar librefang.sidecar.adapters.twitch. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281.

Behaviour parity

  • OAuth PASS / NICK handshake (oauth: prefix auto-added when absent)
  • JOIN #channel for each configured channel
  • PRIVMSG → ChannelMessage with case-insensitive self-skip
  • /cmd / !cmdContent::Command, else Content::Text
  • PING → PONG keepalive
  • MAX_MESSAGE_LEN = 500 chunking
  • Exponential reconnect backoff (1s → 60s)
  • TWITCH_ACCOUNT_ID multi-bot routing (twitch:<account_id> routing key)

Improvements over the Rust adapter (with file/line evidence)

  1. TLS by default. The Rust adapter hard-coded plaintext 6667 (TWITCH_IRC_PORT at crates/librefang-channels/src/twitch.rs:24 in the deleted file) and sent the OAuth token in cleartext on every connect — a credential-leak-on-wire that operators get fixed automatically on upgrade. The sidecar defaults to irc.chat.twitch.tv:6697 and wraps the socket with ssl.create_default_context(). Plaintext is reachable only via TWITCH_PLAINTEXT=1 for local mock listeners (the test suite uses it; production must not).
  2. Per-message reply threading via IRCv3 tags. The Rust parse_privmsg at crates/librefang-channels/src/twitch.rs:100 only handled the bare ":nick!u@h PRIVMSG #ch :text" shape and discarded any leading @-tag block; the adapter never requested IRCv3 capabilities, so chunked replies arrived as a flat sequence with no link back to the source. The sidecar issues CAP REQ :twitch.tv/tags twitch.tv/commands immediately after socket-open, parses the @… tag block on every PRIVMSG, surfaces @id as thread_id, and attaches @reply-parent-msg-id=<id> on outbound PRIVMSG so Twitch renders threaded replies. Matches the bluesky feat(channels)!: migrate bluesky from in-process adapter to sidecar #5277 pattern.
  3. Ban-avoidance token-bucket on outbound. Twitch drops the bot from chat above 20 msgs / 30 s for a non-mod account (100 / 30 s if mod). The Rust send() at crates/librefang-channels/src/twitch.rs:312 wrote PRIVMSG frames straight to the socket with zero throttling — a chatty agent would be silently dropped. The sidecar gates every outbound chunk through an in-process token bucket; defaults 20 / 30 s, override via TWITCH_RATE_LIMIT_MSGS / TWITCH_RATE_LIMIT_SECS.

Cascade removal

  • crates/librefang-channels/src/twitch.rs deleted
  • lib.rs mod decl, channels-allowlist.txt entry
  • channel-twitch feature in librefang-channels + librefang-api (incl. all-channels / all-channels-no-email membership)
  • TwitchConfig struct + Default, ChannelsConfig.twitch field + Default, validation env-var hook
  • channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion
  • routes/channels.rs: ChannelMeta entry + 4 match arms (is_some / serialize / len / ser); new SIDECAR_CATALOG entry pointing at python3 -m librefang.sidecar.adapters.twitch
  • routes/config.rs: ch!(twitch) call
  • Kernel channel_sender for_each_channel_field! macro entry + EXPECTED name-list
  • CLI TUI ChannelDef
  • crates/librefang-types/src/config/mod.rs: unit tests referencing TwitchConfig
  • Docs (EN + ZH): configuration/page.mdx, configuration/channels/page.mdx, integrations/channels/social/page.mdx, integrations/api/communication/page.mdx, architecture/page.mdx[channels.twitch] blocks replaced with [[sidecar_channels]] redirects; architecture row moves twitch into the sidecar list
  • CHANGELOG.md ### Changed bullet under [Unreleased]

New env-var knobs & operator action required

New: TWITCH_NICK, TWITCH_CHANNELS (comma-separated, no #), optional TWITCH_ACCOUNT_ID, optional TWITCH_RATE_LIMIT_MSGS / TWITCH_RATE_LIMIT_SECS, optional TWITCH_PLAINTEXT / TWITCH_HOST / TWITCH_PORT (test escape hatches).

Operator action required: an existing [channels.twitch] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.twitch with env vars TWITCH_NICK, TWITCH_CHANNELS (in [sidecar_channels.env]) and TWITCH_OAUTH_TOKEN (in ~/.librefang/secrets.env). The oauth: prefix is auto-added if you leave it off.

Verification

Commands actually run inside the worktree (/Volumes/Lexar/Workspace/oss/librefang/wt-twitch-sidecar):

cargo check --workspace --lib                                         # clean
cargo clippy -p librefang-types -p librefang-channels \
  -p librefang-kernel -p librefang-api -p librefang-cli \
  --all-targets -- -D warnings                                        # clean
cargo test -p librefang-types -p librefang-channels \
  -p librefang-kernel --lib                                           # 843 passed
cargo test -p librefang-api --lib                                     # 686 passed
cargo test -p librefang-api --test channels_routes_test \
  --test sidecar_describe_test                                        # 38 + 2 passed
cd sdk/python && python -m pytest tests/test_twitch_adapter.py        # 68 passed

Test plan

  • CI green on Ubuntu unit + integration shards (mac/Windows shards optional)
  • OpenAPI / dashboard regen (handled by pre-push hook on the maintainer's machine if needed)
  • Manual smoke against a live Twitch account by a maintainer holding an OAuth token (Claude cannot exercise live LLM/network paths per CLAUDE.md)

Replaces the 535-line in-process librefang-channels::twitch adapter
with the Python sidecar librefang.sidecar.adapters.twitch. Same pattern
as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky
#5277, reddit #5281.

Behaviour parity:
- OAuth PASS / NICK handshake (oauth: prefix auto-added when absent)
- JOIN #channel for each configured channel
- PRIVMSG -> ChannelMessage with case-insensitive self-skip
- /cmd / !cmd -> Content::Command, else Content::Text
- PING -> PONG keepalive
- MAX_MESSAGE_LEN = 500 chunking
- Exponential reconnect backoff (1s -> 60s)
- TWITCH_ACCOUNT_ID multi-bot routing

Three improvements over the Rust adapter (file/line evidence):
1. TLS by default. The Rust adapter hard-coded plaintext port 6667
   (TWITCH_IRC_PORT in crates/librefang-channels/src/twitch.rs:24) and
   sent the OAuth token in cleartext on every connect. The sidecar
   defaults to irc.chat.twitch.tv:6697 with ssl.create_default_context().
   Plaintext is reachable only via TWITCH_PLAINTEXT=1 (tests/mock
   listeners). Credential-leak-on-wire fixed automatically on upgrade.
2. Per-message reply threading via IRCv3 tags. The Rust adapter never
   requested any capability and discarded any @-tag block (parse_privmsg
   in twitch.rs:100 only handled the bare ":nick!u@h PRIVMSG #ch :text"
   shape), so chunked replies arrived as a flat sequence with no link
   back. The sidecar issues CAP REQ :twitch.tv/tags twitch.tv/commands,
   parses @id, surfaces it as thread_id, and attaches
   @reply-parent-msg-id=<id> on outbound PRIVMSG.
3. Token-bucket send rate-limiter. Twitch drops the bot above 20 msgs
   / 30 s (100 / 30 s if mod). The Rust adapter shipped zero throttling
   (send() in twitch.rs:312 wrote PRIVMSG frames straight to the
   socket). Sidecar gates every outbound chunk; tunable via
   TWITCH_RATE_LIMIT_MSGS / TWITCH_RATE_LIMIT_SECS.

Cascade removal: crates/librefang-channels/src/twitch.rs deleted;
lib.rs mod decl, channels-allowlist.txt entry, channel-twitch feature
in librefang-channels + librefang-api (incl. all-channels /
all-channels-no-email membership); TwitchConfig + ChannelsConfig.twitch
field + Default; validation env-var hook; channel_bridge import +
spawn block + find_channel_info! arm + check_channel! arm + default-
empty test assertion; routes/channels.rs ChannelMeta + 4 match arms;
routes/config.rs ch!(twitch); kernel channel_sender macro entry +
EXPECTED name; CLI TUI ChannelDef; types/config/mod.rs unit tests
referencing TwitchConfig; docs (en + zh): configuration / channels /
integrations / architecture / communication pages -- [channels.twitch]
blocks replaced with [[sidecar_channels]] redirect; architecture-row
moves twitch into the sidecar list.

New env-var knobs: TWITCH_NICK, TWITCH_CHANNELS (comma-separated, no
'#'), optional TWITCH_ACCOUNT_ID, optional TWITCH_RATE_LIMIT_MSGS /
TWITCH_RATE_LIMIT_SECS, optional TWITCH_PLAINTEXT / TWITCH_HOST /
TWITCH_PORT (test escape hatches).

Operator action required: an existing [channels.twitch] block is no
longer recognised -- re-declare as a [[sidecar_channels]] running
python3 -m librefang.sidecar.adapters.twitch with env vars
TWITCH_NICK, TWITCH_CHANNELS (in [sidecar_channels.env]) and
TWITCH_OAUTH_TOKEN (~/.librefang/secrets.env).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy -p librefang-types -p librefang-channels -p librefang-kernel
  -p librefang-api -p librefang-cli --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels -p librefang-kernel
  --lib: 843 passed
- cargo test -p librefang-api --lib: 686 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && python -m pytest tests/test_twitch_adapter.py:
  68 passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb8192a443

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

window_secs = 1.0
self.capacity = capacity
self.window = float(window_secs)
self.tokens = float(capacity)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Enforce Twitch per-channel send pacing

Initialize-and-drain token-bucket logic allows an immediate burst of capacity messages (20 by default) because tokens starts full, so chunked replies or back-to-back sends can flush multiple PRIVMSG frames in under a second. Twitch’s documented IRC limits include 1 message/second per channel for non-mod/VIP users, so this implementation still triggers silent drops (or temporary chat suppression) in normal usage despite the new limiter. Please gate sends with an additional per-channel 1 Hz limiter (or equivalent pacing) instead of only a global bucket.

Useful? React with 👍 / 👎.

@github-actions github-actions Bot added area/docs Documentation and guides area/channels Messaging channel adapters area/kernel Core kernel (scheduling, RBAC, workflows) area/sdk JavaScript and Python SDKs labels May 20, 2026
@cloudflare-workers-and-pages

Copy link
Copy Markdown

@houko
houko merged commit 81658b1 into main May 20, 2026
30 of 36 checks passed
@houko
houko deleted the feat/channels-twitch-sidecar branch May 20, 2026 02:58
houko added a commit that referenced this pull request May 20, 2026
Replaces the 640-line in-process librefang-channels::nextcloud adapter
with the Python sidecar librefang.sidecar.adapters.nextcloud. Same
pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264,
bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298.

Behaviour parity:
- GET /ocs/v2.php/cloud/user startup credential probe
- Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default
- Empty allowlist auto-discovers via apps/spreed/api/v4/room
- Same Bearer + OCS-APIRequest: true headers
- Slash-command (/cmd args) routing
- Multi-bot account_id metadata injection
- 32000-char message chunking (MAX_MESSAGE_LEN)
- Per-room transport-error isolation
- 304-as-no-op handling of Talk's long-poll-expired response
- suppress_error_responses (Talk rooms are multi-participant)

Three improvements over the Rust adapter (file/line evidence in
CHANGELOG):
1. Outbound threading via replyTo is now actually wired. The Rust
   api_send_message (nextcloud.rs:130-160) posted just
   {"message": ...} — Talk's replyTo form parameter was never sent
   and chunked/threaded replies always landed at the room root.
   The sidecar surfaces the inbound id (or parentMessage.id when
   inside a thread) as thread_id, and on_send posts replyTo so
   replies thread correctly. Mirrors reddit/bluesky/mastodon/
   rocketchat.
2. Self-skip on (actorType, actorId) == ("users", own_user_id)
   rather than actorId alone. The Rust adapter compared on
   actorId (nextcloud.rs:338) without inspecting actorType, so a
   Talk guest/federated_users actor whose id happened to match
   the bot's user id silently spoofed self-skip. Parallels the
   rocketchat #5298 fix.
3. Dedupe set on id, bounded SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000. The Rust adapter only relied on the
   server-side lastKnownMessageId cursor (nextcloud.rs:347-354);
   under retry/re-poll boundaries Talk can resend the same id.
   Same policy as reddit/rocketchat.

Cascade removal: crates/librefang-channels/src/nextcloud.rs deleted;
lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud
feature in librefang-channels + librefang-api (incl. all-channels /
all-channels-no-email membership); NextcloudConfig +
ChannelsConfig.nextcloud field + Default; validation env-var hook;
channel_bridge import + spawn block + find_channel_info! arm +
check_channel! arm + default-empty test assertion; routes/channels.rs
ChannelMeta + 4 match arms; routes/config.rs ch!(nextcloud); kernel
channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef;
docs (en + zh): configuration / channels / integrations /
architecture / communication / security pages — [channels.nextcloud]
blocks replaced with [[sidecar_channels]] redirect; architecture-row
moves Nextcloud Talk into the sidecar list. routes/channels.rs
SIDECAR_CATALOG gains a nextcloud entry. Golden fixture regenerated.

New env-var knobs: NEXTCLOUD_SERVER_URL (replaces server_url),
optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty =
auto-discover), optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing,
optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1).

Operator action required: an existing [channels.nextcloud] block is
no longer recognised — re-declare as a [[sidecar_channels]] running
python3 -m librefang.sidecar.adapters.nextcloud with env var
NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN
(in ~/.librefang/secrets.env).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels
  -p librefang-kernel --lib: 842 passed
- cargo test -p librefang-api --lib: 686 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && pytest tests/test_nextcloud_adapter.py: 58 passed
houko added a commit that referenced this pull request May 20, 2026
…#5301)

Replaces the 640-line in-process librefang-channels::nextcloud adapter
with the Python sidecar librefang.sidecar.adapters.nextcloud. Same
pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264,
bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298.

Behaviour parity:
- GET /ocs/v2.php/cloud/user startup credential probe
- Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default
- Empty allowlist auto-discovers via apps/spreed/api/v4/room
- Same Bearer + OCS-APIRequest: true headers
- Slash-command (/cmd args) routing
- Multi-bot account_id metadata injection
- 32000-char message chunking (MAX_MESSAGE_LEN)
- Per-room transport-error isolation
- 304-as-no-op handling of Talk's long-poll-expired response
- suppress_error_responses (Talk rooms are multi-participant)

Three improvements over the Rust adapter (file/line evidence in
CHANGELOG):
1. Outbound threading via replyTo is now actually wired. The Rust
   api_send_message (nextcloud.rs:130-160) posted just
   {"message": ...} — Talk's replyTo form parameter was never sent
   and chunked/threaded replies always landed at the room root.
   The sidecar surfaces the inbound id (or parentMessage.id when
   inside a thread) as thread_id, and on_send posts replyTo so
   replies thread correctly. Mirrors reddit/bluesky/mastodon/
   rocketchat.
2. Self-skip on (actorType, actorId) == ("users", own_user_id)
   rather than actorId alone. The Rust adapter compared on
   actorId (nextcloud.rs:338) without inspecting actorType, so a
   Talk guest/federated_users actor whose id happened to match
   the bot's user id silently spoofed self-skip. Parallels the
   rocketchat #5298 fix.
3. Dedupe set on id, bounded SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000. The Rust adapter only relied on the
   server-side lastKnownMessageId cursor (nextcloud.rs:347-354);
   under retry/re-poll boundaries Talk can resend the same id.
   Same policy as reddit/rocketchat.

Cascade removal: crates/librefang-channels/src/nextcloud.rs deleted;
lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud
feature in librefang-channels + librefang-api (incl. all-channels /
all-channels-no-email membership); NextcloudConfig +
ChannelsConfig.nextcloud field + Default; validation env-var hook;
channel_bridge import + spawn block + find_channel_info! arm +
check_channel! arm + default-empty test assertion; routes/channels.rs
ChannelMeta + 4 match arms; routes/config.rs ch!(nextcloud); kernel
channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef;
docs (en + zh): configuration / channels / integrations /
architecture / communication / security pages — [channels.nextcloud]
blocks replaced with [[sidecar_channels]] redirect; architecture-row
moves Nextcloud Talk into the sidecar list. routes/channels.rs
SIDECAR_CATALOG gains a nextcloud entry. Golden fixture regenerated.

New env-var knobs: NEXTCLOUD_SERVER_URL (replaces server_url),
optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty =
auto-discover), optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing,
optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1).

Operator action required: an existing [channels.nextcloud] block is
no longer recognised — re-declare as a [[sidecar_channels]] running
python3 -m librefang.sidecar.adapters.nextcloud with env var
NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN
(in ~/.librefang/secrets.env).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels
  -p librefang-kernel --lib: 842 passed
- cargo test -p librefang-api --lib: 686 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && pytest tests/test_nextcloud_adapter.py: 58 passed
@github-actions github-actions Bot added size/XL 1000+ lines changed ready-for-review PR is ready for maintainer review labels May 20, 2026
houko added a commit that referenced this pull request May 20, 2026
)

* feat(channels)!: migrate webex from in-process adapter to sidecar

Replaces the 645-line in-process librefang-channels::webex adapter
(Cisco Mercury WebSocket gateway for activity events + GET
/messages/<id> REST follow-up + POST /messages publish) with the
Python sidecar librefang.sidecar.adapters.webex. Same pattern as
ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky
#5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299,
nextcloud #5301, slack #5302.

Behaviour parity:
- GET /people/me startup credential probe (bot id + display name)
- Hard-coded Mercury WSS URL with Bearer auth on upgrade request
  (no device-registration handshake — the Rust adapter never did
  one either)
- data.activity envelope parsing; verb=="post" filter; actor-id
  self-skip
- GET /messages/<id> REST follow-up for the full message body
- Room filter (empty allowlist = all rooms the bot is in)
- Slash-command (/cmd args) → Command routing
- roomType == "group" → is_group mapping
- Multi-bot account_id metadata injection
- 7439-char message chunking (WEBEX_MSG_LIMIT parity with the Rust
  MAX_MESSAGE_LEN)
- Exponential reconnect backoff (1s → 60s)
- ChannelType::Custom("webex") preserved as channel_type = "webex"
  on the sidecar entry — existing routing / channel_role_mapping
  keys that reference webex continue to resolve.

Four improvements over the Rust adapter (file/line evidence in
CHANGELOG):
1. parentId outbound threading wired. The Rust api_send_message
   (webex.rs:171-201) posted just {"roomId", "text"} — Webex's
   parentId field was never sent and chunked replies always landed
   at the room root regardless of inbound context; the inbound
   side dropped the message id entirely (thread_id: None at
   webex.rs:438). The sidecar surfaces the inbound id (or inbound
   parentId when inside a thread) as thread_id, and on_send posts
   parentId populated so replies thread correctly. Mirrors reddit
   / rocketchat / nextcloud / mastodon / bluesky.
2. 429 Retry-After honoured on both GET /messages/<id> fetch and
   POST /messages send. The Rust adapter had no 429 handling at
   either call site, so a server-side rate-limit either lost the
   inbound fetch or returned Err and dropped the outbound. The
   sidecar parses Retry-After (default 30 s fallback, floor 1 s,
   cap MAX_BACKOFF_SECS), sleeps, and retries once before
   logging-and-continuing on the second 429. Same shape as
   #5303.
3. Mercury activity-id dedupe. Mercury can re-deliver an
   activity.object.id on reconnect (Rust adapter had no dedupe,
   see unconditional emit at webex.rs:459). Bounded local set on
   activity.object.id with SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000.
4. Explicit HTTP timeouts on every urlopen call (urllib has no
   default timeout; a hung Webex API would hang the producer
   thread forever).

Cascade removal:
- src/webex.rs deleted; lib.rs mod decl, channels-allowlist.txt
  entry, cargo features in librefang-channels + librefang-api
  (incl. membership in all-channels / all-channels-no-email)
  removed
- WebexConfig struct + Default impl + ChannelsConfig.webex field
  + matching Default entry removed; validation hook removed
- channel_bridge.rs: import, spawn block, find_channel_info! arm,
  check_channel! arm, default-empty test assertion removed
- routes/channels.rs: ChannelMeta entry + 4 match arms removed;
  SIDECAR_CATALOG webex entry added
- routes/config.rs ch!(webex) call removed
- kernel channel_sender for_each_channel_field! macro entry +
  EXPECTED name-list entry removed
- CLI: TUI ChannelDef row removed, migrated-to-sidecar comment
  grown
- README / AGENTS for librefang-channels updated to list webex
  among the migrated-to-sidecar adapters
- Docs (EN + ZH): configuration/page.mdx,
  configuration/channels/page.mdx,
  integrations/channels/enterprise/page.mdx,
  configuration/security/page.mdx, architecture/page.mdx —
  [channels.webex] blocks replaced with [[sidecar_channels]]
  redirect blocks (fence language `bash` per Shiki); architecture
  wave table moves webex out of Wave 3 in-process and into the
  sidecar list

New env-var knobs (read from [sidecar_channels.env]):
- WEBEX_ALLOWED_ROOMS (comma-separated room IDs, empty = allow all)
- WEBEX_ACCOUNT_ID (optional, multi-bot routing key)
Secret in ~/.librefang/secrets.env: WEBEX_BOT_TOKEN.

Operator action required: an existing [channels.webex] block is
no longer recognised — re-declare as a [[sidecar_channels]]
running python3 -m librefang.sidecar.adapters.webex with
WEBEX_BOT_TOKEN (in ~/.librefang/secrets.env) and any of the
optional knobs above (in [sidecar_channels.env]).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels
  -p librefang-kernel --lib: 420 + 838 + 1087 passed
- cargo test -p librefang-api --lib: 685 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && pytest tests/test_webex_adapter.py: 78 passed

* fix(channels/webex): apply review nits — display name, suppress flag, dead code, redundant filter

Self-review pass on top of the migration commit; addresses 4 nits from the
pre-merge review without changing the migration's core behaviour:

1. **personDisplayName drives user_name** (`webex.py:281-291, 326-330`). The
   migration commit preserved the Rust adapter's `webex.rs:431` behaviour of
   using `personEmail` unconditionally, which leaks emails into bot logs and
   dashboard UI. Now `parse_webex_message` prefers `personDisplayName` from the
   `/messages/<id>` body and falls back to `personEmail` when the field is
   absent (older Webex orgs / some service-account principals) or empty;
   `personEmail` stays in metadata for routing / audit. 3 new tests
   (`test_parse_prefers_person_display_name_for_user_name`,
   `…_falls_back_to_email_when_display_name_missing`,
   `…_falls_back_to_email_when_display_name_empty`).

2. **Explicit `suppress_error_responses = False`** on `WebexAdapter` with a
   precedent comment naming the discord / slack / twitch chat-room default vs.
   the mastodon / bluesky / reddit / nextcloud public-broadcast default. The
   migration left this implicit; an explicit class-level attr matches how
   `twitch.py:312` documents the choice so the next reader doesn't have to
   reconstruct the reasoning.

3. **`register_webhook` dead-code drop documented in CHANGELOG**. The Rust
   adapter carried a never-wired `register_webhook` helper at `webex.rs:137-168`
   (marked `#[allow(dead_code)]`) for an HTTP-webhook delivery alternative the
   channel-bridge never enabled; the sidecar drops it. The CHANGELOG now names
   this explicitly and points operators who want webhook delivery at the
   canonical generic `[[sidecar_channels]]` running
   `librefang.sidecar.adapters.webhook`.

4. **Skip redundant room filter inside `parse_webex_message`** when called
   from `_handle_envelope` (`webex.py:867-878`). `_handle_envelope` already
   filtered against `self.allowed_rooms` at line 855 before paying for the
   REST fetch; passing `allowed_rooms=[]` into the parser drops one redundant
   list-membership check per inbound message. The parser keeps the parameter
   for direct callers (tests; future code paths).

Verification (run inside the worktree):

* `cd sdk/python && python3 -m pytest tests/test_webex_adapter.py -q` →
  **81 passed** (was 78; +3 personDisplayName tests). No existing test
  modified.
* `cargo check` / `clippy` not re-run — this commit is Python + CHANGELOG only,
  no Rust touched.

Diff: +73 / -5 across `CHANGELOG.md`,
`sdk/python/librefang/sidecar/adapters/webex.py`,
`sdk/python/tests/test_webex_adapter.py`.
houko added a commit that referenced this pull request May 20, 2026
* feat(channels)!: migrate line from in-process adapter to sidecar

Replaces the 881-line in-process librefang-channels::line adapter
(axum webhook route at /channels/line/webhook on the shared API
server, X-Line-Signature HMAC-SHA256 verification, POST
/v2/bot/message/push for outbound) with the Python sidecar
librefang.sidecar.adapters.line. Same pattern as ntfy #5224,
telegram #5241, gotify #5263, mastodon #5264, bluesky #5277,
reddit #5281, twitch #5297, rocketchat #5298, discord #5299,
nextcloud #5301, slack #5302, webex #5309.

Behaviour parity:
- GET /v2/bot/info startup credential probe (bot display name)
- HTTP webhook server (BaseHTTPRequestHandler over
  ThreadingTCPServer) on LINE_WEBHOOK_PORT (default 9090) +
  LINE_WEBHOOK_PATH (default /webhook)
- X-Line-Signature HMAC-SHA256 verification over the raw wire
  bytes (constant-time compare, line.rs:229-250 parity, incl.
  the wire-bytes-vs-JSON-roundtrip regression)
- Only message events with type==text (other event types and
  non-text message types dropped, line.rs:256-273)
- Source-type -> reply_to mapping (user -> userId, group ->
  groupId, room -> roomId; group/room -> is_group=true,
  line.rs:280-290)
- Slash-command routing (/cmd args -> Command)
- Metadata preservation: user_id / reply_to / reply_token /
  source_type
- Multi-bot account_id injection via LINE_ACCOUNT_ID
- 5000-char chunking (LINE_MSG_LIMIT parity with MAX_MESSAGE_LEN
  at line.rs:39)
- Image-branch wire shape (originalContentUrl + previewImageUrl
  both set to the caller URL, caption sent as follow-up text
  push, line.rs:464-490)
- ChannelType::Custom("line") preserved as channel_type = "line"
  on the sidecar entry so existing routing /
  channel_role_mapping keys continue to resolve

Three improvements over the Rust adapter:
1. 429 Retry-After honoured on outbound. The Rust
   api_push_message (line.rs:148-184) had no 429 handling;
   throttled push returned Err and the chunked reply dropped on
   the floor. Sidecar parses Retry-After (default 30 s fallback,
   floor 1 s, cap MAX_BACKOFF_SECS), sleeps, retries once, then
   logs-and-continues on the second 429. Same shape as #5303.
2. Inbound dedupe on message.id. LINE redelivers webhook events
   when the operator's endpoint fails; the Rust handler at
   line.rs:413-427 emitted every event unconditionally, causing
   duplicate agent invocations on transient failures. Bounded
   local set on message.id with SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000 (matches reddit / rocketchat /
   nextcloud / webex policy).
3. Explicit HTTP timeouts on every urlopen call (urllib has no
   default timeout; a hung LINE API would otherwise hang the
   worker thread forever).

Cascade removal:
- src/line.rs deleted; lib.rs mod decl, channels-allowlist.txt
  entry, cargo features in librefang-channels + librefang-api
  (incl. membership in all-channels / all-channels-no-email)
  removed
- LineConfig struct + Default impl + ChannelsConfig.line field
  + matching Default entry removed; validation hook removed
- channel_bridge.rs: import, spawn block, find_channel_info!
  arm, check_channel! arm, default-empty test assertion removed
- routes/channels.rs: ChannelMeta entry + 5 match arms removed
  (is_some / serialize / len / ser / is_channel_configured);
  webhook_route_suffix allowlist entry removed (sidecar runs its
  own webhook listener, not on the shared API port);
  SIDECAR_CATALOG line entry added
- routes/config.rs ch!(line) call removed
- kernel channel_sender for_each_channel_field! entry + EXPECTED
  name-list entry removed
- CLI: TUI ChannelDef row removed, migrated-to-sidecar comment
  grown
- README / AGENTS for librefang-channels updated; the mandatory-
  HMAC paragraph now lists Teams + DingTalk only (LINE moves to
  the sidecar but keeps the same X-Line-Signature contract
  inside the new process)
- Docs (EN + ZH): configuration/page.mdx,
  configuration/channels/page.mdx,
  configuration/security/page.mdx,
  integrations/channels/integrations/page.mdx -
  [channels.line] blocks replaced with [[sidecar_channels]]
  redirect blocks; security env-var table flags both
  LINE_CHANNEL_SECRET and LINE_CHANNEL_ACCESS_TOKEN as
  sidecar-only; validation table drops the LINE row

New env-var knobs (read from [sidecar_channels.env]):
- LINE_WEBHOOK_PORT (default 9090)
- LINE_WEBHOOK_PATH (default /webhook)
- LINE_BIND_HOST (default 0.0.0.0)
- LINE_ACCOUNT_ID (optional, multi-bot routing key)
Secrets in ~/.librefang/secrets.env: LINE_CHANNEL_SECRET +
LINE_CHANNEL_ACCESS_TOKEN.

Operator action required (substantive): the sidecar runs its
OWN HTTP webhook server - it is no longer mounted on the
LibreFang API port - so the webhook URL registered at the LINE
Developers Console must be updated to point at the sidecar host
(typical pattern: an HTTPS reverse proxy in front of the
sidecar's LINE_WEBHOOK_PORT). An existing [channels.line] block
is no longer recognised - re-declare as a [[sidecar_channels]]
running python3 -m librefang.sidecar.adapters.line with
LINE_CHANNEL_SECRET + LINE_CHANNEL_ACCESS_TOKEN (in
~/.librefang/secrets.env) and any of the optional knobs above
(in [sidecar_channels.env]).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo fmt --check: clean
- cargo test --lib -p librefang-types -p librefang-channels
  -p librefang-kernel: 420 + 838 + 1087 passed
- cargo test --lib -p librefang-api: 685 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cargo xtask channel-policy: passed
- cd sdk/python && pytest tests/test_line_adapter.py:
  68 passed

* docs(channels): correct line sidecar HTTP server class references

The module docstring claimed `ThreadingHTTPServer` but the
implementation uses `socketserver.ThreadingTCPServer` (intentional:
skips `HTTPServer.server_bind()`'s `socket.getfqdn()` DNS lookup on
startup). Align the three doc references with the actual class and
spell out why we don't use `ThreadingHTTPServer` so the next reader
doesn't "fix" it back.
houko added a commit that referenced this pull request May 20, 2026
Replaces the 758-line in-process `librefang-channels::qq`
adapter (token mint + WebSocket gateway + HELLO/IDENTIFY/READY
handshake + heartbeat + DISPATCH for 4 event types + REST send
with markdown stripping) with an out-of-process Python sidecar
at `sdk/python/librefang/sidecar/adapters/qq.py`. Same pattern as
ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky
#5277, reddit #5281, twitch #5297, rocketchat #5298, discord
#5299, nextcloud #5301, slack #5302, webex #5309, line #5312,
zulip #5310, mattermost #5315, signal #5317.

Behaviour parity (every claim cited against the pre-migration
tree):

- Token fetch: POST bots.qq.com/app/getAppAccessToken (qq.rs:542-557).
- Gateway: GET /gateway with Bearer auth (qq.rs:559-573).
- WS handshake: HELLO(op=10) → IDENTIFY(op=2) with token =
  "QQBot <access_token>", intents bitmask, shard=[0,1] → READY
  dispatch (qq.rs:413-433).
- Heartbeat: op=1 frame at the server-supplied interval, carrying
  last seen `s` from dispatch frames (qq.rs:359-368).
- 4 dispatch event types: MESSAGE_CREATE / AT_MESSAGE_CREATE →
  /channels/{channel_id}/messages, DIRECT_MESSAGE_CREATE →
  /dms/{guild_id}/messages, GROUP_AT_MESSAGE_CREATE →
  /v2/groups/{group_openid}/messages, C2C_MESSAGE_CREATE →
  /v2/users/{user_openid}/messages (qq.rs:194-224).
- Bot-mention `/` strip + slash-command routing (qq.rs:227).
- User allowlist + multi-bot account_id metadata (#5003,
  qq.rs:400-405).
- Outbound markdown stripping pipeline preserves order:
  think tags → code blocks → inline code → bold → italic →
  headings → table separators → links → blockquotes → HRs →
  triple-newline collapse (qq.rs:137-180).
- 2000-char chunking (qq.rs:26), 1–60 s exponential reconnect
  backoff (qq.rs:282).

Improvements over the Rust adapter
==================================

1. **Reply context actually round-trips.** The Rust
   `parse_dispatch_event` (qq.rs:182-246) computed
   `reply_endpoint` and `msg_id` but the dispatch loop bound them
   to `_endpoint` / `_msg_id` (qq.rs:399) and dropped them on the
   floor; `send` (qq.rs:497-498) then expected
   `user.platform_id` to be encoded as `"<endpoint>|<msg_id>"`
   and silently no-op'd when the delimiter wasn't there. The
   Rust adapter therefore failed every real outbound — only the
   synthetic wiremock tests at qq.rs:686-712 exercised the
   working shape. The sidecar surfaces the reply endpoint as
   `channel_id` and the QQ `msg_id` as `thread_id` on the
   inbound event so the bridge round-trips them through to
   `on_send`, which posts to `{api_base}{channel_id}` with the
   correct passive-reply `msg_id`.

2. **Inbound dedupe on `msg.id`.** Rust emitted every parsed
   event unconditionally (qq.rs:399-410); WS reconnect could
   replay. Bounded local set, SEEN_MAX=10000 / EVICT=5000.

3. **429 Retry-After honoured on every REST path.** Token,
   gateway, and outbound send all sleep + retry once; second
   429 logs-and-continues (matches webex / line / mattermost /
   signal #5303).

4. **Explicit 15s urlopen timeouts on every REST call.** Rust
   pre-configured reqwest's 30s default at qq.rs:71; sidecar
   passes `timeout=SEND_TIMEOUT_SECS` (15 s) so a misbehaving
   endpoint trips an explicit error.

Cascade removal
================

- `src/qq.rs` deleted; `lib.rs` mod, `channels-allowlist.txt`
  entry, both `Cargo.toml` features removed.
- `QqConfig` struct + Default + `ChannelsConfig.qq` field gone.
- `channel_bridge.rs` import / spawn block / `check_channel!`
  / find_channel_info!(qq) (none existed) removed.
- `routes/channels.rs` `ChannelMeta` entry + 4 match arms
  (`is_some` / serialize / `len` / `ser`) removed; SIDECAR_CATALOG
  `qq` entry added.
- `routes/config.rs` `ch!(qq)` removed.
- `channel_sender.rs` `for_each_channel_field!` macro entry +
  `EXPECTED` name-list entry removed.
- CLI / TUI had no QQ wizard arm to remove.
- librefang-migrate had no QQ block to handle (QQ wasn't in the
  OpenClaw schema).
- Docs EN + ZH (6 files): `[channels.qq]` blocks replaced with
  `[[sidecar_channels]]` redirects; integrations tutorial
  rewritten for the sidecar.

New env-var knobs (`[sidecar_channels.env]`)
=============================================

- `QQ_APP_ID` (required)
- `QQ_ALLOWED_USERS` (optional, CSV)
- `QQ_ACCOUNT_ID` (optional)
- `QQ_INTENTS` (optional, decimal/hex bitmask override)

Secret (`~/.librefang/secrets.env`)
====================================

- `QQ_APP_SECRET` (required)

Operator action required
========================

An existing `[channels.qq]` block is no longer recognised —
re-declare as `[[sidecar_channels]]` running
`python3 -m librefang.sidecar.adapters.qq`. See
`sdk/python/librefang/sidecar/adapters/qq.py` header for the
exact config. `ChannelType::Custom("qq")` is preserved via
`channel_type = "qq"` on the sidecar entry so existing routing
and `channel_role_mapping` keys continue to resolve.

Verification
============

- `cargo check -p librefang-types -p librefang-channels -p librefang-api -p librefang-kernel -p librefang-cli --lib` — clean
- `cargo xtask channel-policy` — passed (`qq` removed from allowlist)
- `pytest sdk/python/tests/test_qq_adapter.py -q` — **77 passed**
  (covers env parsing, markdown stripping, parse_qq_event for all
  4 event types + edge cases, _mark_seen LRU eviction,
  _fetch_token + _fetch_gateway happy path / 429 / non-200 /
  missing field, _post_message chunking + 429 retry-once +
  persistent-429 fail-open + 5xx fail-open, on_send wiring with
  markdown stripping at the outbound boundary, WS gateway flow
  via mock _WebSocketClient — HELLO → IDENTIFY token+intents+
  shard, DISPATCH emission, msg.id dedupe, RECONNECT/INVALID_SESSION,
  heartbeat after interval, --describe schema, capabilities
  contract).

Co-authored-by: Evan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/channels Messaging channel adapters area/docs Documentation and guides area/kernel Core kernel (scheduling, RBAC, workflows) area/sdk JavaScript and Python SDKs ready-for-review PR is ready for maintainer review size/XL 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant