feat(channels)!: migrate discord from in-process adapter to sidecar#5299
Conversation
Deploying librefang-docs with
|
| Latest commit: |
3db4409
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://6bda141a.librefang-docs-web.pages.dev |
| Branch Preview URL: | https://feat-discord-sidecar.librefang-docs-web.pages.dev |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3db440960d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// Each field uses `OneOrMany<T>` to support both single-instance (`[channels.slack]`) | ||
| /// and multi-instance (`[[channels.slack]]`) TOML syntax for multi-bot routing. | ||
| #[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema)] | ||
| #[serde(default)] | ||
| pub struct ChannelsConfig { | ||
| /// Discord bot configuration(s). | ||
| pub discord: OneOrMany<DiscordConfig>, | ||
| /// Slack bot configuration(s). | ||
| pub slack: OneOrMany<SlackConfig>, |
There was a problem hiding this comment.
Regenerate schema golden after removing Discord config fields
This change removes ChannelsConfig.discord/DiscordConfig, which changes the generated KernelConfig schema, but the committed golden fixture still documents Discord entries (for example channels.discord and OneOrMany_DiscordConfig in crates/librefang-api/tests/fixtures/kernel_config_schema.golden.json). Leaving these out of sync creates schema drift for reviewers and tooling and will fail the golden-schema workflow once it is run/re-enabled; regenerate the fixture in the same commit.
Useful? React with 👍 / 👎.
Replace the 1747-line in-process librefang-channels::discord adapter (Gateway WS v10 + REST v10) with an out-of-process Python sidecar at sdk/python/librefang/sidecar/adapters/discord.py (stdlib-only). Behaviour parity: - GET /gateway/bot URL discovery + WSS connect with ?v=10&encoding=json - HELLO -> IDENTIFY/RESUME -> DISPATCH state machine (READY captures bot_user_id / session_id / resume_gateway_url; INVALID_SESSION resumable preserves state, non-resumable clears; RECONNECT raises) - MESSAGE_CREATE / MESSAGE_UPDATE -> message event with self-skip, ignore_bots, allowed_users / allowed_guilds filters - Attachment-takes-priority-over-slash-command (Image / Video / Voice / File by MIME prefix; audio/file warn-and-drop on companion text) - Discriminator-aware display name; mention detection via mentions[] array + <@id> / <@!id> tags + case-insensitive mention_patterns - POST /channels/{id}/messages with 2000-UTF-16-unit chunking - POST /channels/{id}/typing for typing indicators - account_id injection into message metadata Three improvements over the Rust adapter: 1. **Periodic client-side heartbeats**. The Rust adapter captured heartbeat_interval from HELLO but never sent its own heartbeats — connections silently dropped after ~45s with code=4000 and re-IDENTIFY'd, losing the session every minute. The sidecar runs proper periodic heartbeats with the RFC-mandated random jitter on the first beat, plus a server-initiated heartbeat slides the scheduler's next-deadline so we never double-beat back-to-back. 2. **429 retry-with-Retry-After**. The Rust api_send_message warned on 429 and returned Ok(()) (fail-open silent message loss); the sidecar honours Retry-After and retries once before logging-and- continuing on the second 429. 3. **select-gated frame waits**. The WebSocket reader uses select() to wait for the socket to become readable BEFORE consuming a frame, so mid-frame stalls can't race a heartbeat tick. Known-fatal close codes (4004 / 4013 / 4014) raise rather than reconnect so the supervisor's circuit-breaker stops a hard config error instead of looping. Two regressions to call out (both match the telegram-sidecar #5241 precedent; flagged so operators aren't surprised): - **Live Discord-guild-role RBAC is gone.** ChannelRoleQuery is a Rust trait the sidecar process cannot implement, so role_query is None for Discord post-migration, the kernel's resolve_role_for_sender falls through to the default-deny Viewer branch, and [channel_role_mapping.discord] is no longer consulted. Workaround: enumerate authorised operators under [users] with channel_bindings = { discord = [...] } and an explicit role. - **Per-channel proxy override (#4795) is not wired through.** The sidecar honours HTTP_PROXY / HTTPS_PROXY / ALL_PROXY via stdlib but has no DISCORD_PROXY_URL env var yet. Filed as a follow-up. Cascade removal: - src/discord.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, cargo features in librefang-channels + librefang-api (incl. membership in all-channels / all-channels-no-email / core-channels / mini) removed - DiscordConfig struct + Default impl + canonical deny_unknown_fields rustdoc anchor moved to SlackConfig; ChannelsConfig.discord field + default removed; validation hook removed - channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion removed - routes/channels.rs: ChannelMeta entry + 5 match arms removed; live-test discord branch (POST to discord.com/api/v10) removed; SIDECAR_CATALOG discord entry added; instance_helper_tests reanchored on slack; downstream_send_failure_returns_502 retired - routes/config.rs ch!(discord) call removed; is_writable_config_path test fixture switched to slack - kernel channel_sender for_each_channel_field! macro entry + EXPECTED name-list entry removed - CLI: librefang setup discord wizard arm + channel list row removed; TUI ChannelDef removed - openclaw migrator: discord channel input now produces a SkippedItem with a sidecar-redirect reason rather than writing [channels.discord] to the output (mirrors how telegram was handled in #5241); test_policy_migration retargeted at slack - openfang test fixtures reanchored on [channels.slack] - librefang.toml.example + cli init_default_config.toml: [channels.discord] block replaced with a commented [[sidecar_channels]] template - docs (en + zh): configuration/page.mdx, configuration/channels/page.mdx, integrations/channels/page.mdx, integrations/channels/core/page.mdx — [channels.discord] blocks replaced with [[sidecar_channels]] templates + a per-channel summary of what the sidecar preserves vs. what changed - ChannelType::Discord enum variant kept (still used by the bridge / router; same pattern as ChannelType::Telegram after #5241) New env-var knobs (read by the sidecar from [sidecar_channels.env]): DISCORD_ALLOWED_GUILDS, DISCORD_ALLOWED_USERS, DISCORD_INTENTS (default 37376), DISCORD_IGNORE_BOTS (default true), DISCORD_MENTION_PATTERNS, DISCORD_ACCOUNT_ID. DISCORD_BOT_TOKEN goes to ~/.librefang/secrets.env. Operator action required: an existing [channels.discord] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.discord. Verification: - cargo check --workspace --lib: clean - cargo check --workspace --features librefang-api/all-channels: clean - cargo clippy --workspace --all-targets --features librefang-api/all-channels -- -D warnings: zero warnings - cargo test -p librefang-channels --features all-channels --lib: 824/824 - cargo test -p librefang-channels --features all-channels --test bridge_integration_test: 27/27 - cargo test -p librefang-types --lib: 842/842 - cargo test -p librefang-kernel --lib: 1085/1085 - cargo test -p librefang-migrate --lib: 51/51 - cargo test -p librefang-api --lib --features all-channels: 692/692 - cargo test -p librefang-api --test channels_routes_test --features all-channels: 38/38 - cd sdk/python && pytest: 293/293 (69 new for discord including test_handle_payload_dispatch_does_not_signal_heartbeat_sent contract test for #3)
3db4409 to
2829178
Compare
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Replace the 1890-line in-process librefang-channels::slack adapter
(Socket Mode WS + Web API) with an out-of-process Python sidecar at
sdk/python/librefang/sidecar/adapters/slack.py (stdlib-only).
Behaviour parity:
- POST /api/auth.test startup probe to discover bot_user_id
- POST /api/apps.connections.open to mint Socket Mode WSS URL
- Envelope-id ACK loop for events_api / interactive
- message + app_mention event handling with message_changed subtype
extraction; all-other-subtype skip
- Self-skip on bot_id presence OR user == bot_user_id
- allowed_channels filter with DM exemption (channels starting with D)
- Slash-command routing on /cmd args -> Command (text otherwise)
- thread_ts captured as thread_id
- DM detection via channel-id prefix; is_group = !channel.startswith("D")
- block_actions interactive payload -> ButtonCallback content with
action_id / trigger_id / block_action metadata
- chat.postMessage send with optional thread_ts + unfurl_links + Block
Kit blocks; 3000-char chunking matches SLACK_MSG_LIMIT
- eyes reaction on receive flipped to white_check_mark on send-complete
(opt-out via SLACK_REACTIONS=false)
- force_flat_replies knob posts replies as top-level channel messages
instead of threads
- sender_user_id metadata key matches Rust SENDER_USER_ID_KEY parity
- account_id injection for multi-bot routing
One improvement over the Rust adapter: pending-reaction map is now
bounded at MAX_PENDING_REACTIONS=2000 entries with oldest-eviction.
The Rust adapter used an unbounded RwLock<HashMap> so a flood of
inbound messages followed by a hang in the agent loop would grow
the map without bound — a small but real memory-leak surface.
Two regressions to call out (both match the telegram-#5241 and
discord-#5299 precedents; flagged so operators aren't surprised):
- **Live Slack workspace-role RBAC is gone.** ChannelRoleQuery is a
Rust trait the sidecar process cannot implement, so role_query is
None for Slack post-migration; the kernel's resolve_role_for_sender
falls through to the default-deny Viewer branch and
[channel_role_mapping.slack] is no longer consulted. Workaround:
enumerate authorised operators under [users] with
channel_bindings = { slack = ["<slack_user_id>"] } and an explicit
role. The parse_users_info precedence parser is preserved in the
sidecar so a future sidecar-protocol query/response pair can
re-use it.
- **Per-channel proxy override (#4795) is not wired through.** The
sidecar honours HTTP_PROXY / HTTPS_PROXY / ALL_PROXY via stdlib but
has no SLACK_PROXY_URL env var yet. Filed as a follow-up.
Cascade removal:
- src/slack.rs deleted; lib.rs mod decl, channels-allowlist.txt
entry, cargo features in librefang-channels + librefang-api (incl.
membership in all-channels / all-channels-no-email / core-channels
/ mini) removed
- SlackConfig struct + Default impl + canonical deny_unknown_fields
rustdoc anchor moved to WhatsAppConfig; ChannelsConfig.slack field
+ default removed; validation hook removed
- channel_bridge.rs: import, spawn block, find_channel_info! arm,
check_channel! arm, default-empty test assertion removed
- routes/channels.rs: ChannelMeta entry + 5 match arms removed;
live-test slack branch (POST to slack.com/api/chat.postMessage)
retired with a comment explaining why; SIDECAR_CATALOG slack
entry added; instance_helper_tests reanchored on matrix;
test_channel tests reanchored on matrix
- routes/config.rs ch!(slack) call removed; is_writable_config_path
test fixtures switched to matrix
- kernel channel_sender for_each_channel_field! macro entry +
EXPECTED name-list entry removed
- CLI: librefang setup slack wizard arm + channel list row removed;
TUI ChannelDef removed
- channels_routes_test: SlackConfig fixtures retargeted to
MatrixConfig (bot_token_env -> access_token_env), test names /
display_name assertions updated, "other rows must not flip" check
uses mattermost instead
- routes/skills.rs tests: channels.slack -> channels.matrix bulk
rename across all TOML write/read tests; ad-hoc local Slack
struct renamed to Matrix
- types/config/mod.rs: test_slack_config_defaults +
test_slack_config_unfurl_links_deserialization tests removed
(covered by slack.rs unit tests which were also deleted);
validate_missing_env_vars retargeted to WhatsAppConfig;
test_one_or_many_single_toml_table + test_one_or_many_array_of_tables
retargeted to MatrixConfig; well_formed_repeated_channel_table_still_parses_5130
drops the slack fixture (slack was the last working anchor — now
only whatsapp + mattermost remain as deny_unknown_fields drift
sentinels)
- openclaw migrator: slack channel input now produces a SkippedItem
with a sidecar-redirect reason rather than writing [channels.slack]
(mirrors telegram + discord); test_policy_migration retargeted at
mattermost (slack and discord both sidecar-skipped); test_full_migration
legacy YAML fixture now includes mattermost.yaml so the
"report.imported.iter().any(Channel)" assertion still has a witness;
test_json5_channel_extraction adds mattermost and asserts all three
of telegram/discord/slack are sidecar-skipped
- openfang test fixtures reanchored on [channels.whatsapp]
- librefang.toml.example + cli init_default_config.toml:
[channels.slack] block replaced with commented [[sidecar_channels]]
template
- docs (en + zh): configuration/page.mdx, configuration/channels/page.mdx,
integrations/channels/page.mdx, integrations/channels/core/page.mdx
— [channels.slack] blocks replaced with [[sidecar_channels]] templates
- ChannelType::Slack enum variant kept (still used by the bridge /
router; same pattern as ChannelType::Telegram / ChannelType::Discord)
New env-var knobs (read by the sidecar from [sidecar_channels.env]):
SLACK_ALLOWED_CHANNELS, SLACK_UNFURL_LINKS (tri-state),
SLACK_FORCE_FLAT_REPLIES, SLACK_REACTIONS, SLACK_ACCOUNT_ID.
SLACK_APP_TOKEN and SLACK_BOT_TOKEN go to ~/.librefang/secrets.env.
Operator action required: an existing [channels.slack] block is no
longer recognised — re-declare as a [[sidecar_channels]] running
python3 -m librefang.sidecar.adapters.slack.
Verification:
- cargo check --workspace --lib: clean
- cargo check --workspace --features librefang-api/all-channels: clean
- cargo clippy --workspace --all-targets --features
librefang-api/all-channels -- -D warnings: zero warnings
- cargo test -p librefang-channels --features all-channels --lib:
766/766
- cargo test -p librefang-types --lib: 838/838
- cargo test -p librefang-kernel --lib: 1085/1085
- cargo test -p librefang-migrate --lib: 51/51
- cargo test -p librefang-api --lib --features all-channels: 692/692
- cargo test -p librefang-api --test channels_routes_test --features
all-channels: 38/38
- cargo test -p librefang-api --test config_routes_integration
--features all-channels: 31/31
- cd sdk/python && pytest: 488/488 (72 new for slack)
…nd mirror `resolve_channel_owner` only scanned the in-process `cfg.channels` (via `for_each_channel_field!`), so once a channel migrated to a sidecar (`[[sidecar_channels]]`) it returned None and the agent's outbound `channel_send` was no longer mirrored back into the channel-owning agent's session (#4824). This silently affected every migrated sidecar channel — telegram (#5241), discord (#5299), nextcloud, rocketchat, reddit, bluesky, mastodon, and now slack — not just the one this PR migrates. Consult `cfg.sidecar_channels[*].default_agent` too (the same field that seeds inbound routing via `AgentRouter.channel_defaults`), keyed by `channel_type` falling back to `name`, so the mirror resolves an owner uniformly across in-process and sidecar channels. First matching entry with a non-empty default_agent wins (deterministic Vec order). Surfaced by automated review on this PR. Extracted the matching into a pure `sidecar_default_agent` helper with unit coverage (`sidecar_default_agent_matches_by_channel_type_then_name`, `sidecar_default_agent_skips_entries_without_agent_and_is_first_match`).
) * feat(channels)!: migrate slack from in-process adapter to sidecar Replace the 1890-line in-process librefang-channels::slack adapter (Socket Mode WS + Web API) with an out-of-process Python sidecar at sdk/python/librefang/sidecar/adapters/slack.py (stdlib-only). Behaviour parity: - POST /api/auth.test startup probe to discover bot_user_id - POST /api/apps.connections.open to mint Socket Mode WSS URL - Envelope-id ACK loop for events_api / interactive - message + app_mention event handling with message_changed subtype extraction; all-other-subtype skip - Self-skip on bot_id presence OR user == bot_user_id - allowed_channels filter with DM exemption (channels starting with D) - Slash-command routing on /cmd args -> Command (text otherwise) - thread_ts captured as thread_id - DM detection via channel-id prefix; is_group = !channel.startswith("D") - block_actions interactive payload -> ButtonCallback content with action_id / trigger_id / block_action metadata - chat.postMessage send with optional thread_ts + unfurl_links + Block Kit blocks; 3000-char chunking matches SLACK_MSG_LIMIT - eyes reaction on receive flipped to white_check_mark on send-complete (opt-out via SLACK_REACTIONS=false) - force_flat_replies knob posts replies as top-level channel messages instead of threads - sender_user_id metadata key matches Rust SENDER_USER_ID_KEY parity - account_id injection for multi-bot routing One improvement over the Rust adapter: pending-reaction map is now bounded at MAX_PENDING_REACTIONS=2000 entries with oldest-eviction. The Rust adapter used an unbounded RwLock<HashMap> so a flood of inbound messages followed by a hang in the agent loop would grow the map without bound — a small but real memory-leak surface. Two regressions to call out (both match the telegram-#5241 and discord-#5299 precedents; flagged so operators aren't surprised): - **Live Slack workspace-role RBAC is gone.** ChannelRoleQuery is a Rust trait the sidecar process cannot implement, so role_query is None for Slack post-migration; the kernel's resolve_role_for_sender falls through to the default-deny Viewer branch and [channel_role_mapping.slack] is no longer consulted. Workaround: enumerate authorised operators under [users] with channel_bindings = { slack = ["<slack_user_id>"] } and an explicit role. The parse_users_info precedence parser is preserved in the sidecar so a future sidecar-protocol query/response pair can re-use it. - **Per-channel proxy override (#4795) is not wired through.** The sidecar honours HTTP_PROXY / HTTPS_PROXY / ALL_PROXY via stdlib but has no SLACK_PROXY_URL env var yet. Filed as a follow-up. Cascade removal: - src/slack.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, cargo features in librefang-channels + librefang-api (incl. membership in all-channels / all-channels-no-email / core-channels / mini) removed - SlackConfig struct + Default impl + canonical deny_unknown_fields rustdoc anchor moved to WhatsAppConfig; ChannelsConfig.slack field + default removed; validation hook removed - channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion removed - routes/channels.rs: ChannelMeta entry + 5 match arms removed; live-test slack branch (POST to slack.com/api/chat.postMessage) retired with a comment explaining why; SIDECAR_CATALOG slack entry added; instance_helper_tests reanchored on matrix; test_channel tests reanchored on matrix - routes/config.rs ch!(slack) call removed; is_writable_config_path test fixtures switched to matrix - kernel channel_sender for_each_channel_field! macro entry + EXPECTED name-list entry removed - CLI: librefang setup slack wizard arm + channel list row removed; TUI ChannelDef removed - channels_routes_test: SlackConfig fixtures retargeted to MatrixConfig (bot_token_env -> access_token_env), test names / display_name assertions updated, "other rows must not flip" check uses mattermost instead - routes/skills.rs tests: channels.slack -> channels.matrix bulk rename across all TOML write/read tests; ad-hoc local Slack struct renamed to Matrix - types/config/mod.rs: test_slack_config_defaults + test_slack_config_unfurl_links_deserialization tests removed (covered by slack.rs unit tests which were also deleted); validate_missing_env_vars retargeted to WhatsAppConfig; test_one_or_many_single_toml_table + test_one_or_many_array_of_tables retargeted to MatrixConfig; well_formed_repeated_channel_table_still_parses_5130 drops the slack fixture (slack was the last working anchor — now only whatsapp + mattermost remain as deny_unknown_fields drift sentinels) - openclaw migrator: slack channel input now produces a SkippedItem with a sidecar-redirect reason rather than writing [channels.slack] (mirrors telegram + discord); test_policy_migration retargeted at mattermost (slack and discord both sidecar-skipped); test_full_migration legacy YAML fixture now includes mattermost.yaml so the "report.imported.iter().any(Channel)" assertion still has a witness; test_json5_channel_extraction adds mattermost and asserts all three of telegram/discord/slack are sidecar-skipped - openfang test fixtures reanchored on [channels.whatsapp] - librefang.toml.example + cli init_default_config.toml: [channels.slack] block replaced with commented [[sidecar_channels]] template - docs (en + zh): configuration/page.mdx, configuration/channels/page.mdx, integrations/channels/page.mdx, integrations/channels/core/page.mdx — [channels.slack] blocks replaced with [[sidecar_channels]] templates - ChannelType::Slack enum variant kept (still used by the bridge / router; same pattern as ChannelType::Telegram / ChannelType::Discord) New env-var knobs (read by the sidecar from [sidecar_channels.env]): SLACK_ALLOWED_CHANNELS, SLACK_UNFURL_LINKS (tri-state), SLACK_FORCE_FLAT_REPLIES, SLACK_REACTIONS, SLACK_ACCOUNT_ID. SLACK_APP_TOKEN and SLACK_BOT_TOKEN go to ~/.librefang/secrets.env. Operator action required: an existing [channels.slack] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.slack. Verification: - cargo check --workspace --lib: clean - cargo check --workspace --features librefang-api/all-channels: clean - cargo clippy --workspace --all-targets --features librefang-api/all-channels -- -D warnings: zero warnings - cargo test -p librefang-channels --features all-channels --lib: 766/766 - cargo test -p librefang-types --lib: 838/838 - cargo test -p librefang-kernel --lib: 1085/1085 - cargo test -p librefang-migrate --lib: 51/51 - cargo test -p librefang-api --lib --features all-channels: 692/692 - cargo test -p librefang-api --test channels_routes_test --features all-channels: 38/38 - cargo test -p librefang-api --test config_routes_integration --features all-channels: 31/31 - cd sdk/python && pytest: 488/488 (72 new for slack) * docs(changelog): add missing attribution and prune stale history Two CHANGELOG fixes: 1. Add the missing `(@houko)` attribution to the Slack and Discord RBAC-regression bullets under [Unreleased] — the only thing failing PR 5302's "CHANGELOG Attribution" / "CHANGELOG Attribution (full [Unreleased])" CI checks. 2. Remove a stale, misplaced second `## [Unreleased]` block (wedged between the 2026.5.2 and 2026.4.28 dated releases, an orphan left by the #4240 dual-section merge) and the pre-CalVer 0.x version history (`[0.7.0]` … `[0.1.0]`). The duplicate [Unreleased] was tripping the pre-commit duplicate-section guard; the 0.x block is long-dead release history still recoverable from git. All 2026.x CalVer history is preserved. * fix(kernel): resolve channel owner for sidecar channels in channel_send mirror `resolve_channel_owner` only scanned the in-process `cfg.channels` (via `for_each_channel_field!`), so once a channel migrated to a sidecar (`[[sidecar_channels]]`) it returned None and the agent's outbound `channel_send` was no longer mirrored back into the channel-owning agent's session (#4824). This silently affected every migrated sidecar channel — telegram (#5241), discord (#5299), nextcloud, rocketchat, reddit, bluesky, mastodon, and now slack — not just the one this PR migrates. Consult `cfg.sidecar_channels[*].default_agent` too (the same field that seeds inbound routing via `AgentRouter.channel_defaults`), keyed by `channel_type` falling back to `name`, so the mirror resolves an owner uniformly across in-process and sidecar channels. First matching entry with a non-empty default_agent wins (deterministic Vec order). Surfaced by automated review on this PR. Extracted the matching into a pure `sidecar_default_agent` helper with unit coverage (`sidecar_default_agent_matches_by_channel_type_then_name`, `sidecar_default_agent_skips_entries_without_agent_and_is_first_match`). * fix(slack): thread top-level replies and finalize 👀 on the right message `parse_slack_event` set `thread_id = thread_ts` only, so a top-level message carried `thread_id = None`. Two consequences: 1. The bot's reply posted at the channel root instead of threading under the triggering message. `force_flat_replies` exists to opt *out* of threading, so threaded-by-default is the intended behaviour (and what the Rust adapter did). 2. `on_send` finalized the `👀` → `✅` reaction using the posting `thread_ts` (forced to None under SLACK_FORCE_FLAT_REPLIES, and None anyway for top-level), so it fell back to "first pending reaction in the channel" and flipped the wrong message under concurrency, leaving the real request stuck on `👀`. `thread_id` now falls back to the message's own `ts` (mirroring rocketchat / nextcloud's `thread_id = parent or own_id`), and `on_send` finalizes against the inbound thread id rather than the force-flattened posting `thread_ts`. The eyes is tracked by the message's own ts, so the top-level case (the dominant @-mention pattern) now finalizes exactly. In-thread replies stay best-effort — the `Send` protocol carries no inbound `message_id` to key on. Surfaced by automated review on PR #5302. Tests: test_parse_event_top_level_thread_id_falls_back_to_ts, test_on_send_force_flat_finalizes_correct_message. Full sdk/python suite: 548 passed.
) * feat(channels)!: migrate webex from in-process adapter to sidecar Replaces the 645-line in-process librefang-channels::webex adapter (Cisco Mercury WebSocket gateway for activity events + GET /messages/<id> REST follow-up + POST /messages publish) with the Python sidecar librefang.sidecar.adapters.webex. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299, nextcloud #5301, slack #5302. Behaviour parity: - GET /people/me startup credential probe (bot id + display name) - Hard-coded Mercury WSS URL with Bearer auth on upgrade request (no device-registration handshake — the Rust adapter never did one either) - data.activity envelope parsing; verb=="post" filter; actor-id self-skip - GET /messages/<id> REST follow-up for the full message body - Room filter (empty allowlist = all rooms the bot is in) - Slash-command (/cmd args) → Command routing - roomType == "group" → is_group mapping - Multi-bot account_id metadata injection - 7439-char message chunking (WEBEX_MSG_LIMIT parity with the Rust MAX_MESSAGE_LEN) - Exponential reconnect backoff (1s → 60s) - ChannelType::Custom("webex") preserved as channel_type = "webex" on the sidecar entry — existing routing / channel_role_mapping keys that reference webex continue to resolve. Four improvements over the Rust adapter (file/line evidence in CHANGELOG): 1. parentId outbound threading wired. The Rust api_send_message (webex.rs:171-201) posted just {"roomId", "text"} — Webex's parentId field was never sent and chunked replies always landed at the room root regardless of inbound context; the inbound side dropped the message id entirely (thread_id: None at webex.rs:438). The sidecar surfaces the inbound id (or inbound parentId when inside a thread) as thread_id, and on_send posts parentId populated so replies thread correctly. Mirrors reddit / rocketchat / nextcloud / mastodon / bluesky. 2. 429 Retry-After honoured on both GET /messages/<id> fetch and POST /messages send. The Rust adapter had no 429 handling at either call site, so a server-side rate-limit either lost the inbound fetch or returned Err and dropped the outbound. The sidecar parses Retry-After (default 30 s fallback, floor 1 s, cap MAX_BACKOFF_SECS), sleeps, and retries once before logging-and-continuing on the second 429. Same shape as #5303. 3. Mercury activity-id dedupe. Mercury can re-deliver an activity.object.id on reconnect (Rust adapter had no dedupe, see unconditional emit at webex.rs:459). Bounded local set on activity.object.id with SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000. 4. Explicit HTTP timeouts on every urlopen call (urllib has no default timeout; a hung Webex API would hang the producer thread forever). Cascade removal: - src/webex.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, cargo features in librefang-channels + librefang-api (incl. membership in all-channels / all-channels-no-email) removed - WebexConfig struct + Default impl + ChannelsConfig.webex field + matching Default entry removed; validation hook removed - channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion removed - routes/channels.rs: ChannelMeta entry + 4 match arms removed; SIDECAR_CATALOG webex entry added - routes/config.rs ch!(webex) call removed - kernel channel_sender for_each_channel_field! macro entry + EXPECTED name-list entry removed - CLI: TUI ChannelDef row removed, migrated-to-sidecar comment grown - README / AGENTS for librefang-channels updated to list webex among the migrated-to-sidecar adapters - Docs (EN + ZH): configuration/page.mdx, configuration/channels/page.mdx, integrations/channels/enterprise/page.mdx, configuration/security/page.mdx, architecture/page.mdx — [channels.webex] blocks replaced with [[sidecar_channels]] redirect blocks (fence language `bash` per Shiki); architecture wave table moves webex out of Wave 3 in-process and into the sidecar list New env-var knobs (read from [sidecar_channels.env]): - WEBEX_ALLOWED_ROOMS (comma-separated room IDs, empty = allow all) - WEBEX_ACCOUNT_ID (optional, multi-bot routing key) Secret in ~/.librefang/secrets.env: WEBEX_BOT_TOKEN. Operator action required: an existing [channels.webex] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.webex with WEBEX_BOT_TOKEN (in ~/.librefang/secrets.env) and any of the optional knobs above (in [sidecar_channels.env]). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib: 420 + 838 + 1087 passed - cargo test -p librefang-api --lib: 685 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cd sdk/python && pytest tests/test_webex_adapter.py: 78 passed * fix(channels/webex): apply review nits — display name, suppress flag, dead code, redundant filter Self-review pass on top of the migration commit; addresses 4 nits from the pre-merge review without changing the migration's core behaviour: 1. **personDisplayName drives user_name** (`webex.py:281-291, 326-330`). The migration commit preserved the Rust adapter's `webex.rs:431` behaviour of using `personEmail` unconditionally, which leaks emails into bot logs and dashboard UI. Now `parse_webex_message` prefers `personDisplayName` from the `/messages/<id>` body and falls back to `personEmail` when the field is absent (older Webex orgs / some service-account principals) or empty; `personEmail` stays in metadata for routing / audit. 3 new tests (`test_parse_prefers_person_display_name_for_user_name`, `…_falls_back_to_email_when_display_name_missing`, `…_falls_back_to_email_when_display_name_empty`). 2. **Explicit `suppress_error_responses = False`** on `WebexAdapter` with a precedent comment naming the discord / slack / twitch chat-room default vs. the mastodon / bluesky / reddit / nextcloud public-broadcast default. The migration left this implicit; an explicit class-level attr matches how `twitch.py:312` documents the choice so the next reader doesn't have to reconstruct the reasoning. 3. **`register_webhook` dead-code drop documented in CHANGELOG**. The Rust adapter carried a never-wired `register_webhook` helper at `webex.rs:137-168` (marked `#[allow(dead_code)]`) for an HTTP-webhook delivery alternative the channel-bridge never enabled; the sidecar drops it. The CHANGELOG now names this explicitly and points operators who want webhook delivery at the canonical generic `[[sidecar_channels]]` running `librefang.sidecar.adapters.webhook`. 4. **Skip redundant room filter inside `parse_webex_message`** when called from `_handle_envelope` (`webex.py:867-878`). `_handle_envelope` already filtered against `self.allowed_rooms` at line 855 before paying for the REST fetch; passing `allowed_rooms=[]` into the parser drops one redundant list-membership check per inbound message. The parser keeps the parameter for direct callers (tests; future code paths). Verification (run inside the worktree): * `cd sdk/python && python3 -m pytest tests/test_webex_adapter.py -q` → **81 passed** (was 78; +3 personDisplayName tests). No existing test modified. * `cargo check` / `clippy` not re-run — this commit is Python + CHANGELOG only, no Rust touched. Diff: +73 / -5 across `CHANGELOG.md`, `sdk/python/librefang/sidecar/adapters/webex.py`, `sdk/python/tests/test_webex_adapter.py`.
* feat(channels)!: migrate line from in-process adapter to sidecar Replaces the 881-line in-process librefang-channels::line adapter (axum webhook route at /channels/line/webhook on the shared API server, X-Line-Signature HMAC-SHA256 verification, POST /v2/bot/message/push for outbound) with the Python sidecar librefang.sidecar.adapters.line. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299, nextcloud #5301, slack #5302, webex #5309. Behaviour parity: - GET /v2/bot/info startup credential probe (bot display name) - HTTP webhook server (BaseHTTPRequestHandler over ThreadingTCPServer) on LINE_WEBHOOK_PORT (default 9090) + LINE_WEBHOOK_PATH (default /webhook) - X-Line-Signature HMAC-SHA256 verification over the raw wire bytes (constant-time compare, line.rs:229-250 parity, incl. the wire-bytes-vs-JSON-roundtrip regression) - Only message events with type==text (other event types and non-text message types dropped, line.rs:256-273) - Source-type -> reply_to mapping (user -> userId, group -> groupId, room -> roomId; group/room -> is_group=true, line.rs:280-290) - Slash-command routing (/cmd args -> Command) - Metadata preservation: user_id / reply_to / reply_token / source_type - Multi-bot account_id injection via LINE_ACCOUNT_ID - 5000-char chunking (LINE_MSG_LIMIT parity with MAX_MESSAGE_LEN at line.rs:39) - Image-branch wire shape (originalContentUrl + previewImageUrl both set to the caller URL, caption sent as follow-up text push, line.rs:464-490) - ChannelType::Custom("line") preserved as channel_type = "line" on the sidecar entry so existing routing / channel_role_mapping keys continue to resolve Three improvements over the Rust adapter: 1. 429 Retry-After honoured on outbound. The Rust api_push_message (line.rs:148-184) had no 429 handling; throttled push returned Err and the chunked reply dropped on the floor. Sidecar parses Retry-After (default 30 s fallback, floor 1 s, cap MAX_BACKOFF_SECS), sleeps, retries once, then logs-and-continues on the second 429. Same shape as #5303. 2. Inbound dedupe on message.id. LINE redelivers webhook events when the operator's endpoint fails; the Rust handler at line.rs:413-427 emitted every event unconditionally, causing duplicate agent invocations on transient failures. Bounded local set on message.id with SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000 (matches reddit / rocketchat / nextcloud / webex policy). 3. Explicit HTTP timeouts on every urlopen call (urllib has no default timeout; a hung LINE API would otherwise hang the worker thread forever). Cascade removal: - src/line.rs deleted; lib.rs mod decl, channels-allowlist.txt entry, cargo features in librefang-channels + librefang-api (incl. membership in all-channels / all-channels-no-email) removed - LineConfig struct + Default impl + ChannelsConfig.line field + matching Default entry removed; validation hook removed - channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion removed - routes/channels.rs: ChannelMeta entry + 5 match arms removed (is_some / serialize / len / ser / is_channel_configured); webhook_route_suffix allowlist entry removed (sidecar runs its own webhook listener, not on the shared API port); SIDECAR_CATALOG line entry added - routes/config.rs ch!(line) call removed - kernel channel_sender for_each_channel_field! entry + EXPECTED name-list entry removed - CLI: TUI ChannelDef row removed, migrated-to-sidecar comment grown - README / AGENTS for librefang-channels updated; the mandatory- HMAC paragraph now lists Teams + DingTalk only (LINE moves to the sidecar but keeps the same X-Line-Signature contract inside the new process) - Docs (EN + ZH): configuration/page.mdx, configuration/channels/page.mdx, configuration/security/page.mdx, integrations/channels/integrations/page.mdx - [channels.line] blocks replaced with [[sidecar_channels]] redirect blocks; security env-var table flags both LINE_CHANNEL_SECRET and LINE_CHANNEL_ACCESS_TOKEN as sidecar-only; validation table drops the LINE row New env-var knobs (read from [sidecar_channels.env]): - LINE_WEBHOOK_PORT (default 9090) - LINE_WEBHOOK_PATH (default /webhook) - LINE_BIND_HOST (default 0.0.0.0) - LINE_ACCOUNT_ID (optional, multi-bot routing key) Secrets in ~/.librefang/secrets.env: LINE_CHANNEL_SECRET + LINE_CHANNEL_ACCESS_TOKEN. Operator action required (substantive): the sidecar runs its OWN HTTP webhook server - it is no longer mounted on the LibreFang API port - so the webhook URL registered at the LINE Developers Console must be updated to point at the sidecar host (typical pattern: an HTTPS reverse proxy in front of the sidecar's LINE_WEBHOOK_PORT). An existing [channels.line] block is no longer recognised - re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.line with LINE_CHANNEL_SECRET + LINE_CHANNEL_ACCESS_TOKEN (in ~/.librefang/secrets.env) and any of the optional knobs above (in [sidecar_channels.env]). Verification: - cargo check --workspace --lib: clean - cargo clippy --workspace --all-targets -- -D warnings: clean - cargo fmt --check: clean - cargo test --lib -p librefang-types -p librefang-channels -p librefang-kernel: 420 + 838 + 1087 passed - cargo test --lib -p librefang-api: 685 passed - cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test: 38 + 2 passed - cargo xtask channel-policy: passed - cd sdk/python && pytest tests/test_line_adapter.py: 68 passed * docs(channels): correct line sidecar HTTP server class references The module docstring claimed `ThreadingHTTPServer` but the implementation uses `socketserver.ThreadingTCPServer` (intentional: skips `HTTPServer.server_bind()`'s `socket.getfqdn()` DNS lookup on startup). Align the three doc references with the actual class and spell out why we don't use `ThreadingHTTPServer` so the next reader doesn't "fix" it back.
Replaces the 758-line in-process `librefang-channels::qq` adapter (token mint + WebSocket gateway + HELLO/IDENTIFY/READY handshake + heartbeat + DISPATCH for 4 event types + REST send with markdown stripping) with an out-of-process Python sidecar at `sdk/python/librefang/sidecar/adapters/qq.py`. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299, nextcloud #5301, slack #5302, webex #5309, line #5312, zulip #5310, mattermost #5315, signal #5317. Behaviour parity (every claim cited against the pre-migration tree): - Token fetch: POST bots.qq.com/app/getAppAccessToken (qq.rs:542-557). - Gateway: GET /gateway with Bearer auth (qq.rs:559-573). - WS handshake: HELLO(op=10) → IDENTIFY(op=2) with token = "QQBot <access_token>", intents bitmask, shard=[0,1] → READY dispatch (qq.rs:413-433). - Heartbeat: op=1 frame at the server-supplied interval, carrying last seen `s` from dispatch frames (qq.rs:359-368). - 4 dispatch event types: MESSAGE_CREATE / AT_MESSAGE_CREATE → /channels/{channel_id}/messages, DIRECT_MESSAGE_CREATE → /dms/{guild_id}/messages, GROUP_AT_MESSAGE_CREATE → /v2/groups/{group_openid}/messages, C2C_MESSAGE_CREATE → /v2/users/{user_openid}/messages (qq.rs:194-224). - Bot-mention `/` strip + slash-command routing (qq.rs:227). - User allowlist + multi-bot account_id metadata (#5003, qq.rs:400-405). - Outbound markdown stripping pipeline preserves order: think tags → code blocks → inline code → bold → italic → headings → table separators → links → blockquotes → HRs → triple-newline collapse (qq.rs:137-180). - 2000-char chunking (qq.rs:26), 1–60 s exponential reconnect backoff (qq.rs:282). Improvements over the Rust adapter ================================== 1. **Reply context actually round-trips.** The Rust `parse_dispatch_event` (qq.rs:182-246) computed `reply_endpoint` and `msg_id` but the dispatch loop bound them to `_endpoint` / `_msg_id` (qq.rs:399) and dropped them on the floor; `send` (qq.rs:497-498) then expected `user.platform_id` to be encoded as `"<endpoint>|<msg_id>"` and silently no-op'd when the delimiter wasn't there. The Rust adapter therefore failed every real outbound — only the synthetic wiremock tests at qq.rs:686-712 exercised the working shape. The sidecar surfaces the reply endpoint as `channel_id` and the QQ `msg_id` as `thread_id` on the inbound event so the bridge round-trips them through to `on_send`, which posts to `{api_base}{channel_id}` with the correct passive-reply `msg_id`. 2. **Inbound dedupe on `msg.id`.** Rust emitted every parsed event unconditionally (qq.rs:399-410); WS reconnect could replay. Bounded local set, SEEN_MAX=10000 / EVICT=5000. 3. **429 Retry-After honoured on every REST path.** Token, gateway, and outbound send all sleep + retry once; second 429 logs-and-continues (matches webex / line / mattermost / signal #5303). 4. **Explicit 15s urlopen timeouts on every REST call.** Rust pre-configured reqwest's 30s default at qq.rs:71; sidecar passes `timeout=SEND_TIMEOUT_SECS` (15 s) so a misbehaving endpoint trips an explicit error. Cascade removal ================ - `src/qq.rs` deleted; `lib.rs` mod, `channels-allowlist.txt` entry, both `Cargo.toml` features removed. - `QqConfig` struct + Default + `ChannelsConfig.qq` field gone. - `channel_bridge.rs` import / spawn block / `check_channel!` / find_channel_info!(qq) (none existed) removed. - `routes/channels.rs` `ChannelMeta` entry + 4 match arms (`is_some` / serialize / `len` / `ser`) removed; SIDECAR_CATALOG `qq` entry added. - `routes/config.rs` `ch!(qq)` removed. - `channel_sender.rs` `for_each_channel_field!` macro entry + `EXPECTED` name-list entry removed. - CLI / TUI had no QQ wizard arm to remove. - librefang-migrate had no QQ block to handle (QQ wasn't in the OpenClaw schema). - Docs EN + ZH (6 files): `[channels.qq]` blocks replaced with `[[sidecar_channels]]` redirects; integrations tutorial rewritten for the sidecar. New env-var knobs (`[sidecar_channels.env]`) ============================================= - `QQ_APP_ID` (required) - `QQ_ALLOWED_USERS` (optional, CSV) - `QQ_ACCOUNT_ID` (optional) - `QQ_INTENTS` (optional, decimal/hex bitmask override) Secret (`~/.librefang/secrets.env`) ==================================== - `QQ_APP_SECRET` (required) Operator action required ======================== An existing `[channels.qq]` block is no longer recognised — re-declare as `[[sidecar_channels]]` running `python3 -m librefang.sidecar.adapters.qq`. See `sdk/python/librefang/sidecar/adapters/qq.py` header for the exact config. `ChannelType::Custom("qq")` is preserved via `channel_type = "qq"` on the sidecar entry so existing routing and `channel_role_mapping` keys continue to resolve. Verification ============ - `cargo check -p librefang-types -p librefang-channels -p librefang-api -p librefang-kernel -p librefang-cli --lib` — clean - `cargo xtask channel-policy` — passed (`qq` removed from allowlist) - `pytest sdk/python/tests/test_qq_adapter.py -q` — **77 passed** (covers env parsing, markdown stripping, parse_qq_event for all 4 event types + edge cases, _mark_seen LRU eviction, _fetch_token + _fetch_gateway happy path / 429 / non-200 / missing field, _post_message chunking + 429 retry-once + persistent-429 fail-open + 5xx fail-open, on_send wiring with markdown stripping at the outbound boundary, WS gateway flow via mock _WebSocketClient — HELLO → IDENTIFY token+intents+ shard, DISPATCH emission, msg.id dedupe, RECONNECT/INVALID_SESSION, heartbeat after interval, --describe schema, capabilities contract). Co-authored-by: Evan <[email protected]>
Summary
sdk/python/librefang/sidecar/adapters/discord.py. Same Gateway v10 protocol, same REST send shape, same filters / mention detection / attachment dispatch / 2 000-UTF-16-unit chunking, but two genuine improvements: periodic client-side heartbeats (the Rust adapter never sent its own, so sessions dropped every ~45 s) andRetry-After-honouring 429 retry (the Rust send-path was fail-open on 429, silently losing messages).discord.rsdeleted,channel-discordcargo features dropped (incl.core-channels/mini),DiscordConfigdeleted (with the canonicaldeny_unknown_fieldsdoc anchor reseated onSlackConfig),ChannelsConfig.discordfield +for_each_channel_field!entry + CLI wizard arm + dashboardChannelMetaentry + live-testdiscordREST branch all gone.ChannelType::Discordenum variant stays — it's used by the router / bridge and was preserved across the telegram migration too (feat(channels)!: remove in-process telegram adapter (now sidecar-only) #5241).SkippedItemwith a sidecar-redirect message instead of writing[channels.discord](mirrors how telegram was handled). EN + ZH docs andlibrefang.toml.exampleupdated.Operator action required: an existing
[channels.discord]block is no longer recognised — re-declare as a[[sidecar_channels]]runningpython3 -m librefang.sidecar.adapters.discordwithDISCORD_BOT_TOKENin~/.librefang/secrets.envand any ofDISCORD_ALLOWED_GUILDS/DISCORD_ALLOWED_USERS/DISCORD_INTENTS/DISCORD_IGNORE_BOTS/DISCORD_MENTION_PATTERNS/DISCORD_ACCOUNT_IDin[sidecar_channels.env].Test plan
cargo check --workspace --lib— cleancargo check --workspace --features 'librefang-api/all-channels'— cleancargo clippy --workspace --all-targets --features 'librefang-api/all-channels' -- -D warnings— zero warningscargo test -p librefang-channels --features all-channels --lib— 824/824cargo test -p librefang-channels --features all-channels --test bridge_integration_test— 27/27cargo test -p librefang-types --lib— 842/842cargo test -p librefang-kernel --lib— 1085/1085cargo test -p librefang-migrate --lib— 51/51cargo test -p librefang-api --lib --features all-channels— 692/692cargo test -p librefang-api --test channels_routes_test --features all-channels— 38/38cd sdk/python && pytest— 293/293 (68 new for discord + 225 pre-existing)DISCORD_BOT_TOKEN, which Claude could not exercise)