Skip to content

feat(channels)!: migrate nextcloud from in-process adapter to sidecar#5301

Merged
houko merged 1 commit into
mainfrom
feat/channels-nextcloud-sidecar
May 20, 2026
Merged

feat(channels)!: migrate nextcloud from in-process adapter to sidecar#5301
houko merged 1 commit into
mainfrom
feat/channels-nextcloud-sidecar

Conversation

@houko

@houko houko commented May 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the 640-line in-process librefang-channels::nextcloud adapter with the Python sidecar librefang.sidecar.adapters.nextcloud. Same pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298.

Behaviour parity

  • GET /ocs/v2.php/cloud/user startup credential probe
  • Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default
  • Empty allowlist auto-discovers via apps/spreed/api/v4/room
  • Same Bearer + OCS-APIRequest: true headers
  • Slash-command (/cmd args) routing
  • Multi-bot account_id metadata injection
  • 32000-char message chunking (MAX_MESSAGE_LEN)
  • Per-room transport-error isolation
  • 304-as-no-op handling of Talk's long-poll-expired response
  • suppress_error_responses (Talk rooms are multi-participant)

Three improvements over the Rust adapter

  1. Outbound threading via replyTo is now actually wired. The Rust api_send_message (crates/librefang-channels/src/nextcloud.rs:130-160 on main) posted just {\"message\": ...} — Talk's replyTo form parameter was never sent and chunked / threaded replies always landed at the room root. The sidecar surfaces the inbound id (or parentMessage.id when inside a thread) as thread_id, and on_send posts replyTo so replies thread correctly. Mirrors reddit / bluesky / mastodon / rocketchat.
  2. Self-skip on (actorType, actorId) == (\"users\", own_user_id) rather than actorId alone. The Rust adapter compared on actorId (nextcloud.rs:338 on main) without inspecting actorType, so a Talk guest / federated_users actor whose id happened to match the bot's user id silently spoofed self-skip. Parallels the rocketchat feat(channels)!: migrate rocketchat from in-process adapter to sidecar #5298 fix.
  3. Dedupe set on id with bounded SEEN_MESSAGES_MAX=10000 / SEEN_MESSAGES_EVICT=5000 eviction. The Rust adapter only relied on the server-side lastKnownMessageId cursor (nextcloud.rs:347-354 on main); under retry / re-poll boundaries Talk can resend the same id. Same policy as reddit / rocketchat.

Cascade

  • crates/librefang-channels/src/nextcloud.rs deleted
  • lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud feature in librefang-channels + librefang-api (incl. all-channels / all-channels-no-email) removed
  • NextcloudConfig + ChannelsConfig.nextcloud field + Default entry removed
  • validation.rs env-var hook removed
  • channel_bridge.rs: import, spawn block, find_channel_info! arm, check_channel! arm, default-empty test assertion removed
  • routes/channels.rs: ChannelMeta entry + 4 match arms (is_some / serialize / len / ser) removed; SIDECAR_CATALOG nextcloud entry added
  • routes/config.rs: ch!(nextcloud) removed
  • kernel/handles/channel_sender.rs: for_each_channel_field! entry + EXPECTED name removed
  • librefang-cli TUI ChannelDef removed; trailing comment widened to include nextcloud
  • Docs (EN + ZH) — [channels.nextcloud] blocks replaced with [[sidecar_channels]] redirect across configuration/page.mdx, configuration/channels/page.mdx, configuration/security/page.mdx, integrations/channels/integrations/page.mdx, integrations/api/communication/page.mdx, architecture/page.mdx
  • Architecture-row tables move Nextcloud Talk into the sidecar list
  • kernel_config_schema.golden.json regenerated via --ignored regenerate_golden
  • CHANGELOG [Unreleased] → Changed bullet added above the rocketchat entry

New env-var knobs

  • NEXTCLOUD_SERVER_URL (replaces server_url)
  • Optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty = auto-discover)
  • Optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing
  • Optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1)

Operator action required

An existing [channels.nextcloud] block is no longer recognised — re-declare as a [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.nextcloud with env var NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN (in ~/.librefang/secrets.env).

Test plan

  • cargo check --workspace --lib — clean
  • cargo clippy --workspace --all-targets -- -D warnings — clean
  • cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib — 842 passed
  • cargo test -p librefang-api --lib — 686 passed
  • cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test — 38 + 2 passed
  • cd sdk/python && python3 -m pytest tests/test_nextcloud_adapter.py -v — 58 passed
  • grep -rn -iE 'channels\.nextcloud|channel-nextcloud|NextcloudAdapter|NextcloudConfig' docs/ crates/ — no non-historical hits remain (the surviving matches are the new sidecar docs' historical-redirect notes, mirroring rocketchat feat(channels)!: migrate rocketchat from in-process adapter to sidecar #5298)

The new pytest suite covers env-var enforcement, server URL normalization, poll-interval clamping, message chunking, _verify_credentials (OCS + Bearer header shape, 401, missing-id fallback), apps/spreed/api/v4/room discovery, _parse_message (basic text, thread-reply uses inbound parentMessage.id incl. string variant, self-skip on (actorType,actorId), guest-with-matching-id is NOT self, self-skip disabled when own_user_id empty, system-message skip, empty-body skip, command form, no-args command, referenceId in metadata, malformed input, non-integer id graceful handling), _poll_once (emit + watermark advance, dedupe across id repeats, self-skip still marks seen, account_id injection, 401 raises, 304 no-op, per-room transport-error isolation, 500 logged-and-skipped, URL + auth-header shape), dedupe-set capacity eviction, _post_message (form-encoded shape, replyTo on thread, multi-chunk preserves replyTo, missing-room rejection, non-2xx surfaced), on_send wiring (uses cmd.user.platform_id as room token, threads via thread_id, falls back to cmd.channel_id, non-text content → placeholder).

Replaces the 640-line in-process librefang-channels::nextcloud adapter
with the Python sidecar librefang.sidecar.adapters.nextcloud. Same
pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264,
bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298.

Behaviour parity:
- GET /ocs/v2.php/cloud/user startup credential probe
- Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default
- Empty allowlist auto-discovers via apps/spreed/api/v4/room
- Same Bearer + OCS-APIRequest: true headers
- Slash-command (/cmd args) routing
- Multi-bot account_id metadata injection
- 32000-char message chunking (MAX_MESSAGE_LEN)
- Per-room transport-error isolation
- 304-as-no-op handling of Talk's long-poll-expired response
- suppress_error_responses (Talk rooms are multi-participant)

Three improvements over the Rust adapter (file/line evidence in
CHANGELOG):
1. Outbound threading via replyTo is now actually wired. The Rust
   api_send_message (nextcloud.rs:130-160) posted just
   {"message": ...} — Talk's replyTo form parameter was never sent
   and chunked/threaded replies always landed at the room root.
   The sidecar surfaces the inbound id (or parentMessage.id when
   inside a thread) as thread_id, and on_send posts replyTo so
   replies thread correctly. Mirrors reddit/bluesky/mastodon/
   rocketchat.
2. Self-skip on (actorType, actorId) == ("users", own_user_id)
   rather than actorId alone. The Rust adapter compared on
   actorId (nextcloud.rs:338) without inspecting actorType, so a
   Talk guest/federated_users actor whose id happened to match
   the bot's user id silently spoofed self-skip. Parallels the
   rocketchat #5298 fix.
3. Dedupe set on id, bounded SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000. The Rust adapter only relied on the
   server-side lastKnownMessageId cursor (nextcloud.rs:347-354);
   under retry/re-poll boundaries Talk can resend the same id.
   Same policy as reddit/rocketchat.

Cascade removal: crates/librefang-channels/src/nextcloud.rs deleted;
lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud
feature in librefang-channels + librefang-api (incl. all-channels /
all-channels-no-email membership); NextcloudConfig +
ChannelsConfig.nextcloud field + Default; validation env-var hook;
channel_bridge import + spawn block + find_channel_info! arm +
check_channel! arm + default-empty test assertion; routes/channels.rs
ChannelMeta + 4 match arms; routes/config.rs ch!(nextcloud); kernel
channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef;
docs (en + zh): configuration / channels / integrations /
architecture / communication / security pages — [channels.nextcloud]
blocks replaced with [[sidecar_channels]] redirect; architecture-row
moves Nextcloud Talk into the sidecar list. routes/channels.rs
SIDECAR_CATALOG gains a nextcloud entry. Golden fixture regenerated.

New env-var knobs: NEXTCLOUD_SERVER_URL (replaces server_url),
optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty =
auto-discover), optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing,
optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1).

Operator action required: an existing [channels.nextcloud] block is
no longer recognised — re-declare as a [[sidecar_channels]] running
python3 -m librefang.sidecar.adapters.nextcloud with env var
NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN
(in ~/.librefang/secrets.env).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels
  -p librefang-kernel --lib: 842 passed
- cargo test -p librefang-api --lib: 686 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && pytest tests/test_nextcloud_adapter.py: 58 passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5239602fee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +457 to +460
parent_msg = msg.get("parentMessage")
parent_msg_id: str | None = None
if isinstance(parent_msg, dict):
pv = parent_msg.get("id")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Read Nextcloud thread parent from the correct response field

The parser only looks for parentMessage.id, but the Talk chat API documents threaded reply context under the parent object. In environments returning the documented shape, inbound replies inside an existing thread will not surface the real parent id, so thread_id falls back to the current message id and outbound replyTo attaches to the wrong message (creating a child thread instead of replying in-place). This breaks the main threading behavior introduced by this migration for real threaded conversations.

Useful? React with 👍 / 👎.

@houko
houko force-pushed the feat/channels-nextcloud-sidecar branch from 5239602 to b8e4383 Compare May 20, 2026 04:26
@houko
houko enabled auto-merge (squash) May 20, 2026 04:26
@houko
houko merged commit 89dbd0b into main May 20, 2026
19 of 20 checks passed
@houko
houko deleted the feat/channels-nextcloud-sidecar branch May 20, 2026 04:26

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b8e4383297

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# thread (rather than starting a child under the user's
# reply). Otherwise use the inbound `id`, which Talk accepts
# as the new thread's parent.
thread_id = parent_msg_id or (str(msg_id) if msg_id else None)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid setting replyTo for command-type parent messages

The adapter always derives thread_id from every inbound message id, so follow-up sends always include replyTo. Nextcloud Talk’s chat API only allows replyTo when the parent message type is not system or command; command parents return a 400 for invalid replyTo. Because _parse_message currently allows messageType == "command" through and still assigns a thread id, bot responses to command messages can fail outright instead of posting normally.

Useful? React with 👍 / 👎.

@github-actions github-actions Bot added area/docs Documentation and guides area/channels Messaging channel adapters area/kernel Core kernel (scheduling, RBAC, workflows) area/sdk JavaScript and Python SDKs labels May 20, 2026
houko added a commit that referenced this pull request May 20, 2026
Audit triggered by #5301's nextcloud Retry-After fix turned up the
same gap in four sibling adapters: `bluesky`, `mastodon`, `rocketchat`,
and `ntfy` all ignored `Retry-After` on 429 and fell through to a
generic 1 s → 60 s (1 s → 120 s for ntfy SSE reconnect) exponential
backoff, so a real rate-limit response would have the producer / publish
loop keep probing inside the server-side block window and extend the
throttling. `discord`, `reddit`, and `telegram` were already correct
(`_parse_retry_after` / `_retry_after_secs` / `_extract_retry_after`
helpers respectively); `gotify`, `twitch`, and `webhook` are not
applicable (push-only / IRC / inbound-only).

Each affected adapter now mirrors the nextcloud pattern:

* HTTP helper(s) (`_http` / `_post_json` / `_get_json`, or inlined
  `urlopen` paths for mastodon / ntfy) thread response headers through
  with lowercase-normalised keys so `Retry-After` is reachable on both
  the success and HTTPError branches.
* New `_retry_after_secs(headers)` static helper: parses seconds-form
  with floor 1 s, cap `MAX_BACKOFF_SECS`, falls back to a per-adapter
  `RETRY_AFTER_DEFAULT_SECS = 30.0` when the header is absent or
  unparseable. HTTP-date form intentionally not decoded — fallback
  covers the divergence.
* Every 429-reachable call site (`_verify_credentials`, channel /
  room discovery, polling, outbound posting; for mastodon and ntfy
  also initial SSE subscribe) now sleeps the indicated interval then
  raises so the outer backoff pauses before its next pass. Discovery
  paths return empty rather than raise (matches the transport-error
  treatment — one-shot, the next iteration retries).

Verification: 29 new pytest cases (7 bluesky + 8 mastodon + 8
rocketchat + 6 ntfy) covering the parser branches, header-present /
absent paths, and each call site's 429 branch. One pre-existing
mastodon test (`test_post_status_5xx_surfaced`) was corrected — its
mock returned a 5xx-status `_FakeResp`, which doesn't match real
`urlopen` behaviour (raises `HTTPError` on 4xx/5xx); the assertion
was updated to match the corrected error message. All 507 sidecar
tests pass (471 existing + 36 new across the 5 adapter test files
including the 7 nextcloud cases from the prior commits in this PR).

Total diff (this commit only): +805 / -55 across 9 files.
houko added a commit that referenced this pull request May 20, 2026
…ndpoint fix

Follow-up to #5301. The freshly-migrated `librefang.sidecar.adapters.nextcloud`
shipped with the same gap the Rust adapter had: the generic exponential-backoff
loop (1 s → 60 s) ignored `Retry-After`, so a 429 from Nextcloud's
`OCS-Bruteforce-Throttling` middleware (returned after a streak of failed auth
attempts from the same IP) would have the producer thread keep probing inside
the server-side block window and extend the throttling.

Behaviour change (Python-only, no Rust touched):

* `_http(...)` now returns `(status, parsed, raw, headers)` — response headers
  are normalised lowercase so callers can do case-insensitive lookups for
  `Retry-After`. Mirrors the reddit sidecar's `_http` signature.
* New `_retry_after_secs(headers)` static helper: seconds-form parse, floor 1 s,
  cap MAX_BACKOFF_SECS, falls back to a new `RETRY_AFTER_DEFAULT_SECS=30.0`
  constant when the header is absent / unparseable.
* `_verify_credentials`, `_list_joined_rooms`, `_poll_once`, and `_post_message`
  each detect 429, sleep the indicated interval, then raise so the producer
  loop pauses before its next pass. `_list_joined_rooms` returns empty (matches
  its transport-error treatment) rather than raise, because discovery is
  one-shot and the next producer iteration retries.

CHANGELOG also retroactively documents the silent endpoint bug-fix that
#5301 quietly shipped: the Rust adapter polled
`/ocs/v2.php/apps/spreed/api/v4/room/<token>/chat` (`nextcloud.rs:274` on
`89dbd0b5^`), an endpoint Talk's OCS API does not expose for chat — its own
`api_send_message` (line 136) used `/api/v1/chat/<token>`, which is the
documented chat endpoint and which the sidecar uses for both poll and post.
Inbound polling on the Rust adapter was likely silently broken for any
operator using it; the sidecar transparently fixed this on migration. Worth
calling out so operators debugging "messages started flowing after the
sidecar migration" can find the explanation.

Verification (run inside the worktree):

* `cargo check --workspace --lib`: clean (no Rust touched)
* `cd sdk/python && python3 -m pytest tests/test_nextcloud_adapter.py -q`:
  65 passed (was 58; +7 new tests for `_retry_after_secs` (default,
  cap-and-floor, header-absent fallback), `_poll_once` 429 (with header
  honours it; without header falls back to default), `_verify_credentials`
  429, `_list_joined_rooms` 429, `_post_message` 429). All existing tests
  unchanged.

Diff: +228 / -16 across CHANGELOG.md, `sdk/python/librefang/sidecar/adapters/nextcloud.py`,
`sdk/python/tests/test_nextcloud_adapter.py`.
houko added a commit that referenced this pull request May 20, 2026
Audit triggered by #5301's nextcloud Retry-After fix turned up the
same gap in four sibling adapters: `bluesky`, `mastodon`, `rocketchat`,
and `ntfy` all ignored `Retry-After` on 429 and fell through to a
generic 1 s → 60 s (1 s → 120 s for ntfy SSE reconnect) exponential
backoff, so a real rate-limit response would have the producer / publish
loop keep probing inside the server-side block window and extend the
throttling. `discord`, `reddit`, and `telegram` were already correct
(`_parse_retry_after` / `_retry_after_secs` / `_extract_retry_after`
helpers respectively); `gotify`, `twitch`, and `webhook` are not
applicable (push-only / IRC / inbound-only).

Each affected adapter now mirrors the nextcloud pattern:

* HTTP helper(s) (`_http` / `_post_json` / `_get_json`, or inlined
  `urlopen` paths for mastodon / ntfy) thread response headers through
  with lowercase-normalised keys so `Retry-After` is reachable on both
  the success and HTTPError branches.
* New `_retry_after_secs(headers)` static helper: parses seconds-form
  with floor 1 s, cap `MAX_BACKOFF_SECS`, falls back to a per-adapter
  `RETRY_AFTER_DEFAULT_SECS = 30.0` when the header is absent or
  unparseable. HTTP-date form intentionally not decoded — fallback
  covers the divergence.
* Every 429-reachable call site (`_verify_credentials`, channel /
  room discovery, polling, outbound posting; for mastodon and ntfy
  also initial SSE subscribe) now sleeps the indicated interval then
  raises so the outer backoff pauses before its next pass. Discovery
  paths return empty rather than raise (matches the transport-error
  treatment — one-shot, the next iteration retries).

Verification: 29 new pytest cases (7 bluesky + 8 mastodon + 8
rocketchat + 6 ntfy) covering the parser branches, header-present /
absent paths, and each call site's 429 branch. One pre-existing
mastodon test (`test_post_status_5xx_surfaced`) was corrected — its
mock returned a 5xx-status `_FakeResp`, which doesn't match real
`urlopen` behaviour (raises `HTTPError` on 4xx/5xx); the assertion
was updated to match the corrected error message. All 507 sidecar
tests pass (471 existing + 36 new across the 5 adapter test files
including the 7 nextcloud cases from the prior commits in this PR).

Total diff (this commit only): +805 / -55 across 9 files.
houko added a commit that referenced this pull request May 20, 2026
)

* fix(channels/nextcloud): honour OCS Retry-After on 429; doc v1 chat endpoint fix

Follow-up to #5301. The freshly-migrated `librefang.sidecar.adapters.nextcloud`
shipped with the same gap the Rust adapter had: the generic exponential-backoff
loop (1 s → 60 s) ignored `Retry-After`, so a 429 from Nextcloud's
`OCS-Bruteforce-Throttling` middleware (returned after a streak of failed auth
attempts from the same IP) would have the producer thread keep probing inside
the server-side block window and extend the throttling.

Behaviour change (Python-only, no Rust touched):

* `_http(...)` now returns `(status, parsed, raw, headers)` — response headers
  are normalised lowercase so callers can do case-insensitive lookups for
  `Retry-After`. Mirrors the reddit sidecar's `_http` signature.
* New `_retry_after_secs(headers)` static helper: seconds-form parse, floor 1 s,
  cap MAX_BACKOFF_SECS, falls back to a new `RETRY_AFTER_DEFAULT_SECS=30.0`
  constant when the header is absent / unparseable.
* `_verify_credentials`, `_list_joined_rooms`, `_poll_once`, and `_post_message`
  each detect 429, sleep the indicated interval, then raise so the producer
  loop pauses before its next pass. `_list_joined_rooms` returns empty (matches
  its transport-error treatment) rather than raise, because discovery is
  one-shot and the next producer iteration retries.

CHANGELOG also retroactively documents the silent endpoint bug-fix that
#5301 quietly shipped: the Rust adapter polled
`/ocs/v2.php/apps/spreed/api/v4/room/<token>/chat` (`nextcloud.rs:274` on
`89dbd0b5^`), an endpoint Talk's OCS API does not expose for chat — its own
`api_send_message` (line 136) used `/api/v1/chat/<token>`, which is the
documented chat endpoint and which the sidecar uses for both poll and post.
Inbound polling on the Rust adapter was likely silently broken for any
operator using it; the sidecar transparently fixed this on migration. Worth
calling out so operators debugging "messages started flowing after the
sidecar migration" can find the explanation.

Verification (run inside the worktree):

* `cargo check --workspace --lib`: clean (no Rust touched)
* `cd sdk/python && python3 -m pytest tests/test_nextcloud_adapter.py -q`:
  65 passed (was 58; +7 new tests for `_retry_after_secs` (default,
  cap-and-floor, header-absent fallback), `_poll_once` 429 (with header
  honours it; without header falls back to default), `_verify_credentials`
  429, `_list_joined_rooms` 429, `_post_message` 429). All existing tests
  unchanged.

Diff: +228 / -16 across CHANGELOG.md, `sdk/python/librefang/sidecar/adapters/nextcloud.py`,
`sdk/python/tests/test_nextcloud_adapter.py`.

* fix(channels/nextcloud): address review nits — author handle + raise consistency

* CHANGELOG: `(@Vip)` was a typo for `(@houko)` — `@vip` resolves to a
  different (real) GitHub account, so the @-mention would have pinged
  someone unrelated. The `(@user)` pre-commit hook (#3400) only checks
  for duplicate attributions on the same line, not handle-vs-author
  match, so it didn't catch this.

* `_post_message`: the 429 raise was the odd one out
  (`"nextcloud chat POST 429 — rate-limited (retry-after=Ns)"`); the
  other three sites (`_verify_credentials`, `_list_joined_rooms`,
  `_poll_once`) raise the terser `"nextcloud 429 — rate-limited"`. The
  `retry_after_secs` value is already on the structured `log.warn`, so
  dropping it from the exception message removes the duplication.
  Existing `pytest.raises(match="429")` assertions still cover it.

Verification: `pytest tests/test_nextcloud_adapter.py -q` → 65 passed.

* fix(channels): honour Retry-After across sibling sidecar adapters

Audit triggered by #5301's nextcloud Retry-After fix turned up the
same gap in four sibling adapters: `bluesky`, `mastodon`, `rocketchat`,
and `ntfy` all ignored `Retry-After` on 429 and fell through to a
generic 1 s → 60 s (1 s → 120 s for ntfy SSE reconnect) exponential
backoff, so a real rate-limit response would have the producer / publish
loop keep probing inside the server-side block window and extend the
throttling. `discord`, `reddit`, and `telegram` were already correct
(`_parse_retry_after` / `_retry_after_secs` / `_extract_retry_after`
helpers respectively); `gotify`, `twitch`, and `webhook` are not
applicable (push-only / IRC / inbound-only).

Each affected adapter now mirrors the nextcloud pattern:

* HTTP helper(s) (`_http` / `_post_json` / `_get_json`, or inlined
  `urlopen` paths for mastodon / ntfy) thread response headers through
  with lowercase-normalised keys so `Retry-After` is reachable on both
  the success and HTTPError branches.
* New `_retry_after_secs(headers)` static helper: parses seconds-form
  with floor 1 s, cap `MAX_BACKOFF_SECS`, falls back to a per-adapter
  `RETRY_AFTER_DEFAULT_SECS = 30.0` when the header is absent or
  unparseable. HTTP-date form intentionally not decoded — fallback
  covers the divergence.
* Every 429-reachable call site (`_verify_credentials`, channel /
  room discovery, polling, outbound posting; for mastodon and ntfy
  also initial SSE subscribe) now sleeps the indicated interval then
  raises so the outer backoff pauses before its next pass. Discovery
  paths return empty rather than raise (matches the transport-error
  treatment — one-shot, the next iteration retries).

Verification: 29 new pytest cases (7 bluesky + 8 mastodon + 8
rocketchat + 6 ntfy) covering the parser branches, header-present /
absent paths, and each call site's 429 branch. One pre-existing
mastodon test (`test_post_status_5xx_surfaced`) was corrected — its
mock returned a 5xx-status `_FakeResp`, which doesn't match real
`urlopen` behaviour (raises `HTTPError` on 4xx/5xx); the assertion
was updated to match the corrected error message. All 507 sidecar
tests pass (471 existing + 36 new across the 5 adapter test files
including the 7 nextcloud cases from the prior commits in this PR).

Total diff (this commit only): +805 / -55 across 9 files.
@github-actions github-actions Bot added the size/XL 1000+ lines changed label May 20, 2026
houko added a commit that referenced this pull request May 20, 2026
)

* feat(channels)!: migrate webex from in-process adapter to sidecar

Replaces the 645-line in-process librefang-channels::webex adapter
(Cisco Mercury WebSocket gateway for activity events + GET
/messages/<id> REST follow-up + POST /messages publish) with the
Python sidecar librefang.sidecar.adapters.webex. Same pattern as
ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky
#5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299,
nextcloud #5301, slack #5302.

Behaviour parity:
- GET /people/me startup credential probe (bot id + display name)
- Hard-coded Mercury WSS URL with Bearer auth on upgrade request
  (no device-registration handshake — the Rust adapter never did
  one either)
- data.activity envelope parsing; verb=="post" filter; actor-id
  self-skip
- GET /messages/<id> REST follow-up for the full message body
- Room filter (empty allowlist = all rooms the bot is in)
- Slash-command (/cmd args) → Command routing
- roomType == "group" → is_group mapping
- Multi-bot account_id metadata injection
- 7439-char message chunking (WEBEX_MSG_LIMIT parity with the Rust
  MAX_MESSAGE_LEN)
- Exponential reconnect backoff (1s → 60s)
- ChannelType::Custom("webex") preserved as channel_type = "webex"
  on the sidecar entry — existing routing / channel_role_mapping
  keys that reference webex continue to resolve.

Four improvements over the Rust adapter (file/line evidence in
CHANGELOG):
1. parentId outbound threading wired. The Rust api_send_message
   (webex.rs:171-201) posted just {"roomId", "text"} — Webex's
   parentId field was never sent and chunked replies always landed
   at the room root regardless of inbound context; the inbound
   side dropped the message id entirely (thread_id: None at
   webex.rs:438). The sidecar surfaces the inbound id (or inbound
   parentId when inside a thread) as thread_id, and on_send posts
   parentId populated so replies thread correctly. Mirrors reddit
   / rocketchat / nextcloud / mastodon / bluesky.
2. 429 Retry-After honoured on both GET /messages/<id> fetch and
   POST /messages send. The Rust adapter had no 429 handling at
   either call site, so a server-side rate-limit either lost the
   inbound fetch or returned Err and dropped the outbound. The
   sidecar parses Retry-After (default 30 s fallback, floor 1 s,
   cap MAX_BACKOFF_SECS), sleeps, and retries once before
   logging-and-continuing on the second 429. Same shape as
   #5303.
3. Mercury activity-id dedupe. Mercury can re-deliver an
   activity.object.id on reconnect (Rust adapter had no dedupe,
   see unconditional emit at webex.rs:459). Bounded local set on
   activity.object.id with SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000.
4. Explicit HTTP timeouts on every urlopen call (urllib has no
   default timeout; a hung Webex API would hang the producer
   thread forever).

Cascade removal:
- src/webex.rs deleted; lib.rs mod decl, channels-allowlist.txt
  entry, cargo features in librefang-channels + librefang-api
  (incl. membership in all-channels / all-channels-no-email)
  removed
- WebexConfig struct + Default impl + ChannelsConfig.webex field
  + matching Default entry removed; validation hook removed
- channel_bridge.rs: import, spawn block, find_channel_info! arm,
  check_channel! arm, default-empty test assertion removed
- routes/channels.rs: ChannelMeta entry + 4 match arms removed;
  SIDECAR_CATALOG webex entry added
- routes/config.rs ch!(webex) call removed
- kernel channel_sender for_each_channel_field! macro entry +
  EXPECTED name-list entry removed
- CLI: TUI ChannelDef row removed, migrated-to-sidecar comment
  grown
- README / AGENTS for librefang-channels updated to list webex
  among the migrated-to-sidecar adapters
- Docs (EN + ZH): configuration/page.mdx,
  configuration/channels/page.mdx,
  integrations/channels/enterprise/page.mdx,
  configuration/security/page.mdx, architecture/page.mdx —
  [channels.webex] blocks replaced with [[sidecar_channels]]
  redirect blocks (fence language `bash` per Shiki); architecture
  wave table moves webex out of Wave 3 in-process and into the
  sidecar list

New env-var knobs (read from [sidecar_channels.env]):
- WEBEX_ALLOWED_ROOMS (comma-separated room IDs, empty = allow all)
- WEBEX_ACCOUNT_ID (optional, multi-bot routing key)
Secret in ~/.librefang/secrets.env: WEBEX_BOT_TOKEN.

Operator action required: an existing [channels.webex] block is
no longer recognised — re-declare as a [[sidecar_channels]]
running python3 -m librefang.sidecar.adapters.webex with
WEBEX_BOT_TOKEN (in ~/.librefang/secrets.env) and any of the
optional knobs above (in [sidecar_channels.env]).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels
  -p librefang-kernel --lib: 420 + 838 + 1087 passed
- cargo test -p librefang-api --lib: 685 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && pytest tests/test_webex_adapter.py: 78 passed

* fix(channels/webex): apply review nits — display name, suppress flag, dead code, redundant filter

Self-review pass on top of the migration commit; addresses 4 nits from the
pre-merge review without changing the migration's core behaviour:

1. **personDisplayName drives user_name** (`webex.py:281-291, 326-330`). The
   migration commit preserved the Rust adapter's `webex.rs:431` behaviour of
   using `personEmail` unconditionally, which leaks emails into bot logs and
   dashboard UI. Now `parse_webex_message` prefers `personDisplayName` from the
   `/messages/<id>` body and falls back to `personEmail` when the field is
   absent (older Webex orgs / some service-account principals) or empty;
   `personEmail` stays in metadata for routing / audit. 3 new tests
   (`test_parse_prefers_person_display_name_for_user_name`,
   `…_falls_back_to_email_when_display_name_missing`,
   `…_falls_back_to_email_when_display_name_empty`).

2. **Explicit `suppress_error_responses = False`** on `WebexAdapter` with a
   precedent comment naming the discord / slack / twitch chat-room default vs.
   the mastodon / bluesky / reddit / nextcloud public-broadcast default. The
   migration left this implicit; an explicit class-level attr matches how
   `twitch.py:312` documents the choice so the next reader doesn't have to
   reconstruct the reasoning.

3. **`register_webhook` dead-code drop documented in CHANGELOG**. The Rust
   adapter carried a never-wired `register_webhook` helper at `webex.rs:137-168`
   (marked `#[allow(dead_code)]`) for an HTTP-webhook delivery alternative the
   channel-bridge never enabled; the sidecar drops it. The CHANGELOG now names
   this explicitly and points operators who want webhook delivery at the
   canonical generic `[[sidecar_channels]]` running
   `librefang.sidecar.adapters.webhook`.

4. **Skip redundant room filter inside `parse_webex_message`** when called
   from `_handle_envelope` (`webex.py:867-878`). `_handle_envelope` already
   filtered against `self.allowed_rooms` at line 855 before paying for the
   REST fetch; passing `allowed_rooms=[]` into the parser drops one redundant
   list-membership check per inbound message. The parser keeps the parameter
   for direct callers (tests; future code paths).

Verification (run inside the worktree):

* `cd sdk/python && python3 -m pytest tests/test_webex_adapter.py -q` →
  **81 passed** (was 78; +3 personDisplayName tests). No existing test
  modified.
* `cargo check` / `clippy` not re-run — this commit is Python + CHANGELOG only,
  no Rust touched.

Diff: +73 / -5 across `CHANGELOG.md`,
`sdk/python/librefang/sidecar/adapters/webex.py`,
`sdk/python/tests/test_webex_adapter.py`.
houko added a commit that referenced this pull request May 20, 2026
* feat(channels)!: migrate line from in-process adapter to sidecar

Replaces the 881-line in-process librefang-channels::line adapter
(axum webhook route at /channels/line/webhook on the shared API
server, X-Line-Signature HMAC-SHA256 verification, POST
/v2/bot/message/push for outbound) with the Python sidecar
librefang.sidecar.adapters.line. Same pattern as ntfy #5224,
telegram #5241, gotify #5263, mastodon #5264, bluesky #5277,
reddit #5281, twitch #5297, rocketchat #5298, discord #5299,
nextcloud #5301, slack #5302, webex #5309.

Behaviour parity:
- GET /v2/bot/info startup credential probe (bot display name)
- HTTP webhook server (BaseHTTPRequestHandler over
  ThreadingTCPServer) on LINE_WEBHOOK_PORT (default 9090) +
  LINE_WEBHOOK_PATH (default /webhook)
- X-Line-Signature HMAC-SHA256 verification over the raw wire
  bytes (constant-time compare, line.rs:229-250 parity, incl.
  the wire-bytes-vs-JSON-roundtrip regression)
- Only message events with type==text (other event types and
  non-text message types dropped, line.rs:256-273)
- Source-type -> reply_to mapping (user -> userId, group ->
  groupId, room -> roomId; group/room -> is_group=true,
  line.rs:280-290)
- Slash-command routing (/cmd args -> Command)
- Metadata preservation: user_id / reply_to / reply_token /
  source_type
- Multi-bot account_id injection via LINE_ACCOUNT_ID
- 5000-char chunking (LINE_MSG_LIMIT parity with MAX_MESSAGE_LEN
  at line.rs:39)
- Image-branch wire shape (originalContentUrl + previewImageUrl
  both set to the caller URL, caption sent as follow-up text
  push, line.rs:464-490)
- ChannelType::Custom("line") preserved as channel_type = "line"
  on the sidecar entry so existing routing /
  channel_role_mapping keys continue to resolve

Three improvements over the Rust adapter:
1. 429 Retry-After honoured on outbound. The Rust
   api_push_message (line.rs:148-184) had no 429 handling;
   throttled push returned Err and the chunked reply dropped on
   the floor. Sidecar parses Retry-After (default 30 s fallback,
   floor 1 s, cap MAX_BACKOFF_SECS), sleeps, retries once, then
   logs-and-continues on the second 429. Same shape as #5303.
2. Inbound dedupe on message.id. LINE redelivers webhook events
   when the operator's endpoint fails; the Rust handler at
   line.rs:413-427 emitted every event unconditionally, causing
   duplicate agent invocations on transient failures. Bounded
   local set on message.id with SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000 (matches reddit / rocketchat /
   nextcloud / webex policy).
3. Explicit HTTP timeouts on every urlopen call (urllib has no
   default timeout; a hung LINE API would otherwise hang the
   worker thread forever).

Cascade removal:
- src/line.rs deleted; lib.rs mod decl, channels-allowlist.txt
  entry, cargo features in librefang-channels + librefang-api
  (incl. membership in all-channels / all-channels-no-email)
  removed
- LineConfig struct + Default impl + ChannelsConfig.line field
  + matching Default entry removed; validation hook removed
- channel_bridge.rs: import, spawn block, find_channel_info!
  arm, check_channel! arm, default-empty test assertion removed
- routes/channels.rs: ChannelMeta entry + 5 match arms removed
  (is_some / serialize / len / ser / is_channel_configured);
  webhook_route_suffix allowlist entry removed (sidecar runs its
  own webhook listener, not on the shared API port);
  SIDECAR_CATALOG line entry added
- routes/config.rs ch!(line) call removed
- kernel channel_sender for_each_channel_field! entry + EXPECTED
  name-list entry removed
- CLI: TUI ChannelDef row removed, migrated-to-sidecar comment
  grown
- README / AGENTS for librefang-channels updated; the mandatory-
  HMAC paragraph now lists Teams + DingTalk only (LINE moves to
  the sidecar but keeps the same X-Line-Signature contract
  inside the new process)
- Docs (EN + ZH): configuration/page.mdx,
  configuration/channels/page.mdx,
  configuration/security/page.mdx,
  integrations/channels/integrations/page.mdx -
  [channels.line] blocks replaced with [[sidecar_channels]]
  redirect blocks; security env-var table flags both
  LINE_CHANNEL_SECRET and LINE_CHANNEL_ACCESS_TOKEN as
  sidecar-only; validation table drops the LINE row

New env-var knobs (read from [sidecar_channels.env]):
- LINE_WEBHOOK_PORT (default 9090)
- LINE_WEBHOOK_PATH (default /webhook)
- LINE_BIND_HOST (default 0.0.0.0)
- LINE_ACCOUNT_ID (optional, multi-bot routing key)
Secrets in ~/.librefang/secrets.env: LINE_CHANNEL_SECRET +
LINE_CHANNEL_ACCESS_TOKEN.

Operator action required (substantive): the sidecar runs its
OWN HTTP webhook server - it is no longer mounted on the
LibreFang API port - so the webhook URL registered at the LINE
Developers Console must be updated to point at the sidecar host
(typical pattern: an HTTPS reverse proxy in front of the
sidecar's LINE_WEBHOOK_PORT). An existing [channels.line] block
is no longer recognised - re-declare as a [[sidecar_channels]]
running python3 -m librefang.sidecar.adapters.line with
LINE_CHANNEL_SECRET + LINE_CHANNEL_ACCESS_TOKEN (in
~/.librefang/secrets.env) and any of the optional knobs above
(in [sidecar_channels.env]).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo fmt --check: clean
- cargo test --lib -p librefang-types -p librefang-channels
  -p librefang-kernel: 420 + 838 + 1087 passed
- cargo test --lib -p librefang-api: 685 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cargo xtask channel-policy: passed
- cd sdk/python && pytest tests/test_line_adapter.py:
  68 passed

* docs(channels): correct line sidecar HTTP server class references

The module docstring claimed `ThreadingHTTPServer` but the
implementation uses `socketserver.ThreadingTCPServer` (intentional:
skips `HTTPServer.server_bind()`'s `socket.getfqdn()` DNS lookup on
startup). Align the three doc references with the actual class and
spell out why we don't use `ThreadingHTTPServer` so the next reader
doesn't "fix" it back.
houko added a commit that referenced this pull request May 20, 2026
Replaces the 713-line in-process `librefang-channels::zulip` (Basic
auth + `POST /api/v1/register` + long-poll `GET /api/v1/events` +
`POST /api/v1/messages`) with an out-of-process Python sidecar at
`sdk/python/librefang/sidecar/adapters/zulip.py` (stdlib-only:
urllib + base64).

Behaviour parity:
- HTTP Basic auth `<bot_email>:<api_key>` on every REST call
- `GET /api/v1/users/me` startup probe → bot's stable `user_id` + `full_name`
- event-queue register with optional `narrow=[["stream", …], …]` when
  `ZULIP_STREAMS` is set
- long-poll `/events?dont_block=false` with 70 s HTTP timeout (Rust used
  POLL_TIMEOUT + 10 s at zulip.rs:244)
- `BAD_EVENT_QUEUE_ID` re-register (zulip.rs:262-308 parity)
- client-side stream filter as defence-in-depth against best-effort narrow
- slash-command routing `/cmd args` → Command
- DM vs group via `message.type == "stream"`
- 10 000-char chunking (`ZULIP_MSG_LIMIT` parity)
- outbound DM heuristic: `@` in platform_id ⇒ type=direct
- `account_id` metadata injection
- exponential backoff 1 s → 60 s

Four improvements (each with file/line evidence on the migrating
tree):

1. **outbound topic round-trip via `thread_id`** — the Rust `send`
   (zulip.rs:463) hard-coded `topic = "LibreFang"` for every stream
   reply, losing the inbound topic context so the bot's response
   always landed in a "LibreFang" topic regardless of what triggered
   it. The sidecar surfaces inbound `message.subject` as `thread_id`
   and `on_send` routes every stream send through that topic, so the
   reply lands in the originating topic.

2. **429 `Retry-After` honoured on every REST path** — the Rust
   adapter had no 429 handling, only generic 1 s → 60 s exponential
   backoff (zulip.rs:228-313). `_validate` / `_register_queue` /
   `_poll_once` / `_post_message` all parse `Retry-After` (1 s floor,
   `MAX_BACKOFF_SECS` cap, 30 s fallback) and retry once.

3. **bounded `message.id` dedupe** — `last_event_id` narrows the
   event range server-side, but `BAD_EVENT_QUEUE_ID` re-register
   starts fresh and can re-emit. Rust emit at zulip.rs:434 was
   unconditional. Sidecar dedupes on `message.id` with bounded
   `SEEN_MESSAGES_MAX = 10 000` / `EVICT = 5 000` cap (same policy
   as reddit / rocketchat / nextcloud / webex).

4. **self-skip by stable integer `sender_id`** — Rust compared
   `sender_email == bot_email` (zulip.rs:357); email rotation
   breaks self-skip. Sidecar prefers `sender_id == own_user_id`
   (the integer `/users/me` returns) and falls back to email only
   when sender_id is absent (parallels rocketchat #5298 /
   nextcloud #5301).

Cascade:
- `src/zulip.rs` deleted; `lib.rs` mod, `channels-allowlist.txt`
  entry, both Cargo.toml features (incl. `all-channels` /
  `all-channels-no-email`) removed
- `ZulipConfig` struct + Default + `ChannelsConfig.zulip` field +
  default + validation hook + `test_zulip_config_defaults` /
  `…_serde` unit tests all gone
- `channel_bridge.rs` import / spawn block / `find_channel_info!` /
  `check_channel!` / default-empty assertion removed
- `routes/channels.rs` `ChannelMeta` + 4 match arms removed;
  `SIDECAR_CATALOG` `zulip` entry added at the end
- `routes/config.rs` `ch!(zulip)` removed
- `channel_sender.rs` `for_each_channel_field!` + `EXPECTED` entries
  removed
- CLI TUI `ChannelDef` removed
- README + AGENTS "migrated to sidecars" sentence updated
- docs EN + ZH: `[channels.zulip]` blocks replaced with
  `[[sidecar_channels]]` redirect; `ZULIP_API_KEY` env table marked
  (sidecar); validation table entry removed

New env-var knobs (read from `[sidecar_channels.env]`):
`ZULIP_SERVER_URL`, `ZULIP_BOT_EMAIL`, optional `ZULIP_STREAMS`,
optional `ZULIP_ACCOUNT_ID`. Bot API key in
`~/.librefang/secrets.env` as `ZULIP_API_KEY`.

`ChannelType::Custom("zulip")` (the channel-type token the Rust
adapter advertised at zulip.rs:197) is preserved across this
migration via `channel_type = "zulip"` on the sidecar entry, so
existing routing and `channel_role_mapping` keys that reference
`zulip` continue to resolve.

Operator action required: an existing `[channels.zulip]` block is
no longer recognised — re-declare as `[[sidecar_channels]]` running
`python3 -m librefang.sidecar.adapters.zulip`.

Verification (run inside the worktree):
- `cargo check --workspace --lib` clean
- `cargo clippy --workspace --all-targets -- -D warnings` clean
- `cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib`:
  837 passed
- `cargo test -p librefang-api --lib`: 685 passed
- `cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test`:
  38 + 2 passed
- `cd sdk/python && python3 -m pytest tests/test_zulip_adapter.py -q`: **74 passed**
- `grep -rn -iE "channels\\.zulip|channel-zulip|ZulipAdapter|ZulipConfig" docs/ crates/`:
  only the intentional sidecar-redirect notes remain in docs (and
  the sidecar-redirect comment in `types.rs`)

`kernel_config_schema.golden.json` is `#[ignore]`'d on main per
its own header (#4084 follow-up); leave the fixture as-is, matches
twitch / webex precedent.
houko added a commit that referenced this pull request May 20, 2026
)

Replaces the 713-line in-process `librefang-channels::zulip` (Basic
auth + `POST /api/v1/register` + long-poll `GET /api/v1/events` +
`POST /api/v1/messages`) with an out-of-process Python sidecar at
`sdk/python/librefang/sidecar/adapters/zulip.py` (stdlib-only:
urllib + base64).

Behaviour parity:
- HTTP Basic auth `<bot_email>:<api_key>` on every REST call
- `GET /api/v1/users/me` startup probe → bot's stable `user_id` + `full_name`
- event-queue register with optional `narrow=[["stream", …], …]` when
  `ZULIP_STREAMS` is set
- long-poll `/events?dont_block=false` with 70 s HTTP timeout (Rust used
  POLL_TIMEOUT + 10 s at zulip.rs:244)
- `BAD_EVENT_QUEUE_ID` re-register (zulip.rs:262-308 parity)
- client-side stream filter as defence-in-depth against best-effort narrow
- slash-command routing `/cmd args` → Command
- DM vs group via `message.type == "stream"`
- 10 000-char chunking (`ZULIP_MSG_LIMIT` parity)
- outbound DM heuristic: `@` in platform_id ⇒ type=direct
- `account_id` metadata injection
- exponential backoff 1 s → 60 s

Four improvements (each with file/line evidence on the migrating
tree):

1. **outbound topic round-trip via `thread_id`** — the Rust `send`
   (zulip.rs:463) hard-coded `topic = "LibreFang"` for every stream
   reply, losing the inbound topic context so the bot's response
   always landed in a "LibreFang" topic regardless of what triggered
   it. The sidecar surfaces inbound `message.subject` as `thread_id`
   and `on_send` routes every stream send through that topic, so the
   reply lands in the originating topic.

2. **429 `Retry-After` honoured on every REST path** — the Rust
   adapter had no 429 handling, only generic 1 s → 60 s exponential
   backoff (zulip.rs:228-313). `_validate` / `_register_queue` /
   `_poll_once` / `_post_message` all parse `Retry-After` (1 s floor,
   `MAX_BACKOFF_SECS` cap, 30 s fallback) and retry once.

3. **bounded `message.id` dedupe** — `last_event_id` narrows the
   event range server-side, but `BAD_EVENT_QUEUE_ID` re-register
   starts fresh and can re-emit. Rust emit at zulip.rs:434 was
   unconditional. Sidecar dedupes on `message.id` with bounded
   `SEEN_MESSAGES_MAX = 10 000` / `EVICT = 5 000` cap (same policy
   as reddit / rocketchat / nextcloud / webex).

4. **self-skip by stable integer `sender_id`** — Rust compared
   `sender_email == bot_email` (zulip.rs:357); email rotation
   breaks self-skip. Sidecar prefers `sender_id == own_user_id`
   (the integer `/users/me` returns) and falls back to email only
   when sender_id is absent (parallels rocketchat #5298 /
   nextcloud #5301).

Cascade:
- `src/zulip.rs` deleted; `lib.rs` mod, `channels-allowlist.txt`
  entry, both Cargo.toml features (incl. `all-channels` /
  `all-channels-no-email`) removed
- `ZulipConfig` struct + Default + `ChannelsConfig.zulip` field +
  default + validation hook + `test_zulip_config_defaults` /
  `…_serde` unit tests all gone
- `channel_bridge.rs` import / spawn block / `find_channel_info!` /
  `check_channel!` / default-empty assertion removed
- `routes/channels.rs` `ChannelMeta` + 4 match arms removed;
  `SIDECAR_CATALOG` `zulip` entry added at the end
- `routes/config.rs` `ch!(zulip)` removed
- `channel_sender.rs` `for_each_channel_field!` + `EXPECTED` entries
  removed
- CLI TUI `ChannelDef` removed
- README + AGENTS "migrated to sidecars" sentence updated
- docs EN + ZH: `[channels.zulip]` blocks replaced with
  `[[sidecar_channels]]` redirect; `ZULIP_API_KEY` env table marked
  (sidecar); validation table entry removed

New env-var knobs (read from `[sidecar_channels.env]`):
`ZULIP_SERVER_URL`, `ZULIP_BOT_EMAIL`, optional `ZULIP_STREAMS`,
optional `ZULIP_ACCOUNT_ID`. Bot API key in
`~/.librefang/secrets.env` as `ZULIP_API_KEY`.

`ChannelType::Custom("zulip")` (the channel-type token the Rust
adapter advertised at zulip.rs:197) is preserved across this
migration via `channel_type = "zulip"` on the sidecar entry, so
existing routing and `channel_role_mapping` keys that reference
`zulip` continue to resolve.

Operator action required: an existing `[channels.zulip]` block is
no longer recognised — re-declare as `[[sidecar_channels]]` running
`python3 -m librefang.sidecar.adapters.zulip`.

Verification (run inside the worktree):
- `cargo check --workspace --lib` clean
- `cargo clippy --workspace --all-targets -- -D warnings` clean
- `cargo test -p librefang-types -p librefang-channels -p librefang-kernel --lib`:
  837 passed
- `cargo test -p librefang-api --lib`: 685 passed
- `cargo test -p librefang-api --test channels_routes_test --test sidecar_describe_test`:
  38 + 2 passed
- `cd sdk/python && python3 -m pytest tests/test_zulip_adapter.py -q`: **74 passed**
- `grep -rn -iE "channels\\.zulip|channel-zulip|ZulipAdapter|ZulipConfig" docs/ crates/`:
  only the intentional sidecar-redirect notes remain in docs (and
  the sidecar-redirect comment in `types.rs`)

`kernel_config_schema.golden.json` is `#[ignore]`'d on main per
its own header (#4084 follow-up); leave the fixture as-is, matches
twitch / webex precedent.
houko added a commit that referenced this pull request May 20, 2026
Replaces the 758-line in-process `librefang-channels::qq`
adapter (token mint + WebSocket gateway + HELLO/IDENTIFY/READY
handshake + heartbeat + DISPATCH for 4 event types + REST send
with markdown stripping) with an out-of-process Python sidecar
at `sdk/python/librefang/sidecar/adapters/qq.py`. Same pattern as
ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky
#5277, reddit #5281, twitch #5297, rocketchat #5298, discord
#5299, nextcloud #5301, slack #5302, webex #5309, line #5312,
zulip #5310, mattermost #5315, signal #5317.

Behaviour parity (every claim cited against the pre-migration
tree):

- Token fetch: POST bots.qq.com/app/getAppAccessToken (qq.rs:542-557).
- Gateway: GET /gateway with Bearer auth (qq.rs:559-573).
- WS handshake: HELLO(op=10) → IDENTIFY(op=2) with token =
  "QQBot <access_token>", intents bitmask, shard=[0,1] → READY
  dispatch (qq.rs:413-433).
- Heartbeat: op=1 frame at the server-supplied interval, carrying
  last seen `s` from dispatch frames (qq.rs:359-368).
- 4 dispatch event types: MESSAGE_CREATE / AT_MESSAGE_CREATE →
  /channels/{channel_id}/messages, DIRECT_MESSAGE_CREATE →
  /dms/{guild_id}/messages, GROUP_AT_MESSAGE_CREATE →
  /v2/groups/{group_openid}/messages, C2C_MESSAGE_CREATE →
  /v2/users/{user_openid}/messages (qq.rs:194-224).
- Bot-mention `/` strip + slash-command routing (qq.rs:227).
- User allowlist + multi-bot account_id metadata (#5003,
  qq.rs:400-405).
- Outbound markdown stripping pipeline preserves order:
  think tags → code blocks → inline code → bold → italic →
  headings → table separators → links → blockquotes → HRs →
  triple-newline collapse (qq.rs:137-180).
- 2000-char chunking (qq.rs:26), 1–60 s exponential reconnect
  backoff (qq.rs:282).

Improvements over the Rust adapter
==================================

1. **Reply context actually round-trips.** The Rust
   `parse_dispatch_event` (qq.rs:182-246) computed
   `reply_endpoint` and `msg_id` but the dispatch loop bound them
   to `_endpoint` / `_msg_id` (qq.rs:399) and dropped them on the
   floor; `send` (qq.rs:497-498) then expected
   `user.platform_id` to be encoded as `"<endpoint>|<msg_id>"`
   and silently no-op'd when the delimiter wasn't there. The
   Rust adapter therefore failed every real outbound — only the
   synthetic wiremock tests at qq.rs:686-712 exercised the
   working shape. The sidecar surfaces the reply endpoint as
   `channel_id` and the QQ `msg_id` as `thread_id` on the
   inbound event so the bridge round-trips them through to
   `on_send`, which posts to `{api_base}{channel_id}` with the
   correct passive-reply `msg_id`.

2. **Inbound dedupe on `msg.id`.** Rust emitted every parsed
   event unconditionally (qq.rs:399-410); WS reconnect could
   replay. Bounded local set, SEEN_MAX=10000 / EVICT=5000.

3. **429 Retry-After honoured on every REST path.** Token,
   gateway, and outbound send all sleep + retry once; second
   429 logs-and-continues (matches webex / line / mattermost /
   signal #5303).

4. **Explicit 15s urlopen timeouts on every REST call.** Rust
   pre-configured reqwest's 30s default at qq.rs:71; sidecar
   passes `timeout=SEND_TIMEOUT_SECS` (15 s) so a misbehaving
   endpoint trips an explicit error.

Cascade removal
================

- `src/qq.rs` deleted; `lib.rs` mod, `channels-allowlist.txt`
  entry, both `Cargo.toml` features removed.
- `QqConfig` struct + Default + `ChannelsConfig.qq` field gone.
- `channel_bridge.rs` import / spawn block / `check_channel!`
  / find_channel_info!(qq) (none existed) removed.
- `routes/channels.rs` `ChannelMeta` entry + 4 match arms
  (`is_some` / serialize / `len` / `ser`) removed; SIDECAR_CATALOG
  `qq` entry added.
- `routes/config.rs` `ch!(qq)` removed.
- `channel_sender.rs` `for_each_channel_field!` macro entry +
  `EXPECTED` name-list entry removed.
- CLI / TUI had no QQ wizard arm to remove.
- librefang-migrate had no QQ block to handle (QQ wasn't in the
  OpenClaw schema).
- Docs EN + ZH (6 files): `[channels.qq]` blocks replaced with
  `[[sidecar_channels]]` redirects; integrations tutorial
  rewritten for the sidecar.

New env-var knobs (`[sidecar_channels.env]`)
=============================================

- `QQ_APP_ID` (required)
- `QQ_ALLOWED_USERS` (optional, CSV)
- `QQ_ACCOUNT_ID` (optional)
- `QQ_INTENTS` (optional, decimal/hex bitmask override)

Secret (`~/.librefang/secrets.env`)
====================================

- `QQ_APP_SECRET` (required)

Operator action required
========================

An existing `[channels.qq]` block is no longer recognised —
re-declare as `[[sidecar_channels]]` running
`python3 -m librefang.sidecar.adapters.qq`. See
`sdk/python/librefang/sidecar/adapters/qq.py` header for the
exact config. `ChannelType::Custom("qq")` is preserved via
`channel_type = "qq"` on the sidecar entry so existing routing
and `channel_role_mapping` keys continue to resolve.

Verification
============

- `cargo check -p librefang-types -p librefang-channels -p librefang-api -p librefang-kernel -p librefang-cli --lib` — clean
- `cargo xtask channel-policy` — passed (`qq` removed from allowlist)
- `pytest sdk/python/tests/test_qq_adapter.py -q` — **77 passed**
  (covers env parsing, markdown stripping, parse_qq_event for all
  4 event types + edge cases, _mark_seen LRU eviction,
  _fetch_token + _fetch_gateway happy path / 429 / non-200 /
  missing field, _post_message chunking + 429 retry-once +
  persistent-429 fail-open + 5xx fail-open, on_send wiring with
  markdown stripping at the outbound boundary, WS gateway flow
  via mock _WebSocketClient — HELLO → IDENTIFY token+intents+
  shard, DISPATCH emission, msg.id dedupe, RECONNECT/INVALID_SESSION,
  heartbeat after interval, --describe schema, capabilities
  contract).

Co-authored-by: Evan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/channels Messaging channel adapters area/docs Documentation and guides area/kernel Core kernel (scheduling, RBAC, workflows) area/sdk JavaScript and Python SDKs size/XL 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant