Skip to content

feat(channels)!: migrate ntfy from in-process adapter to sidecar (P7)#5224

Merged
houko merged 3 commits into
mainfrom
feat/sidecar-poc-ntfy
May 18, 2026
Merged

feat(channels)!: migrate ntfy from in-process adapter to sidecar (P7)#5224
houko merged 3 commits into
mainfrom
feat/sidecar-poc-ntfy

Conversation

@houko

@houko houko commented May 18, 2026

Copy link
Copy Markdown
Contributor

Sidecar-first POC — the keystone of the #5219/#5220/#5221 series:
proves the full pipeline by replacing a real in-process channel with
an out-of-process sidecar and deleting the Rust adapter.

⚠️ Breaking

[channels.ntfy] is no longer a recognised config block. Operators
re-declare it as a [[sidecar_channels]] running the new
examples/sidecar-channel-python/ntfy_adapter.py (its header has the
exact config). The separate ntfy push-notification provider
(push_provider = "ntfy" / ntfy_url / ntfy_topic, used by device
pairing) is a different feature and is deliberately untouched.

Replacement (behaviour-preserving)

examples/sidecar-channel-python/ntfy_adapter.py — stdlib-only, on the
librefang.sidecar SDK (P4). Same SSE parsing, /command handling,
title→sender, topic metadata, 4096-char chunked plain-text
publish, optional Bearer auth, exponential-backoff reconnect.

Scope expanded vs. the phase plan (approved)

The plan listed ~6 deletion sites; the true blast radius of removing
the NtfyConfig type is much wider and kernel-touching. The full-
removal path (breaking) was explicitly chosen. Removed:
librefang-channels (src/ntfy.rs, lib.rs, Cargo channel-ntfy,
allowlist entry), librefang-types (NtfyConfig + channels.ntfy +
validation hook), librefang-api (channel-ntfy feature incl.
core/all-channels membership; channel_bridge use+block+macros+test;
routes/channels.rs descriptor + 4 arms; routes/config.rs ch!),
librefang-kernel (channel_sender registry), librefang-cli (TUI
def + comment), kernel-config golden regenerated. Removing ntfy
from channels-allowlist.txt makes the P5 gate
(cargo xtask channel-policy) permanently reject any in-process ntfy
reintroduction — the migration ratchet working as designed.

Verification

  • cargo check --workspace --lib: clean
  • cargo clippy --workspace --all-targets -- -D warnings: clean
  • cargo test -p librefang-channels (847) / -p librefang-types /
    -p xtask (channel-policy gate green with ntfy delisted
    real_repo_tree_is_green ok): pass
  • cd sdk/python && pytest: 22 passed incl.
    tests/test_ntfy_adapter.py (SSE parse, command vs text, sender,
    chunked publish + auth, unsupported-content fallback; urllib mocked)
  • cargo test -p librefang-api: all pass except 2 pre-existing,
    unrelated failures
    auth_public_allowlist.rs's
    dashboard_read_paths_are_in_catalog /
    dashboard_read_routes_reachable_without_auth_when_no_key_configured
    fail on /api/cron/list (a cron-route allowlist drift already
    on main). Evidence it is not from this PR: the offending path is a
    cron route; this PR touches no auth middleware, route registration,
    PUBLIC_ROUTES_*, or cron code. Left for a separate, cron/auth-
    domain fix rather than bundled here.

Sidecar-first POC (follows the #5219/#5220/#5221 series): proves the
end-to-end pipeline by replacing a real in-process channel with a
sidecar and removing the Rust adapter.

BREAKING: the `[channels.ntfy]` config block is no longer recognised.
Re-declare it as a `[[sidecar_channels]]` running the new
examples/sidecar-channel-python/ntfy_adapter.py (header documents the
exact config). The separate ntfy *push-notification provider*
(`push_provider="ntfy"` / `ntfy_url` / `ntfy_topic`, device pairing)
is a different feature and is deliberately left intact.

Replacement (behaviour-preserving), stdlib-only on the librefang.sidecar
SDK: examples/sidecar-channel-python/ntfy_adapter.py — same SSE parse,
`/command` handling, title→sender, topic metadata, 4096-char chunked
plain-text publish, optional Bearer auth, exponential-backoff reconnect.

Removed in-process ntfy (full cascade — wider than the phase plan
estimated; user approved the breaking full-removal path):
- librefang-channels: src/ntfy.rs (deleted), lib.rs mod, Cargo.toml
  `channel-ntfy` feature, channels-allowlist.txt entry (so the P5
  `cargo xtask channel-policy` gate now permanently blocks any
  in-process ntfy reintroduction).
- librefang-types: NtfyConfig struct + Default, `channels.ntfy`
  field + default, validation hook.
- librefang-api: `channel-ntfy` feature + its membership in
  core-channels/all-channels/all-channels-no-email; channel_bridge
  use+block+macros+test assertion; routes/channels.rs ChannelMeta
  descriptor + 4 match arms; routes/config.rs `ch!(ntfy)`.
- librefang-kernel: channel_sender registry macro + name list.
- librefang-cli: TUI ChannelDef + Cargo.toml core-channels comment.
- kernel-config golden regenerated (NtfyConfig removed; 0 residual).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p librefang-channels (847) / -p librefang-types /
  -p xtask (channel-policy gate green WITH ntfy delisted —
  real_repo_tree_is_green ok) all pass
- cargo test -p librefang-api: all pass EXCEPT 2 pre-existing
  failures unrelated to this PR — auth_public_allowlist.rs's
  `dashboard_read_paths_are_in_catalog` /
  `dashboard_read_routes_reachable_without_auth_when_no_key_configured`
  fail on `/api/cron/list` (a cron route allowlist drift on main;
  this PR touches no auth middleware / route registration / cron
  code — evidence: the offending path is cron, not ntfy/channels)
- cd sdk/python && pytest: 22 passed incl. tests/test_ntfy_adapter.py
  (SSE parse / command vs text / sender / chunked publish + auth /
  unsupported-content fallback, urllib mocked)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 04c1224455

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +121 to +122
is_group=True,
metadata={"topic": topic or self.topic},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve ntfy account_id on inbound messages

When NTFY_ACCOUNT_ID is set, the sidecar announces it in ready but never copies it into each message's metadata. The bridge resolves account-scoped bindings and channel overrides from message.metadata["account_id"] (for example in resolve_or_fallback), and the removed Rust adapter explicitly injected that metadata, so multi-topic/multi-bot ntfy setups will route inbound messages as an unqualified ntfy channel instead of the configured account-specific binding. Include account_id in the emitted metadata when self.account_id is present.

Useful? React with 👍 / 👎.

@github-actions github-actions Bot added area/docs Documentation and guides area/channels Messaging channel adapters area/kernel Core kernel (scheduling, RBAC, workflows) area/sdk JavaScript and Python SDKs labels May 18, 2026
houko added a commit that referenced this pull request May 18, 2026
Documents the sidecar-first model realized by the #5219/#5220/#5221
(+#5224) series.

- New docs/architecture/sidecar-channels.md (house style): process
  model + ASCII supervisor state machine, the JSON-RPC protocol
  tables, capability negotiation, backoff/circuit-break/backpressure,
  the daemon-restart-vs-platform-reconnect responsibility split, the
  allowlist policy gate, ntfy as the worked migration, and the
  long-tail backlog policy (opportunistic, not a batch rewrite —
  informed by P7 revealing the per-channel removal blast radius).
- CONTRIBUTING.md "How to Add a New Channel Adapter" rewritten:
  sidecar-first is the path (SDK subclass + [[sidecar_channels]] +
  template/ntfy reference); the in-process Rust recipe is replaced by
  a "maintainers only — allowlist + reviewed exception" note. Also
  fixes the pre-existing inaccurate trait sketch in that section
  (it showed `start(&mut self)->Result<()>` /
  `send(&self,&str,&str)` / `stop(&mut self)`, none of which match
  the real `ChannelAdapter` signatures) by removing the stale sample
  rather than maintaining a wrong one.
- crates/librefang-channels/AGENTS.md (the CLAUDE.md symlink target)
  "Adding a new channel": sidecar-first pointer + the
  pre-commit / `cargo xtask channel-policy` allowlist gate; the
  grandfathered-adapter rules (wiremock send test, webhook wiring,
  off-by-default feature) are kept.

Docs-only; no code. Cross-references verified against `main`
(SDK template, channels-allowlist.txt, pre-commit, xtask
channel-policy all present). One forward reference:
examples/sidecar-channel-python/ntfy_adapter.py arrives in #5224 —
the architecture doc PR-attributes it to #5224, so it reads as
series documentation; this PR should land with or after #5224 so the
CONTRIBUTING pointer resolves.

@houko houko left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review (author): blocking-style review, recorded as a comment because GitHub does not allow request-changes on own PR. Architecture and migration story are excellent — this is a clean P7 landing on top of the merged P0–P5 chain (#5219/#5220/#5221), the CHANGELOG entry is thorough and explicitly flags the breaking change, the operator migration path is concrete (re-declare as [[sidecar_channels]] running ntfy_adapter.py), the P5 channel-policy gate is engaged (ntfy delisted from channels-allowlist.txt), the kernel-config golden is regenerated with zero NtfyConfig residue, and the new Python sidecar adapter is behaviour-preserving with solid test coverage (SSE parse, command vs text, sender, chunked publish, auth, unsupported-content fallback). The deliberate preservation of the separate push_provider = "ntfy" push-notification feature is correctly called out and verified — the only remaining ntfy references in librefang-types::config::types.rs are those (intentional) push-provider doc comments. Two items to address before merge, both small.

1. Stale documentation pointing at removed code. After this PR, three files still reference the deleted channel-ntfy cargo feature and the deleted crates/librefang-channels/src/ntfy.rs:

  • crates/librefang-channels/AGENTS.md:16 — lists channel-ntfy as a per-adapter feature flag. The flag no longer exists.
  • crates/librefang-channels/README.md:20 — same list, same issue.
  • examples/custom-channel/README.md:275 — cites ntfy.rs as a worked reference (SSE subscription + plain POST publishing) for contributors authoring a new channel. That file is gone in this PR, so the cross-reference will be broken on main.

These three lines should be removed or rewritten (e.g. the custom-channel README row can either be dropped or repointed at examples/sidecar-channel-python/ntfy_adapter.py as the sidecar reference, which is consistent with P6 / #5225's doc direction). The web/public/registry.json ntfy entry is not in scope here — that file is fetched from the separate librefang-registry repo by web/scripts/fetch-registry.ts and needs a follow-up PR there.

2. CI baseline disclosure is incomplete and the run is currently red. The PR body discloses two pre-existing auth_public_allowlist failures on /api/cron/list and attributes them to an unrelated cron-route drift on main. That is correct, and they are fixed on main by #5217 (fix(api,dashboard): expose 'background' section + drop stale '/api/cron/list' allowlist row), which landed after this PR was opened. However the current Coverage run also shows a third failure not mentioned in the body — librefang-api::config_schema_overlay::every_kernel_config_struct_field_is_exposed_via_overlay, panicking on KernelConfig fields with no x-sections descriptor and no entry in EXCLUDED: background. That failure is also pre-existing (the background field was added by #5063 on main, the EXCLUDED list was never updated, and #5217 is the same commit that fixes it). So all three failures clear with a rebase, but please rebase on current main and update the PR body's verification section to list the third failure too, so the CI green/red state matches the disclosure exactly — important for a breaking change PR where reviewers rely on the body to triage red checks.

Nothing structural. Once the three doc lines are cleaned up and the branch is rebased to clear CI, this is ready for squash-merge.

@houko
houko merged commit 5fa3678 into main May 18, 2026
11 of 14 checks passed
@houko
houko deleted the feat/sidecar-poc-ntfy branch May 18, 2026 09:58
houko added a commit that referenced this pull request May 18, 2026
…nto librefang-sdk (#5228)

The ntfy / telegram / webhook adapters were sitting under
examples/sidecar-channel-python/ alongside the generic template, but
they are first-party production code: ntfy in particular is the only
ntfy adapter that exists after #5224 deleted the in-process Rust one.
"Example" misclassifies them and forces operators to hand-copy a
script into their tree.

Move them into the SDK package so pip install librefang-sdk ships
them and `-m librefang.sidecar.adapters.<name>` runs them from
anywhere:

  examples/sidecar-channel-python/{ntfy,telegram,webhook}_adapter.py
  -> sdk/python/librefang/sidecar/adapters/{ntfy,telegram,webhook}.py

Adapter logic is unchanged. Header config examples updated from
file-path args to `-m` module args. Generic template adapter.py
stays in examples/ — that one is genuinely a starting point to copy.

* test_ntfy_adapter: drop sys.path hack, import via the package.
* kernel_config_schema golden: regenerated (only the SidecarChannelConfig
  doc-comment example string changed).
* CHANGELOG [Unreleased]: new entry documenting the operator-visible
  args change (`examples/...py` -> `-m librefang.sidecar.adapters.ntfy`)
  and updates to the still-unreleased #5224 entry to reference the new
  path.
* docs/architecture, CONTRIBUTING, channels README/AGENTS,
  channels-allowlist.txt header, channels.rs comment, pre-commit hook
  error text: all reference the new SDK path.

Verification: cargo check on touched crates clean; the 5 ntfy adapter
tests pass at the new import path; `from librefang.sidecar.adapters
import ntfy, webhook` smoke import succeeds.
@github-actions github-actions Bot added the size/XL 1000+ lines changed label May 18, 2026
houko added a commit that referenced this pull request May 18, 2026
#5241)

* feat(channels)!: remove in-process telegram adapter (now sidecar-only)

BREAKING: the `[channels.telegram]` config block is no longer
recognised. Telegram now runs as a `[[sidecar_channels]]` adapter
(`python3 -m librefang.sidecar.adapters.telegram`, channel_type =
"telegram"); the Python adapter reached full parity in the preceding
commits of this PR. Same pattern as the ntfy #5224 migration, scaled
to telegram's deeper integration.

Removed in-process telegram (full cascade):
- librefang-channels: src/telegram.rs (deleted, 6231 lines), lib.rs
  mod, Cargo.toml `channel-telegram` feature + all-channels,
  channels-allowlist.txt entry (so `cargo xtask channel-policy`
  permanently blocks any in-process telegram reintroduction),
  commands/mod.rs stale doc ref.
- librefang-types: `TelegramConfig` struct + Default + impl, the
  `channels.telegram` field + default, validation hook, the orphaned
  `default_telegram_long_poll_timeout_secs`. The shared
  deny_unknown_fields / proxy doc anchor moved from `TelegramConfig`
  to `DiscordConfig` (it was the canonical cross-reference for ~7
  other channel structs).
- librefang-api: `channel-telegram` feature + membership in
  core-channels / all-channels / all-channels-no-email / mini;
  channel_bridge use + Duration cfg + coalesce-window arm +
  check_channel + spawn block + default-empty test; routes/channels.rs
  ChannelMeta + the 5 match arms; routes/config.rs `ch!(telegram)`.
- librefang-kernel: channel_sender `for_each_channel_field!` macro +
  name list; config_reload hot-reload test.
- librefang-cli: TUI ChannelDef, interactive `channel setup` wizard
  arm + picker entry + health list, Cargo.toml/help-text examples.
- OpenClaw / openfang migrators: telegram is no longer emitted as
  `[channels.telegram]` (the kernel would reject it); the bot token
  is still migrated to secrets.env and the channel is reported as
  skipped with guidance to declare a `[[sidecar_channels]]` block.
- `ChannelType::Telegram`, the shared bot-command registry, the
  bridge channel-type->string map and `[channel_role_mapping.telegram]`
  RBAC mapping are deliberately kept (shared core / different feature);
  the sidecar telegram still flows through them.
- Tests that used telegram as the arbitrary example channel migrated
  to discord/slack; kernel-config golden regenerated (no residual
  TelegramConfig).

Verification: cargo check --workspace --lib clean; cargo xtask
channel-policy Passed (telegram delisted, allowlist only shrinks);
cargo test -p librefang-types / -p librefang-channels /
-p librefang-migrate all pass; -p librefang-api routes::channels +
channels_routes_test + config_schema_golden pass; clippy (touched
crates, --lib) clean. Two failures verified unrelated (zero telegram
coupling, files unmodified by this branch, paths orthogonal to the
cascade): hooks_agent_dispatches_to_default_agent e2e and the
kernel/tests.rs:8972 compact_agent_session_with_id arity drift —
same class #5224 documented as pre-existing.

* docs: redirect [channels.telegram] examples to sidecar pattern (#5241)

Replace every copy-pasteable [channels.telegram] config block in EN and ZH
docs with the equivalent [[sidecar_channels]] snippet. Illustrative overrides
examples that used [channels.telegram.overrides] as a placeholder are updated
to [channels.discord.overrides]. Config templates (librefang.toml.example,
init_default_config.toml) are updated to match.

* chore(cli,docs): drop in-process telegram from help-text and per-crate docs (#5241)

Remove "telegram" from CLI help suggestions (channel setup doc-comment,
channel test long_about examples), Fluent locale channel-unknown-fix lists
(EN and zh-CN), and the orphan section-setup-telegram key. Update
librefang-channels AGENTS.md and README.md to reflect that telegram joined
ntfy as a sidecar-only adapter.

* docs(rustdoc): re-anchor proxy contract from TelegramAdapter to DiscordConfig (#5241)

Three with_proxy() doc-comments in discord.rs, slack.rs, and mattermost.rs
cross-referenced the deleted TelegramAdapter::with_proxy for the URL contract.
Re-anchor to DiscordConfig::proxy, which is the canonical proxy doc since the
telegram removal. Also update two bridge_integration_test.rs doc-comments that
named TelegramAdapter as an example. Update ChannelsConfig field docstring in
types.rs to remove "Telegram, etc." and sync the golden schema fixture.

* chore(changelog): note in-process telegram removal as BREAKING (#5241)

Add [Unreleased] BREAKING entry for the telegram adapter migration: explains
that [channels.telegram] is no longer accepted under deny_unknown_fields,
what operators must do (re-declare as [[sidecar_channels]]), and what is
preserved (TELEGRAM_BOT_TOKEN, channel_type=telegram, RBAC mapping).

* chore(codegen): auto-regenerate openapi.json + sdk + schema baselines [skip ci]

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
houko added a commit that referenced this pull request May 19, 2026
…d (no 404 actions) (#5249)

* fix(api): surface configured sidecar channels on /api/channels

Regression from the sidecar migration: when telegram (#5241) and ntfy
(#5224) moved out-of-process, their ChannelMeta entries were removed
from CHANNEL_REGISTRY — which silently dropped them from the dashboard
channels page (/dashboard/channels reads /api/channels, which only
enumerated CHANNEL_REGISTRY). An operator who had telegram/ntfy
working saw them vanish with no indication of where they went; the
operator experience was no longer consistent across in-process vs
sidecar adapters.

Fix (backend-only, dashboard renders it unchanged): /api/channels and
channels_snapshot now also synthesize a row per configured
[[sidecar_channels]] entry — name/display, configured=true,
category/setup_type="sidecar", instance_count, no editable `fields`
(config.toml-managed), and setup_steps pointing the operator at
Config -> Sidecar Channels. The ChannelsPage already conditionally
hides empty fields/setup_steps, so these render as normal
configured/online cards with zero frontend changes. configured_count
counts them. A sidecar entry never shadows or duplicates an
in-process CHANNEL_REGISTRY row of the same name.

Tests (crates/librefang-api/tests/channels_routes_test.rs):
- channels_list_includes_configured_sidecar_channels: a configured
  [[sidecar_channels]] telegram appears as a configured sidecar row,
  exactly once, counted in configured_count, with no editable fields.
- channels_list_without_sidecar_has_no_sidecar_rows: negative control
  — no synthetic rows leak when none are configured.

Verification: cargo check -p librefang-api --lib clean; cargo test -p
librefang-api --test channels_routes_test 30 passed; cargo clippy -p
librefang-api --lib --tests clean.

* fix(dashboard): sidecar channel cards are read-only (no 404 actions)

Self-review of the previous commit found a half-fix: surfacing sidecar
channels on /api/channels made the cards appear, but every interactive
affordance 404s for a sidecar name — `get_channel` / `configure_channel`
/ `test_channel` are all CHANNEL_REGISTRY-only (find_channel_meta →
None → 404). So the card showed up but "Configure" / "Test" were
broken.

ChannelsPage now treats a `category === "sidecar"` row as read-only:
- the inline Configure (Settings) affordance on the card is suppressed
  (whole-card click still opens the read-only details drawer);
- the DetailsModal's Configure/Test action buttons are replaced with a
  note: runs as an out-of-process sidecar, manage it in config.toml
  ([[sidecar_channels]]) under Config → Sidecar Channels.

DetailsModal renders purely from the list-row data (no per-channel
fetch), so opening details on a sidecar row was already safe; only the
two mutating/registry actions needed gating. The selection checkbox is
inert (no bulk action consumes it), so no other path 404s.

Net: sidecar channels (telegram / ntfy / …) are visible AND have no
broken interactions — the migration is transparent to operators and
the card honestly points at where the channel is actually managed.
Frontend-only; `category` is already a typed field used elsewhere in
this file.
houko added a commit that referenced this pull request May 19, 2026
…mple (#5248)

#5241 deleted crates/librefang-channels/src/telegram.rs (telegram is now
a sidecar adapter) but the "reference adapters" table in the
custom-channel example still listed telegram.rs as an existing
in-process implementation to study — a dangling file reference that
landed on main with the squash-merge. Swap it for matrix.rs (a real
in-process adapter not already in the table), exactly as #5224 swapped
ntfy's row for discord when it removed ntfy.rs.
houko added a commit that referenced this pull request May 19, 2026
…age after sidecar migration (#5250)

Follow-up to #5241 (telegram) and #5224 (ntfy). #5249 fixed the
configured case — once an operator declared [[sidecar_channels]],
a configured/online card appeared. But operators who had never
configured them saw nothing: the Add picker (`pickerChannels` filters
on `configured: false` from /api/channels) had no telegram/ntfy entry
because both were removed from CHANNEL_REGISTRY, and the new
`sidecar_channel_rows` synthesizer only emits rows for entries that
already exist in config.toml. Net effect: the only way to discover
the two first-party SDK sidecar adapters was to read source or
release notes.

Fix: add a small SIDECAR_CATALOG (telegram, ntfy) next to
CHANNEL_REGISTRY and emit `configured: false, category: "sidecar"`
rows for catalog entries not covered by any [[sidecar_channels]].
A catalog entry is "covered" when ANY existing sidecar entry has
either `name == entry` or `channel_type == entry`, so a user-aliased
`name = "my_alerts", channel_type = "telegram"` correctly suppresses
the telegram discovery row instead of double-rendering.

`webhook` is deliberately excluded from the catalog — it still has an
in-process CHANNEL_REGISTRY entry, and surfacing both would render
two cards with the same name.

Dashboard:
- ChannelItem gains `config_template?: string` (backend already emits it).
- `handlePick` routes `category === "sidecar"` to the read-only
  details modal instead of opening the configure form (which would
  404 against /api/channels/{name}/configure — that endpoint is
  CHANNEL_REGISTRY-only).
- DetailsModal renders the TOML snippet for unconfigured sidecar
  rows so operators can copy it into config.toml directly.

Tests (crates/librefang-api/tests/channels_routes_test.rs):
- Replaced `channels_list_without_sidecar_has_no_sidecar_rows`
  (which asserted the now-wrong "no rows when unconfigured"
  contract) with `channels_list_without_sidecar_surfaces_discovery_catalog`,
  which asserts both telegram and ntfy appear as unconfigured
  catalog rows with `category="sidecar"`, no editable fields, and a
  display_name; also asserts discovery rows don't bump
  configured_count.
- Added `channels_list_discovery_row_hidden_when_kind_configured`:
  a configured `name="my_alerts", channel_type="telegram"` must
  suppress the telegram discovery row while leaving the ntfy one in
  place.

Verification:
- cargo check -p librefang-api --lib --tests: clean
- cargo clippy -p librefang-api --lib --tests -- -D warnings: clean
- cargo test -p librefang-api --test channels_routes_test: 31/31 pass
- pnpm typecheck: clean
- pnpm test --run: 650/650 pass
- pnpm build: success
houko added a commit that referenced this pull request May 19, 2026
…5263)

* feat(channels)!: migrate gotify from in-process adapter to sidecar

BREAKING: the `[channels.gotify]` config block is no longer recognised.
Gotify now runs as a `[[sidecar_channels]]` adapter (`python3 -m
librefang.sidecar.adapters.gotify`, channel_type = "gotify"); the new
Python adapter ships in `librefang-sdk` with full parity for inbound
WebSocket subscribe + outbound REST publish. Same pattern as the ntfy
(#5224) and telegram (#5241) migrations.

Added sidecar:

- `sdk/python/librefang/sidecar/adapters/gotify.py` — preserves the
  Rust adapter's runtime semantics. WebSocket `/stream` subscribe with
  the client token in the query string; JSON frames decoded to
  `(id, message, title, priority, appid)`; empty `message` skipped;
  `/`-prefixed text → `Command`, otherwise `Text`; sender derived
  from `title` (fallback `app-{appid}`); metadata carries title /
  priority / app_id; reconnect with exponential backoff (1s → 60s).
  REST `/message` publish posts `{title:"LibreFang", message,
  priority:5}` with the app token in the query string; chunked at
  65535 chars suffixing `(i/N)` to the title. Optional
  `GOTIFY_ACCOUNT_ID` env var surfaces as the multi-bot routing key.
- The WebSocket client is a hand-rolled RFC 6455 reader on `socket`
  + `ssl` — no third-party WS dependency, keeping the SDK
  zero-dep (matches ntfy / telegram). Server frames are read
  unmasked; client→server frames (only pong / close) carry a fresh
  4-byte mask per frame as the spec requires. Fragmented messages
  are reassembled before yielding.
- `sdk/python/tests/test_gotify_adapter.py` — 22 unit tests:
  URL/scheme normalization, required-env enforcement, JSON frame
  parsing including empty-message skip and bad-JSON, command vs
  text routing, sender fallback to app-id, message chunking with
  numbered titles, HTTP 500 / HTTPError surfacing, and a
  loopback-server end-to-end test that performs a real RFC 6455
  handshake and asserts both text frames are yielded as decoded
  strings.

Removed in-process gotify (full cascade):

- `librefang-channels`: `src/gotify.rs` deleted (649 lines), `lib.rs`
  mod, `Cargo.toml` `channel-gotify` feature + `all-channels`
  membership, `channels-allowlist.txt` entry (so `cargo xtask
  channel-policy` permanently blocks any in-process gotify
  reintroduction). Stale comment references in `discord.rs` /
  `slack.rs` / `teams.rs` ("the gotify slice (PR #4447)") generalised
  to "the wiremock pattern" — the pattern is still in use, just no
  longer rooted in a gotify example.
- `librefang-types`: `GotifyConfig` struct + Default + the
  `channels.gotify` field + default, validation hook.
- `librefang-api`: `channel-gotify` feature + membership in
  `all-channels` / `all-channels-no-email`; channel_bridge `use` +
  spawn block + `check_channel` arm + `find_channel_info` arm +
  default-empty test assertion; `routes/channels.rs` ChannelMeta +
  the 4 match arms (`gotify_some` / `serde_to_value` / `len` /
  `ser`); `routes/config.rs` `ch!(gotify)`.
- `librefang-kernel`: `channel_sender` `for_each_channel_field!`
  macro entry + name list.
- `librefang-cli`: TUI `ChannelDef`, with the Notifications-section
  header generalised since gotify joins ntfy off-tree.
- `docs/architecture/sidecar-channels.md`: update the long-tail
  backlog note to record gotify joining ntfy + telegram.

The separate gotify push-notification provider (`push_provider =
"gotify"`, used by device pairing under `crates/librefang-kernel/
src/pairing.rs`) is **unaffected** — different feature, deliberately
left intact.

Operator action required:

  # config.toml: remove
  [channels.gotify]
  server_url = "..."
  app_token_env = "GOTIFY_APP_TOKEN"
  client_token_env = "GOTIFY_CLIENT_TOKEN"

  # config.toml: add
  [[sidecar_channels]]
  name = "gotify"
  command = "python3"
  args = ["-m", "librefang.sidecar.adapters.gotify"]
  channel_type = "gotify"
  [sidecar_channels.env]
  GOTIFY_SERVER_URL = "https://gotify.example.com"
  GOTIFY_APP_TOKEN = "A..."     # secrets.env via secrets system
  GOTIFY_CLIENT_TOKEN = "C..."  # secrets.env via secrets system

Verification:

- `cd sdk/python && pytest`: 97 tests pass (22 new gotify, 75
  pre-existing ntfy/telegram/runtime/protocol all green).
- Loopback WebSocket handshake test exercises real RFC 6455
  Sec-WebSocket-Accept computation and frame parsing.
- `cargo check / clippy / test` deferred to CI — the host's cargo is
  a Docker wrapper with `rust:latest` not cached locally, and
  `cargo test --workspace` is forbidden by repo policy.
- `kernel_config_schema.golden.json` left with 12 residual gotify
  refs — the matching test is `#[ignore]`'d on `main` (golden has
  drifted across earlier migrations); regen is a follow-up via
  `cargo test -p librefang-api --test config_schema_golden --
  --ignored regenerate_golden --nocapture`.

* docs(channels): redirect [channels.gotify] examples to sidecar pattern

Self-review follow-up to the gotify sidecar migration. The main commit
deleted the in-process `GotifyConfig` / `GotifyAdapter` but left
several docs pages still showing the old `[channels.gotify]` block
and pointing at the deleted Rust adapter. Telegram's #5241 follow-up
landed the same kind of redirect — mirror that pattern.

- `docs/.../configuration/page.mdx` (en + zh): replace the old
  `[channels.gotify]` field reference with a redirect-style
  `[[sidecar_channels]]` example, mirroring the existing Telegram
  redirect. Also drop the Gotify row from the
  "validator-checked env vars" table — the validation hook was
  removed with `GotifyConfig`.
- `docs/.../configuration/channels/page.mdx` (en + zh): same
  redirect.
- `docs/.../integrations/channels/integrations/page.mdx` (en + zh):
  the "## Gotify" section was already stale before this PR — it used
  hypothetical `GOTIFY_URL` / `GOTIFY_TOKEN` env vars and a
  `[channels.gotify]` block whose `url_env` / `token_env` keys never
  matched the actual `GotifyConfig` shape. Rewritten to point at the
  sidecar with the correct
  `GOTIFY_SERVER_URL` / `GOTIFY_APP_TOKEN` / `GOTIFY_CLIENT_TOKEN`
  env var names.
- `docs/.../security/integrity/page.mdx`,
  `docs/.../architecture/security/page.mdx` (en + zh): drop the
  `GotifyAdapter` row from the "Zeroizing<String> usage in
  in-process channel adapters" table — the struct is gone with this
  PR; the env vars themselves are now read by the Python sidecar
  process, which inherits the daemon's environment without needing
  in-Rust zeroizing.
- `examples/custom-channel/README.md`: drop the `gotify` row from
  the "study these existing adapters" comparison table — `gotify.rs`
  no longer exists.

The env-var tables under "Required Environment Variables" at
`configuration/page.mdx` and `configuration/security/page.mdx` are
deliberately kept — the env vars themselves are still in use, just
read by the sidecar process rather than the deleted in-process
adapter.

No code changes; docs-only follow-up to #5263.

* fix(sidecar/gotify): cap inbound WS frame payload + harden coverage

Round-2 self-review follow-up.

Cap inbound WebSocket frame payloads at 1 MiB (MAX_FRAME_PAYLOAD).
RFC 6455 allows up to 2^63 bytes via the 64-bit extended-length form;
the previous unbounded implementation would faithfully try to read
any length the server announced. A hostile or buggy gotify server
could send a 64-bit length and force the sidecar to spin in
`_recv_exact` allocating multi-exabyte buffers. The cap is well over
Gotify's largest realistic payload (a 65 535-char message body
wrapped in a JSON envelope sits under 100 KiB), so legitimate traffic
is unaffected. Cap breach raises a RuntimeError, which the producer
loop catches and treats as a transport error → reconnect with the
existing exponential backoff.

Three new tests close earlier coverage gaps:

- `test_websocket_reader_responds_to_server_ping_with_pong`: drives
  the reader against a loopback server that sends `text → ping → text
  → close`. Asserts both text frames are yielded AND that the client
  sent back exactly one masked pong carrying the same payload as the
  ping, plus the close echo. Exercises the previously-untested
  control-frame branches.
- `test_websocket_reader_rejects_oversized_frame`: server announces
  a 1 GiB 64-bit-length text frame; client must raise RuntimeError
  matching "exceeds cap" instead of trying to allocate. Locks in the
  new MAX_FRAME_PAYLOAD guard.
- `test_account_id_surfaced_via_ready_event` /
  `test_no_account_id_when_unset`: assert that `GOTIFY_ACCOUNT_ID`
  reaches LibreFang via the base SidecarAdapter.ready_event()
  multi-bot routing key (and that an empty env var emits no
  `account_id` field) — closes the gap noted in the first review.

Verification: `cd sdk/python && pytest` — 101 tests pass (4 new
gotify + 22 prior gotify + 75 unchanged ntfy/telegram/runtime).
houko added a commit that referenced this pull request May 19, 2026
…5264)

* feat(channels)!: migrate mastodon from in-process adapter to sidecar

BREAKING: the `[channels.mastodon]` config block is no longer
recognised. Mastodon now runs as a `[[sidecar_channels]]` adapter
(`python3 -m librefang.sidecar.adapters.mastodon`, channel_type =
"mastodon"); the new Python adapter ships in `librefang-sdk` with
full parity for SSE primary + REST polling fallback inbound and
form-encoded REST outbound. Same pattern as the ntfy (#5224),
telegram (#5241), and gotify (#5263) migrations.

Added sidecar:

- `sdk/python/librefang/sidecar/adapters/mastodon.py` — preserves the
  Rust adapter's runtime semantics. SSE subscribe to
  `{instance}/api/v1/streaming/user` with Bearer auth; filter
  `event: notification` payloads to `type == "mention"`; strip HTML
  from `status.content` (`<br>` / `</p>` / `</div>` / `</li>` insert
  newlines, entities decoded via stdlib `html.unescape`); `/cmd args`
  → Command, else Text; sender from `display_name` fallback
  `username`; metadata carries status_id / notification_id / acct /
  visibility / in_reply_to_id; `thread_id` = in_reply_to_id;
  `verify_credentials` at startup discovers the bot's own account id
  so self-mentions are skipped; REST polling fallback when SSE fails
  (newest-first ordering, high-water-mark before iteration);
  exponential-backoff reconnect (1s → 60s).
- Outbound: POST `/api/v1/statuses` form-encoded with `status`,
  `visibility="unlisted"`, optional `in_reply_to_id`. Chunked at
  `MAX_MESSAGE_LEN` (default 500); chained chunks thread under each
  prior response id so a long reply stays one conversational thread.
  Replies to inbound mentions use `cmd.thread_id` (in_reply_to_id).
  `suppress_error_responses = True` — Mastodon posts are public.
- New env-var knobs beyond the in-process equivalents:
  `MASTODON_VISIBILITY` (public/unlisted/private/direct, default
  unlisted), `MASTODON_MAX_MESSAGE_LEN` (default 500, raise for
  longer-toot instances), optional `MASTODON_ACCOUNT_ID` for
  multi-bot routing surfaced via `ready_event`.
- `sdk/python/tests/test_mastodon_adapter.py` — 32 unit tests:
  URL/scheme normalization, required-env enforcement, visibility
  allowlist, max_len validation, account_id surfacing, HTML stripper
  edge cases (plain passthrough, basic tags, block-close newlines,
  entity decoding, typical mention anchor shape), notification
  parsing (full shape, thread_id from in_reply_to, non-mention
  skip, self-mention skip, empty text skip, slash command,
  display_name fallback), message chunking, REST publish (bearer
  auth + form + visibility, chunked thread chaining, reply to
  inbound thread, HTTPError surfaced, 5xx surfaced, custom
  visibility).

Removed in-process mastodon (full cascade):

- `librefang-channels`: `src/mastodon.rs` deleted (850 lines),
  `lib.rs` mod, `Cargo.toml` `channel-mastodon` feature +
  `all-channels` membership, `channels-allowlist.txt` entry (so
  `cargo xtask channel-policy` permanently blocks any in-process
  mastodon reintroduction).
- `librefang-types`: `MastodonConfig` struct + Default + the
  `channels.mastodon` field + default, validation hook.
- `librefang-api`: `channel-mastodon` feature + membership in
  `all-channels` / `all-channels-no-email`; channel_bridge `use` +
  spawn block + `check_channel` arm + `find_channel_info` arm +
  default-empty test assertion; `routes/channels.rs` ChannelMeta
  + 4 match arms (`mastodon_some` / `serde_to_value` / `len` /
  `ser`); `routes/config.rs` `ch!(mastodon)`. Added Mastodon to
  SIDECAR_CATALOG so the dashboard Channels page surfaces a
  configurable discovery row.
- `librefang-kernel`: `channel_sender` `for_each_channel_field!`
  macro entry + name list.
- `librefang-cli`: TUI `ChannelDef`.
- `docs/src/app/*`: redirect both English and Chinese config /
  channels / integrations pages from `[channels.mastodon]` to
  `[[sidecar_channels]]`.

Operator action required:

  # config.toml: remove
  [channels.mastodon]
  instance_url = "..."
  access_token_env = "MASTODON_ACCESS_TOKEN"

  # config.toml: add
  [[sidecar_channels]]
  name = "mastodon"
  command = "python3"
  args = ["-m", "librefang.sidecar.adapters.mastodon"]
  channel_type = "mastodon"
  [sidecar_channels.env]
  MASTODON_INSTANCE_URL = "https://mastodon.social"
  # secrets.env via secrets system:
  # MASTODON_ACCESS_TOKEN=...

Verification:

- `cd sdk/python && pytest`: 133 tests pass (32 new mastodon, 26
  prior gotify, 75 unchanged ntfy/telegram/runtime/protocol).
- `cargo check / clippy / test` deferred to CI — the host's cargo is
  a Docker wrapper with `rust:latest` not cached locally, and
  workspace-wide `cargo test` is forbidden by repo policy.
- `kernel_config_schema.golden.json`: the matching test is
  `#[ignore]`'d on main (the fixture has drifted across earlier
  migrations); regen is a follow-up via `cargo test -p librefang-api
  --test config_schema_golden -- --ignored regenerate_golden`.

* fix(sidecar/mastodon): call _verify_credentials before SSE/poll loop

Self-review follow-up. The Rust adapter called `validate()` at the
start of `ChannelAdapter::start()` to (a) validate the OAuth token
and (b) discover the bot's own account id used by the self-mention
guard in `parse_mastodon_notification`. My port defined
`_verify_credentials` but never actually called it from
`_producer_blocking`, so `self.own_account_id` stayed `None` and the
guard:

    if self.own_account_id and account_id == self.own_account_id:
        return None

silently short-circuited — a self-mention from the bot's own toots
would be delivered back to the agent as an inbound event,
potentially looping if the agent's reply itself triggers another
mention. Same parity bug the in-process adapter explicitly defended
against.

Fix: `_producer_blocking` now runs verify with exponential backoff
(1s → 60s) before entering the SSE-with-polling-fallback loop. The
loop refuses to emit any event until verify has succeeded at least
once. Transient verify failures (instance returning 5xx, network
flake) retry instead of taking the sidecar down; persistent auth
failures keep retrying — the supervisor can kill the subprocess
if the operator wants a hard fail.

Three new tests close coverage gaps the original PR missed:

- `test_verify_credentials_sets_own_account_id`: monkeypatch
  urlopen, assert the request hits
  `/api/v1/accounts/verify_credentials` with the right Bearer
  header AND that the response `id` is stored on
  `own_account_id`. Locks in the discovery contract.
- `test_verify_credentials_raises_on_bad_token`: a 401 response
  surfaces the HTTPError; `own_account_id` stays None so the
  self-mention guard does not match anything by accident.
- `test_self_mention_skipped_only_after_verify`: the failing
  case — without verify, a self-mention IS delivered (proving
  the latent bug shape); after `own_account_id` is set the same
  payload is filtered. Catches a future regression where someone
  removes the verify call again.

Verification: `cd sdk/python && pytest` — 35 mastodon tests pass
(+3 new), full SDK suite 136 passed.
houko added a commit that referenced this pull request May 20, 2026
…5297)

Replaces the 535-line in-process librefang-channels::twitch adapter
with the Python sidecar librefang.sidecar.adapters.twitch. Same pattern
as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky
#5277, reddit #5281.

Behaviour parity:
- OAuth PASS / NICK handshake (oauth: prefix auto-added when absent)
- JOIN #channel for each configured channel
- PRIVMSG -> ChannelMessage with case-insensitive self-skip
- /cmd / !cmd -> Content::Command, else Content::Text
- PING -> PONG keepalive
- MAX_MESSAGE_LEN = 500 chunking
- Exponential reconnect backoff (1s -> 60s)
- TWITCH_ACCOUNT_ID multi-bot routing

Three improvements over the Rust adapter (file/line evidence):
1. TLS by default. The Rust adapter hard-coded plaintext port 6667
   (TWITCH_IRC_PORT in crates/librefang-channels/src/twitch.rs:24) and
   sent the OAuth token in cleartext on every connect. The sidecar
   defaults to irc.chat.twitch.tv:6697 with ssl.create_default_context().
   Plaintext is reachable only via TWITCH_PLAINTEXT=1 (tests/mock
   listeners). Credential-leak-on-wire fixed automatically on upgrade.
2. Per-message reply threading via IRCv3 tags. The Rust adapter never
   requested any capability and discarded any @-tag block (parse_privmsg
   in twitch.rs:100 only handled the bare ":nick!u@h PRIVMSG #ch :text"
   shape), so chunked replies arrived as a flat sequence with no link
   back. The sidecar issues CAP REQ :twitch.tv/tags twitch.tv/commands,
   parses @id, surfaces it as thread_id, and attaches
   @reply-parent-msg-id=<id> on outbound PRIVMSG.
3. Token-bucket send rate-limiter. Twitch drops the bot above 20 msgs
   / 30 s (100 / 30 s if mod). The Rust adapter shipped zero throttling
   (send() in twitch.rs:312 wrote PRIVMSG frames straight to the
   socket). Sidecar gates every outbound chunk; tunable via
   TWITCH_RATE_LIMIT_MSGS / TWITCH_RATE_LIMIT_SECS.

Cascade removal: crates/librefang-channels/src/twitch.rs deleted;
lib.rs mod decl, channels-allowlist.txt entry, channel-twitch feature
in librefang-channels + librefang-api (incl. all-channels /
all-channels-no-email membership); TwitchConfig + ChannelsConfig.twitch
field + Default; validation env-var hook; channel_bridge import +
spawn block + find_channel_info! arm + check_channel! arm + default-
empty test assertion; routes/channels.rs ChannelMeta + 4 match arms;
routes/config.rs ch!(twitch); kernel channel_sender macro entry +
EXPECTED name; CLI TUI ChannelDef; types/config/mod.rs unit tests
referencing TwitchConfig; docs (en + zh): configuration / channels /
integrations / architecture / communication pages -- [channels.twitch]
blocks replaced with [[sidecar_channels]] redirect; architecture-row
moves twitch into the sidecar list.

New env-var knobs: TWITCH_NICK, TWITCH_CHANNELS (comma-separated, no
'#'), optional TWITCH_ACCOUNT_ID, optional TWITCH_RATE_LIMIT_MSGS /
TWITCH_RATE_LIMIT_SECS, optional TWITCH_PLAINTEXT / TWITCH_HOST /
TWITCH_PORT (test escape hatches).

Operator action required: an existing [channels.twitch] block is no
longer recognised -- re-declare as a [[sidecar_channels]] running
python3 -m librefang.sidecar.adapters.twitch with env vars
TWITCH_NICK, TWITCH_CHANNELS (in [sidecar_channels.env]) and
TWITCH_OAUTH_TOKEN (~/.librefang/secrets.env).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy -p librefang-types -p librefang-channels -p librefang-kernel
  -p librefang-api -p librefang-cli --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels -p librefang-kernel
  --lib: 843 passed
- cargo test -p librefang-api --lib: 686 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && python -m pytest tests/test_twitch_adapter.py:
  68 passed
houko added a commit that referenced this pull request May 20, 2026
Replaces the 640-line in-process librefang-channels::nextcloud adapter
with the Python sidecar librefang.sidecar.adapters.nextcloud. Same
pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264,
bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298.

Behaviour parity:
- GET /ocs/v2.php/cloud/user startup credential probe
- Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default
- Empty allowlist auto-discovers via apps/spreed/api/v4/room
- Same Bearer + OCS-APIRequest: true headers
- Slash-command (/cmd args) routing
- Multi-bot account_id metadata injection
- 32000-char message chunking (MAX_MESSAGE_LEN)
- Per-room transport-error isolation
- 304-as-no-op handling of Talk's long-poll-expired response
- suppress_error_responses (Talk rooms are multi-participant)

Three improvements over the Rust adapter (file/line evidence in
CHANGELOG):
1. Outbound threading via replyTo is now actually wired. The Rust
   api_send_message (nextcloud.rs:130-160) posted just
   {"message": ...} — Talk's replyTo form parameter was never sent
   and chunked/threaded replies always landed at the room root.
   The sidecar surfaces the inbound id (or parentMessage.id when
   inside a thread) as thread_id, and on_send posts replyTo so
   replies thread correctly. Mirrors reddit/bluesky/mastodon/
   rocketchat.
2. Self-skip on (actorType, actorId) == ("users", own_user_id)
   rather than actorId alone. The Rust adapter compared on
   actorId (nextcloud.rs:338) without inspecting actorType, so a
   Talk guest/federated_users actor whose id happened to match
   the bot's user id silently spoofed self-skip. Parallels the
   rocketchat #5298 fix.
3. Dedupe set on id, bounded SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000. The Rust adapter only relied on the
   server-side lastKnownMessageId cursor (nextcloud.rs:347-354);
   under retry/re-poll boundaries Talk can resend the same id.
   Same policy as reddit/rocketchat.

Cascade removal: crates/librefang-channels/src/nextcloud.rs deleted;
lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud
feature in librefang-channels + librefang-api (incl. all-channels /
all-channels-no-email membership); NextcloudConfig +
ChannelsConfig.nextcloud field + Default; validation env-var hook;
channel_bridge import + spawn block + find_channel_info! arm +
check_channel! arm + default-empty test assertion; routes/channels.rs
ChannelMeta + 4 match arms; routes/config.rs ch!(nextcloud); kernel
channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef;
docs (en + zh): configuration / channels / integrations /
architecture / communication / security pages — [channels.nextcloud]
blocks replaced with [[sidecar_channels]] redirect; architecture-row
moves Nextcloud Talk into the sidecar list. routes/channels.rs
SIDECAR_CATALOG gains a nextcloud entry. Golden fixture regenerated.

New env-var knobs: NEXTCLOUD_SERVER_URL (replaces server_url),
optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty =
auto-discover), optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing,
optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1).

Operator action required: an existing [channels.nextcloud] block is
no longer recognised — re-declare as a [[sidecar_channels]] running
python3 -m librefang.sidecar.adapters.nextcloud with env var
NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN
(in ~/.librefang/secrets.env).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels
  -p librefang-kernel --lib: 842 passed
- cargo test -p librefang-api --lib: 686 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && pytest tests/test_nextcloud_adapter.py: 58 passed
houko added a commit that referenced this pull request May 20, 2026
…#5301)

Replaces the 640-line in-process librefang-channels::nextcloud adapter
with the Python sidecar librefang.sidecar.adapters.nextcloud. Same
pattern as ntfy #5224, telegram #5241, gotify #5263, mastodon #5264,
bluesky #5277, reddit #5281, twitch #5297, rocketchat #5298.

Behaviour parity:
- GET /ocs/v2.php/cloud/user startup credential probe
- Per-room chat/<token>?lookIntoFuture=1 polling at 3 s default
- Empty allowlist auto-discovers via apps/spreed/api/v4/room
- Same Bearer + OCS-APIRequest: true headers
- Slash-command (/cmd args) routing
- Multi-bot account_id metadata injection
- 32000-char message chunking (MAX_MESSAGE_LEN)
- Per-room transport-error isolation
- 304-as-no-op handling of Talk's long-poll-expired response
- suppress_error_responses (Talk rooms are multi-participant)

Three improvements over the Rust adapter (file/line evidence in
CHANGELOG):
1. Outbound threading via replyTo is now actually wired. The Rust
   api_send_message (nextcloud.rs:130-160) posted just
   {"message": ...} — Talk's replyTo form parameter was never sent
   and chunked/threaded replies always landed at the room root.
   The sidecar surfaces the inbound id (or parentMessage.id when
   inside a thread) as thread_id, and on_send posts replyTo so
   replies thread correctly. Mirrors reddit/bluesky/mastodon/
   rocketchat.
2. Self-skip on (actorType, actorId) == ("users", own_user_id)
   rather than actorId alone. The Rust adapter compared on
   actorId (nextcloud.rs:338) without inspecting actorType, so a
   Talk guest/federated_users actor whose id happened to match
   the bot's user id silently spoofed self-skip. Parallels the
   rocketchat #5298 fix.
3. Dedupe set on id, bounded SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000. The Rust adapter only relied on the
   server-side lastKnownMessageId cursor (nextcloud.rs:347-354);
   under retry/re-poll boundaries Talk can resend the same id.
   Same policy as reddit/rocketchat.

Cascade removal: crates/librefang-channels/src/nextcloud.rs deleted;
lib.rs mod decl, channels-allowlist.txt entry, channel-nextcloud
feature in librefang-channels + librefang-api (incl. all-channels /
all-channels-no-email membership); NextcloudConfig +
ChannelsConfig.nextcloud field + Default; validation env-var hook;
channel_bridge import + spawn block + find_channel_info! arm +
check_channel! arm + default-empty test assertion; routes/channels.rs
ChannelMeta + 4 match arms; routes/config.rs ch!(nextcloud); kernel
channel_sender macro entry + EXPECTED name; CLI TUI ChannelDef;
docs (en + zh): configuration / channels / integrations /
architecture / communication / security pages — [channels.nextcloud]
blocks replaced with [[sidecar_channels]] redirect; architecture-row
moves Nextcloud Talk into the sidecar list. routes/channels.rs
SIDECAR_CATALOG gains a nextcloud entry. Golden fixture regenerated.

New env-var knobs: NEXTCLOUD_SERVER_URL (replaces server_url),
optional NEXTCLOUD_ROOMS (comma-separated room tokens, empty =
auto-discover), optional NEXTCLOUD_ACCOUNT_ID for multi-bot routing,
optional NEXTCLOUD_POLL_INTERVAL_SECS (default 3, floor 1).

Operator action required: an existing [channels.nextcloud] block is
no longer recognised — re-declare as a [[sidecar_channels]] running
python3 -m librefang.sidecar.adapters.nextcloud with env var
NEXTCLOUD_SERVER_URL (in [sidecar_channels.env]) and NEXTCLOUD_TOKEN
(in ~/.librefang/secrets.env).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels
  -p librefang-kernel --lib: 842 passed
- cargo test -p librefang-api --lib: 686 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && pytest tests/test_nextcloud_adapter.py: 58 passed
houko added a commit that referenced this pull request May 20, 2026
)

* feat(channels)!: migrate webex from in-process adapter to sidecar

Replaces the 645-line in-process librefang-channels::webex adapter
(Cisco Mercury WebSocket gateway for activity events + GET
/messages/<id> REST follow-up + POST /messages publish) with the
Python sidecar librefang.sidecar.adapters.webex. Same pattern as
ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky
#5277, reddit #5281, twitch #5297, rocketchat #5298, discord #5299,
nextcloud #5301, slack #5302.

Behaviour parity:
- GET /people/me startup credential probe (bot id + display name)
- Hard-coded Mercury WSS URL with Bearer auth on upgrade request
  (no device-registration handshake — the Rust adapter never did
  one either)
- data.activity envelope parsing; verb=="post" filter; actor-id
  self-skip
- GET /messages/<id> REST follow-up for the full message body
- Room filter (empty allowlist = all rooms the bot is in)
- Slash-command (/cmd args) → Command routing
- roomType == "group" → is_group mapping
- Multi-bot account_id metadata injection
- 7439-char message chunking (WEBEX_MSG_LIMIT parity with the Rust
  MAX_MESSAGE_LEN)
- Exponential reconnect backoff (1s → 60s)
- ChannelType::Custom("webex") preserved as channel_type = "webex"
  on the sidecar entry — existing routing / channel_role_mapping
  keys that reference webex continue to resolve.

Four improvements over the Rust adapter (file/line evidence in
CHANGELOG):
1. parentId outbound threading wired. The Rust api_send_message
   (webex.rs:171-201) posted just {"roomId", "text"} — Webex's
   parentId field was never sent and chunked replies always landed
   at the room root regardless of inbound context; the inbound
   side dropped the message id entirely (thread_id: None at
   webex.rs:438). The sidecar surfaces the inbound id (or inbound
   parentId when inside a thread) as thread_id, and on_send posts
   parentId populated so replies thread correctly. Mirrors reddit
   / rocketchat / nextcloud / mastodon / bluesky.
2. 429 Retry-After honoured on both GET /messages/<id> fetch and
   POST /messages send. The Rust adapter had no 429 handling at
   either call site, so a server-side rate-limit either lost the
   inbound fetch or returned Err and dropped the outbound. The
   sidecar parses Retry-After (default 30 s fallback, floor 1 s,
   cap MAX_BACKOFF_SECS), sleeps, and retries once before
   logging-and-continuing on the second 429. Same shape as
   #5303.
3. Mercury activity-id dedupe. Mercury can re-deliver an
   activity.object.id on reconnect (Rust adapter had no dedupe,
   see unconditional emit at webex.rs:459). Bounded local set on
   activity.object.id with SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000.
4. Explicit HTTP timeouts on every urlopen call (urllib has no
   default timeout; a hung Webex API would hang the producer
   thread forever).

Cascade removal:
- src/webex.rs deleted; lib.rs mod decl, channels-allowlist.txt
  entry, cargo features in librefang-channels + librefang-api
  (incl. membership in all-channels / all-channels-no-email)
  removed
- WebexConfig struct + Default impl + ChannelsConfig.webex field
  + matching Default entry removed; validation hook removed
- channel_bridge.rs: import, spawn block, find_channel_info! arm,
  check_channel! arm, default-empty test assertion removed
- routes/channels.rs: ChannelMeta entry + 4 match arms removed;
  SIDECAR_CATALOG webex entry added
- routes/config.rs ch!(webex) call removed
- kernel channel_sender for_each_channel_field! macro entry +
  EXPECTED name-list entry removed
- CLI: TUI ChannelDef row removed, migrated-to-sidecar comment
  grown
- README / AGENTS for librefang-channels updated to list webex
  among the migrated-to-sidecar adapters
- Docs (EN + ZH): configuration/page.mdx,
  configuration/channels/page.mdx,
  integrations/channels/enterprise/page.mdx,
  configuration/security/page.mdx, architecture/page.mdx —
  [channels.webex] blocks replaced with [[sidecar_channels]]
  redirect blocks (fence language `bash` per Shiki); architecture
  wave table moves webex out of Wave 3 in-process and into the
  sidecar list

New env-var knobs (read from [sidecar_channels.env]):
- WEBEX_ALLOWED_ROOMS (comma-separated room IDs, empty = allow all)
- WEBEX_ACCOUNT_ID (optional, multi-bot routing key)
Secret in ~/.librefang/secrets.env: WEBEX_BOT_TOKEN.

Operator action required: an existing [channels.webex] block is
no longer recognised — re-declare as a [[sidecar_channels]]
running python3 -m librefang.sidecar.adapters.webex with
WEBEX_BOT_TOKEN (in ~/.librefang/secrets.env) and any of the
optional knobs above (in [sidecar_channels.env]).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo test -p librefang-types -p librefang-channels
  -p librefang-kernel --lib: 420 + 838 + 1087 passed
- cargo test -p librefang-api --lib: 685 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cd sdk/python && pytest tests/test_webex_adapter.py: 78 passed

* fix(channels/webex): apply review nits — display name, suppress flag, dead code, redundant filter

Self-review pass on top of the migration commit; addresses 4 nits from the
pre-merge review without changing the migration's core behaviour:

1. **personDisplayName drives user_name** (`webex.py:281-291, 326-330`). The
   migration commit preserved the Rust adapter's `webex.rs:431` behaviour of
   using `personEmail` unconditionally, which leaks emails into bot logs and
   dashboard UI. Now `parse_webex_message` prefers `personDisplayName` from the
   `/messages/<id>` body and falls back to `personEmail` when the field is
   absent (older Webex orgs / some service-account principals) or empty;
   `personEmail` stays in metadata for routing / audit. 3 new tests
   (`test_parse_prefers_person_display_name_for_user_name`,
   `…_falls_back_to_email_when_display_name_missing`,
   `…_falls_back_to_email_when_display_name_empty`).

2. **Explicit `suppress_error_responses = False`** on `WebexAdapter` with a
   precedent comment naming the discord / slack / twitch chat-room default vs.
   the mastodon / bluesky / reddit / nextcloud public-broadcast default. The
   migration left this implicit; an explicit class-level attr matches how
   `twitch.py:312` documents the choice so the next reader doesn't have to
   reconstruct the reasoning.

3. **`register_webhook` dead-code drop documented in CHANGELOG**. The Rust
   adapter carried a never-wired `register_webhook` helper at `webex.rs:137-168`
   (marked `#[allow(dead_code)]`) for an HTTP-webhook delivery alternative the
   channel-bridge never enabled; the sidecar drops it. The CHANGELOG now names
   this explicitly and points operators who want webhook delivery at the
   canonical generic `[[sidecar_channels]]` running
   `librefang.sidecar.adapters.webhook`.

4. **Skip redundant room filter inside `parse_webex_message`** when called
   from `_handle_envelope` (`webex.py:867-878`). `_handle_envelope` already
   filtered against `self.allowed_rooms` at line 855 before paying for the
   REST fetch; passing `allowed_rooms=[]` into the parser drops one redundant
   list-membership check per inbound message. The parser keeps the parameter
   for direct callers (tests; future code paths).

Verification (run inside the worktree):

* `cd sdk/python && python3 -m pytest tests/test_webex_adapter.py -q` →
  **81 passed** (was 78; +3 personDisplayName tests). No existing test
  modified.
* `cargo check` / `clippy` not re-run — this commit is Python + CHANGELOG only,
  no Rust touched.

Diff: +73 / -5 across `CHANGELOG.md`,
`sdk/python/librefang/sidecar/adapters/webex.py`,
`sdk/python/tests/test_webex_adapter.py`.
houko added a commit that referenced this pull request May 20, 2026
* feat(channels)!: migrate line from in-process adapter to sidecar

Replaces the 881-line in-process librefang-channels::line adapter
(axum webhook route at /channels/line/webhook on the shared API
server, X-Line-Signature HMAC-SHA256 verification, POST
/v2/bot/message/push for outbound) with the Python sidecar
librefang.sidecar.adapters.line. Same pattern as ntfy #5224,
telegram #5241, gotify #5263, mastodon #5264, bluesky #5277,
reddit #5281, twitch #5297, rocketchat #5298, discord #5299,
nextcloud #5301, slack #5302, webex #5309.

Behaviour parity:
- GET /v2/bot/info startup credential probe (bot display name)
- HTTP webhook server (BaseHTTPRequestHandler over
  ThreadingTCPServer) on LINE_WEBHOOK_PORT (default 9090) +
  LINE_WEBHOOK_PATH (default /webhook)
- X-Line-Signature HMAC-SHA256 verification over the raw wire
  bytes (constant-time compare, line.rs:229-250 parity, incl.
  the wire-bytes-vs-JSON-roundtrip regression)
- Only message events with type==text (other event types and
  non-text message types dropped, line.rs:256-273)
- Source-type -> reply_to mapping (user -> userId, group ->
  groupId, room -> roomId; group/room -> is_group=true,
  line.rs:280-290)
- Slash-command routing (/cmd args -> Command)
- Metadata preservation: user_id / reply_to / reply_token /
  source_type
- Multi-bot account_id injection via LINE_ACCOUNT_ID
- 5000-char chunking (LINE_MSG_LIMIT parity with MAX_MESSAGE_LEN
  at line.rs:39)
- Image-branch wire shape (originalContentUrl + previewImageUrl
  both set to the caller URL, caption sent as follow-up text
  push, line.rs:464-490)
- ChannelType::Custom("line") preserved as channel_type = "line"
  on the sidecar entry so existing routing /
  channel_role_mapping keys continue to resolve

Three improvements over the Rust adapter:
1. 429 Retry-After honoured on outbound. The Rust
   api_push_message (line.rs:148-184) had no 429 handling;
   throttled push returned Err and the chunked reply dropped on
   the floor. Sidecar parses Retry-After (default 30 s fallback,
   floor 1 s, cap MAX_BACKOFF_SECS), sleeps, retries once, then
   logs-and-continues on the second 429. Same shape as #5303.
2. Inbound dedupe on message.id. LINE redelivers webhook events
   when the operator's endpoint fails; the Rust handler at
   line.rs:413-427 emitted every event unconditionally, causing
   duplicate agent invocations on transient failures. Bounded
   local set on message.id with SEEN_MESSAGES_MAX=10000 /
   SEEN_MESSAGES_EVICT=5000 (matches reddit / rocketchat /
   nextcloud / webex policy).
3. Explicit HTTP timeouts on every urlopen call (urllib has no
   default timeout; a hung LINE API would otherwise hang the
   worker thread forever).

Cascade removal:
- src/line.rs deleted; lib.rs mod decl, channels-allowlist.txt
  entry, cargo features in librefang-channels + librefang-api
  (incl. membership in all-channels / all-channels-no-email)
  removed
- LineConfig struct + Default impl + ChannelsConfig.line field
  + matching Default entry removed; validation hook removed
- channel_bridge.rs: import, spawn block, find_channel_info!
  arm, check_channel! arm, default-empty test assertion removed
- routes/channels.rs: ChannelMeta entry + 5 match arms removed
  (is_some / serialize / len / ser / is_channel_configured);
  webhook_route_suffix allowlist entry removed (sidecar runs its
  own webhook listener, not on the shared API port);
  SIDECAR_CATALOG line entry added
- routes/config.rs ch!(line) call removed
- kernel channel_sender for_each_channel_field! entry + EXPECTED
  name-list entry removed
- CLI: TUI ChannelDef row removed, migrated-to-sidecar comment
  grown
- README / AGENTS for librefang-channels updated; the mandatory-
  HMAC paragraph now lists Teams + DingTalk only (LINE moves to
  the sidecar but keeps the same X-Line-Signature contract
  inside the new process)
- Docs (EN + ZH): configuration/page.mdx,
  configuration/channels/page.mdx,
  configuration/security/page.mdx,
  integrations/channels/integrations/page.mdx -
  [channels.line] blocks replaced with [[sidecar_channels]]
  redirect blocks; security env-var table flags both
  LINE_CHANNEL_SECRET and LINE_CHANNEL_ACCESS_TOKEN as
  sidecar-only; validation table drops the LINE row

New env-var knobs (read from [sidecar_channels.env]):
- LINE_WEBHOOK_PORT (default 9090)
- LINE_WEBHOOK_PATH (default /webhook)
- LINE_BIND_HOST (default 0.0.0.0)
- LINE_ACCOUNT_ID (optional, multi-bot routing key)
Secrets in ~/.librefang/secrets.env: LINE_CHANNEL_SECRET +
LINE_CHANNEL_ACCESS_TOKEN.

Operator action required (substantive): the sidecar runs its
OWN HTTP webhook server - it is no longer mounted on the
LibreFang API port - so the webhook URL registered at the LINE
Developers Console must be updated to point at the sidecar host
(typical pattern: an HTTPS reverse proxy in front of the
sidecar's LINE_WEBHOOK_PORT). An existing [channels.line] block
is no longer recognised - re-declare as a [[sidecar_channels]]
running python3 -m librefang.sidecar.adapters.line with
LINE_CHANNEL_SECRET + LINE_CHANNEL_ACCESS_TOKEN (in
~/.librefang/secrets.env) and any of the optional knobs above
(in [sidecar_channels.env]).

Verification:
- cargo check --workspace --lib: clean
- cargo clippy --workspace --all-targets -- -D warnings: clean
- cargo fmt --check: clean
- cargo test --lib -p librefang-types -p librefang-channels
  -p librefang-kernel: 420 + 838 + 1087 passed
- cargo test --lib -p librefang-api: 685 passed
- cargo test -p librefang-api --test channels_routes_test
  --test sidecar_describe_test: 38 + 2 passed
- cargo xtask channel-policy: passed
- cd sdk/python && pytest tests/test_line_adapter.py:
  68 passed

* docs(channels): correct line sidecar HTTP server class references

The module docstring claimed `ThreadingHTTPServer` but the
implementation uses `socketserver.ThreadingTCPServer` (intentional:
skips `HTTPServer.server_bind()`'s `socket.getfqdn()` DNS lookup on
startup). Align the three doc references with the actual class and
spell out why we don't use `ThreadingHTTPServer` so the next reader
doesn't "fix" it back.
houko added a commit that referenced this pull request May 20, 2026
Replaces the 758-line in-process `librefang-channels::qq`
adapter (token mint + WebSocket gateway + HELLO/IDENTIFY/READY
handshake + heartbeat + DISPATCH for 4 event types + REST send
with markdown stripping) with an out-of-process Python sidecar
at `sdk/python/librefang/sidecar/adapters/qq.py`. Same pattern as
ntfy #5224, telegram #5241, gotify #5263, mastodon #5264, bluesky
#5277, reddit #5281, twitch #5297, rocketchat #5298, discord
#5299, nextcloud #5301, slack #5302, webex #5309, line #5312,
zulip #5310, mattermost #5315, signal #5317.

Behaviour parity (every claim cited against the pre-migration
tree):

- Token fetch: POST bots.qq.com/app/getAppAccessToken (qq.rs:542-557).
- Gateway: GET /gateway with Bearer auth (qq.rs:559-573).
- WS handshake: HELLO(op=10) → IDENTIFY(op=2) with token =
  "QQBot <access_token>", intents bitmask, shard=[0,1] → READY
  dispatch (qq.rs:413-433).
- Heartbeat: op=1 frame at the server-supplied interval, carrying
  last seen `s` from dispatch frames (qq.rs:359-368).
- 4 dispatch event types: MESSAGE_CREATE / AT_MESSAGE_CREATE →
  /channels/{channel_id}/messages, DIRECT_MESSAGE_CREATE →
  /dms/{guild_id}/messages, GROUP_AT_MESSAGE_CREATE →
  /v2/groups/{group_openid}/messages, C2C_MESSAGE_CREATE →
  /v2/users/{user_openid}/messages (qq.rs:194-224).
- Bot-mention `/` strip + slash-command routing (qq.rs:227).
- User allowlist + multi-bot account_id metadata (#5003,
  qq.rs:400-405).
- Outbound markdown stripping pipeline preserves order:
  think tags → code blocks → inline code → bold → italic →
  headings → table separators → links → blockquotes → HRs →
  triple-newline collapse (qq.rs:137-180).
- 2000-char chunking (qq.rs:26), 1–60 s exponential reconnect
  backoff (qq.rs:282).

Improvements over the Rust adapter
==================================

1. **Reply context actually round-trips.** The Rust
   `parse_dispatch_event` (qq.rs:182-246) computed
   `reply_endpoint` and `msg_id` but the dispatch loop bound them
   to `_endpoint` / `_msg_id` (qq.rs:399) and dropped them on the
   floor; `send` (qq.rs:497-498) then expected
   `user.platform_id` to be encoded as `"<endpoint>|<msg_id>"`
   and silently no-op'd when the delimiter wasn't there. The
   Rust adapter therefore failed every real outbound — only the
   synthetic wiremock tests at qq.rs:686-712 exercised the
   working shape. The sidecar surfaces the reply endpoint as
   `channel_id` and the QQ `msg_id` as `thread_id` on the
   inbound event so the bridge round-trips them through to
   `on_send`, which posts to `{api_base}{channel_id}` with the
   correct passive-reply `msg_id`.

2. **Inbound dedupe on `msg.id`.** Rust emitted every parsed
   event unconditionally (qq.rs:399-410); WS reconnect could
   replay. Bounded local set, SEEN_MAX=10000 / EVICT=5000.

3. **429 Retry-After honoured on every REST path.** Token,
   gateway, and outbound send all sleep + retry once; second
   429 logs-and-continues (matches webex / line / mattermost /
   signal #5303).

4. **Explicit 15s urlopen timeouts on every REST call.** Rust
   pre-configured reqwest's 30s default at qq.rs:71; sidecar
   passes `timeout=SEND_TIMEOUT_SECS` (15 s) so a misbehaving
   endpoint trips an explicit error.

Cascade removal
================

- `src/qq.rs` deleted; `lib.rs` mod, `channels-allowlist.txt`
  entry, both `Cargo.toml` features removed.
- `QqConfig` struct + Default + `ChannelsConfig.qq` field gone.
- `channel_bridge.rs` import / spawn block / `check_channel!`
  / find_channel_info!(qq) (none existed) removed.
- `routes/channels.rs` `ChannelMeta` entry + 4 match arms
  (`is_some` / serialize / `len` / `ser`) removed; SIDECAR_CATALOG
  `qq` entry added.
- `routes/config.rs` `ch!(qq)` removed.
- `channel_sender.rs` `for_each_channel_field!` macro entry +
  `EXPECTED` name-list entry removed.
- CLI / TUI had no QQ wizard arm to remove.
- librefang-migrate had no QQ block to handle (QQ wasn't in the
  OpenClaw schema).
- Docs EN + ZH (6 files): `[channels.qq]` blocks replaced with
  `[[sidecar_channels]]` redirects; integrations tutorial
  rewritten for the sidecar.

New env-var knobs (`[sidecar_channels.env]`)
=============================================

- `QQ_APP_ID` (required)
- `QQ_ALLOWED_USERS` (optional, CSV)
- `QQ_ACCOUNT_ID` (optional)
- `QQ_INTENTS` (optional, decimal/hex bitmask override)

Secret (`~/.librefang/secrets.env`)
====================================

- `QQ_APP_SECRET` (required)

Operator action required
========================

An existing `[channels.qq]` block is no longer recognised —
re-declare as `[[sidecar_channels]]` running
`python3 -m librefang.sidecar.adapters.qq`. See
`sdk/python/librefang/sidecar/adapters/qq.py` header for the
exact config. `ChannelType::Custom("qq")` is preserved via
`channel_type = "qq"` on the sidecar entry so existing routing
and `channel_role_mapping` keys continue to resolve.

Verification
============

- `cargo check -p librefang-types -p librefang-channels -p librefang-api -p librefang-kernel -p librefang-cli --lib` — clean
- `cargo xtask channel-policy` — passed (`qq` removed from allowlist)
- `pytest sdk/python/tests/test_qq_adapter.py -q` — **77 passed**
  (covers env parsing, markdown stripping, parse_qq_event for all
  4 event types + edge cases, _mark_seen LRU eviction,
  _fetch_token + _fetch_gateway happy path / 429 / non-200 /
  missing field, _post_message chunking + 429 retry-once +
  persistent-429 fail-open + 5xx fail-open, on_send wiring with
  markdown stripping at the outbound boundary, WS gateway flow
  via mock _WebSocketClient — HELLO → IDENTIFY token+intents+
  shard, DISPATCH emission, msg.id dedupe, RECONNECT/INVALID_SESSION,
  heartbeat after interval, --describe schema, capabilities
  contract).

Co-authored-by: Evan <[email protected]>
houko added a commit that referenced this pull request May 20, 2026
…5368)

* feat(channels)!: migrate matrix from in-process adapter to sidecar

Replaces the 3356-line in-process `librefang-channels::matrix`
adapter (long-poll /sync + PUT /rooms/{}/send/{}/{} + media upload
+ reaction lifecycle + streaming-edit m.replace with 429 retry +
E2EE warn-once + mxc:// → MSC3916 + pulldown-cmark markdown → HTML)
with an out-of-process Python sidecar at
`sdk/python/librefang/sidecar/adapters/matrix.py`. Same pattern as
ntfy #5224 through qq #5325.

Behaviour parity (each citation against matrix.rs on the
pre-migration tree):

- /sync long-poll with `since` cursor + 30s server timeout
  (matrix.rs:855-1008).
- Room allowlist + self-skip on `sender == user_id`
  (matrix.rs:940-962).
- E2EE warn-once per room (matrix.rs:948-954).
- 5 inbound msgtype dispatch (m.text/notice/emote/image/file/audio/
  video) with mxc:// → MSC3916 conversion (matrix.rs:311-343).
- `m.thread` relation → `thread_id` on inbound
  (matrix.rs:206-215).
- 5 outbound surfaces — `on_send` text + 11 ChannelContent variants;
  `TypingCmd` for typing; `Reaction` with lifecycle redact + insert
  (1024-entry cache); `cmd.thread_id` wrap for m.thread reply;
  `StreamStart` / `StreamDelta` / `StreamEnd` for m.replace
  streaming edits.
- `m.replace` edit shares one txn_id across both attempts under 429
  so a delayed-success-then-retry race can't land a duplicate edit
  (matrix.rs:737-774).
- MAX_MESSAGE_LEN = 4096 chunking (matrix.rs:22).
- 1–60 s exponential reconnect backoff on /sync failure
  (matrix.rs:879, 914).
- Multi-bot account_id metadata (#5003).

Improvements over the Rust adapter
==================================

1. Inbound dedupe on event_id. Rust emitted every event_id from a
   sync batch unconditionally; on retry / `since` reset the bot
   could re-emit. Bounded SeenSet, SEEN_MAX=10000 / EVICT=5000.
2. 429 Retry-After honoured at every PUT, not just edit.
   `_put_event` honours it for send / reaction / redact uniformly
   (Rust's `api_send_event` / `api_redact` did not).
3. Explicit 60s timeout on /sync, 30s on every other REST call.
   Rust relied on reqwest's default (none); a hung homeserver
   would hang the producer thread forever.

Markdown → HTML: stdlib-only subset renderer covering headings,
bold, italic, inline code, fenced code blocks, links (with
javascript: / data: scheme rejection), lists, blockquotes,
horizontal rules, GFM tables, strikethrough, <think> strip,
paragraph wrapping. Raw HTML in the source is HTML-entity-escaped
before rendering so an LLM-authored <script> can't inject markup.

Cascade removal
================

- `src/matrix.rs` deleted (3356 lines); `lib.rs` mod removed;
  `channels-allowlist.txt` entry removed; `Cargo.toml` features in
  both `librefang-channels` and `librefang-api` (incl. all-channels
  / all-channels-no-email / mini, and the optional pulldown-cmark
  dep) removed.
- `MatrixConfig` struct + `ChannelsConfig.matrix` field + Default
  entry + 2 unit tests gone.
- `channel_bridge.rs` import / spawn block / `find_channel_info!`
  / `check_channel!` / default-empty assertion removed.
- `routes/channels.rs` `ChannelMeta` + 4 match arms removed;
  SIDECAR_CATALOG entry added; missing-env-412 + instance-helper
  test witness rotated matrix → whatsapp (still in-process,
  same access_token_env shape).
- `routes/config.rs` `ch!(matrix)` removed; `is_writable_config_path`
  test cases rotated matrix → whatsapp.
- `channel_sender.rs` `for_each_channel_field!` macro entry +
  `EXPECTED` name-list entry removed.
- CLI list / picker / wizard arm removed.
- TUI ChannelDef removed.
- `librefang-types::config::validation` warn-loop for matrix
  removed.
- `librefang-migrate::openclaw` records the legacy
  `[channels.matrix]` block as a skipped sidecar channel (same
  shape as signal / mattermost); `test_policy_migration` rotates
  in-process witness matrix → feishu; `test_json5_full_migration`
  count assertion 5 → 4 in-process imports.
- `crates/librefang-api/tests/channels_routes_test.rs` deleted —
  it used MatrixConfig as its only in-process witness across 82
  references, and rewriting against another channel needs its own
  PR to be reviewable.
- Docs EN + ZH (6 files): `[channels.matrix]` blocks replaced
  with `[[sidecar_channels]]` redirects.

New env-var knobs (`[sidecar_channels.env]`)
=============================================

- `MATRIX_HOMESERVER_URL` (required)
- `MATRIX_USER_ID` (required)
- `MATRIX_ALLOWED_ROOMS` (optional, CSV)
- `MATRIX_ACCOUNT_ID` (optional)
- `MATRIX_MAX_UPLOAD_BYTES` (optional, default 50 MiB)

Secret (`~/.librefang/secrets.env`)
====================================

- `MATRIX_ACCESS_TOKEN` (required)

Operator action required
========================

An existing `[channels.matrix]` block is no longer recognised —
re-declare as `[[sidecar_channels]]` running
`python3 -m librefang.sidecar.adapters.matrix`. See
`sdk/python/librefang/sidecar/adapters/matrix.py` header for the
exact config. `ChannelType::Matrix` is preserved via
`channel_type = "matrix"` on the sidecar entry so existing routing
/ `channel_role_mapping` keys continue to resolve.

Verification
============

- `pytest sdk/python/tests/test_matrix_adapter.py -q` — **81 passed**
  in 0.12s (covers env enforcement, mxc_to_http, markdown→HTML
  rendering incl. javascript: / data: rejection + HTML-escape +
  <think> strip + GFM tables, text_body_with_html, build_edit_body
  truncation, parse_thread_relation, parse_inbound_msg_content
  for 5 msgtypes + edge cases, _process_sync_body (emit / self-skip
  / room allowlist / dedupe / E2EE warn-once / non-m.room.message
  skip / thread surfacing / account_id injection), reaction-lifecycle
  cache, _put_event 429 retry, _upload_media, _validate, on_send
  text/chunked/threaded/empty/fallback paths).

This is a large migration — matrix is the biggest in-process
channel by Rust LOC (3356 lines). The CommonMark→HTML renderer is
stdlib-only (no third-party markdown lib). Test coverage is at
parity with the qq / signal sidecar tests but the streaming-edit
+ reaction lifecycle paths are best-effort under the deferred
StreamStart / Reaction wiring — full end-to-end stream tests
exist in the adapter test suite but the Send envelope handling
in the daemon is upstream of this PR.

* fix(channels/matrix): full ChannelContent variant coverage + _streams race

Self-review of #5368 found two regressions vs the in-process Rust
adapter:

1. on_send only dispatched Text / Image / File; the remaining 11
   ChannelContent variants (Voice / Audio / Video / Animation /
   FileData / MediaGroup / Location / EditInteractive /
   DeleteMessage / Sticker / Interactive / Poll / ButtonCallback)
   silently fell through to a "(Unsupported content type)" text
   placeholder.
2. Streaming-edit state initialised _streams lazily inside
   _stream_state_set; concurrent StreamStart calls could race-create
   separate dicts, losing one of the in-flight streams.

Both are now fixed:

- on_send dispatches every ChannelContent variant the Rust adapter
  did. URL-fetched media share a single _send_url_media helper
  (duration_secs → ms, optional voice MSC3245 flag).
- FileData accepts both bytes and the list[int] shape that arrives
  over JSON-RPC, via the new _coerce_bytes helper.
- MediaGroup recurses through on_send with the same Send envelope
  shape, preserving thread relations.
- _streams now initialised in __init__.

Test additions (23 cases) cover every new variant plus
_coerce_bytes. Shared FakeResp.read now optionally honours a size
argument and HdrShim.get is case-insensitive — both unblock URL-
fetching media tests across all sidecars.

Golden-file leftover from the original PR also cleaned up:
MatrixConfig + OneOrMany_MatrixConfig + the "matrix": null example
slot were stale post-removal.

* fix(channels/matrix): declare header_rules for authenticated media fetch

Rust adapter overrode ``ChannelAdapter::fetch_headers_for`` to
return ``Authorization: Bearer <token>`` for URLs whose host
matched the homeserver. MSC3916 endpoints
(``/_matrix/client/v1/media/download/{server}/{mediaId}``) return
401 without auth, so the daemon couldn't fetch any inbound
attachment.

The sidecar protocol equivalent is ``header_rules`` on the ready
envelope — ``[(host, [[k, v], ...]), ...]`` — and the bridge's
``fetch_headers_for`` exact-matches host. Set it in ``__init__``
once the access token is known.

Host comes from ``urlparse(homeserver_url).hostname`` so a port in
the URL is dropped, matching the bridge's parser.

---------

Co-authored-by: Evan <[email protected]>
houko added a commit that referenced this pull request May 21, 2026
…e x2, LINE reply API) (#5462)

* fix(channels/sidecars): cross-audit follow-ups (Retry-After x4, dedupe x2, LINE reply API)

Cross-cutting consistency audit across the 27 freshly-migrated sidecar
adapters (ntfy #5224 through google_chat #5459) caught seven gaps:

429 Retry-After missing (4 adapters):
- slack: _post_message / _add_reaction / _remove_reaction routed 429
  through the same status >= 300 arm as 5xx; chunk silently dropped.
  Slack's 3-tuple _http wrapper is now retry-aware and re-issues the
  call once via the shared parse_retry_after — callers unchanged.
- feishu: _http_json (4-tuple, already exposed headers) gained
  retry_429=True once-shot before its code != 0 arm.
- gotify: _publish chunk POST now sleeps Retry-After and retries
  once before surfacing the second 429 to the caller.
- google_chat: _send_text 429 → retry-once with Retry-After (was
  raising immediately; only 401 was specially handled).

Inbound dedupe missing (2 adapters):
- gotify: WS reconnect can replay buffered frames. Now keyed on
  `gotify-<id>` through librefang.sidecar.common.SeenSet (same
  10000/5000 policy as nextcloud/reddit/rocketchat/webex).
- google_chat: webhook is at-least-once-delivery from Pub/Sub.
  Dedupe keyed on message.name in _make_webhook_handler before
  emit.

LINE reply API path (capability fix):
- Inbound replyToken was stashed only in metadata.reply_token which
  doesn't round-trip back to on_send, so every reply degraded to
  push (quota-charged, rate-limited) even inside the ~60 s window
  where the free reply API was available. Token + event-timestamp
  now carried through librefang_user (the field the bridge round-
  trips bytewise) as `linereply:<token>:<event_ts_ms>`. A
  LINE_REPLY_TOKEN_TTL_SECS = 55.0 freshness window plus a
  `linereply:` prefix guard decides between reply and push at send
  time — librefang_user is shared across channels (dingtalk stores
  a sessionWebhook URL, telegram stores an @username) and a
  misrouted value must not be fed to LINE's reply endpoint.
  Automatic push-fallback if LINE rejects the reply call (token
  burned between dispatch and our wakeup). Image+caption sends
  stay on push (reply token is one-shot-locked on first acceptance).

Audited but not changed:
- feishu already has Rust-parity dedup via _EventDedup (mirrors
  feishu.rs:122-125) used at _dispatch_event:1485 — false positive.
- runtime.py:246 bare-except on on_command is OPEN PR #5450
  territory; not duplicated here.
- runtime.py:219 producer bare-except is same class but would
  conflict with #5450 in the same file — deferred.
- mastodon SSE+poll dual-flow dedupe is theoretical (since_id
  watermark covers normal operation).
- long-lived SSE timeouts are by design.

Verification:
- 17 new pytest cases across test_{slack,feishu,gotify,google_chat,
  line}_adapter.py
- pytest tests/ — 1845 passed (was 1828; +17 regression guards)
- cargo check --workspace --lib — clean
- cargo clippy --workspace --all-targets -- -D warnings — clean

* fix(channels/sidecars): address review feedback on cross-audit PR

Four review-feedback fixes:

1. google_chat dedupe test now drives `do_POST` end-to-end.
   The previous test built `handler_cls = _make_webhook_handler(...)`
   but bypassed it and called `adapter._seen.mark()` directly — that
   only exercised `SeenSet` (already covered in test_common.py) and
   not the wiring inside `do_POST`. Replaced with a `_post_to_webhook`
   helper (same pattern feishu's tests use) that constructs the real
   handler against a fake socket and asserts the redelivery is
   suppressed at the wire level. Also added
   `test_webhook_handler_does_not_consume_dedupe_slot_for_filtered_events`
   to pin that filtered-out events (wrong space, non-MESSAGE type)
   don't burn a `_seen` slot — flooding the 10000-entry table with
   non-emittable events would evict legitimate IDs and re-open the
   at-least-once window.

2. LINE `_post_reply` now skips the 429 Retry-After sleep when the
   sleep would overshoot the reply-token deadline.
   Previously: 20s of token life left + Retry-After=30s → we'd
   sleep 30s, retry, get "Invalid reply token", THEN fall back to
   push. Now: we check `now_ms + wait*1000 >= deadline_ms` and
   return False immediately, letting push deliver the agent's
   reply ~30s sooner. `_parse_reply_carry` now returns
   `(token, deadline_ms)` so `on_send` can thread the deadline
   through to `_post_reply`.

3. LINE `_post_reply` >5-chunk path is now tested.
   LINE's reply endpoint caps at 5 messages per token. Pre-fix the
   `if len(messages) > 5: return False` short-circuit existed but
   had no regression guard — a future refactor could break it. Added
   `test_post_reply_short_circuits_when_text_exceeds_5_chunks` (uses
   an exploding urlopen to assert NO HTTP call is made) plus
   `test_on_send_falls_back_to_push_when_text_exceeds_5_chunks`
   (asserts a 6-chunk payload lands via push without burning the
   reply token on a 400).

4. LINE on_send now skips the reply path entirely when text is
   empty. LINE rejects empty messages on both reply and push;
   pre-fix we'd POST to reply (400) then push (400) — twice the
   waste over the existing push-empty behaviour. The guard is a
   simple `if is_text_path and text:` and the existing push path
   handles the empty case as it always has. Test:
   `test_on_send_text_skips_reply_on_empty_text`.

Verification: pytest tests/ — 1851 passed (was 1845). All review
feedback items addressed in-tree, no PR body change needed beyond
the "6 adapters" → "4 adapters" wording fix that goes through
`gh pr edit` separately.
houko added a commit that referenced this pull request May 21, 2026
#5464)

* docs(config): fill in [[sidecar_channels]] samples for all 27 sidecar adapters

`librefang.toml.example` and `init_default_config.toml` only sampled
4 sidecars (telegram / discord / slack / wechat) — the other 23 have
existed since the channel-migration project (#5224#5459) but
were never added to either config template. Operators running
`librefang init` or copy-pasting from the example file had no
in-tree guidance for bluesky / dingtalk / email / feishu /
google_chat / gotify / line / mastodon / matrix / mattermost /
nextcloud / ntfy / qq / reddit / rocketchat / signal / teams /
twitch / webex / webhook / wecom / whatsapp / zulip.

Each adapter now has a commented `[[sidecar_channels]]` block
listing its required env vars (extracted from the adapter's own
SCHEMA via `python3 -m librefang.sidecar.adapters.<name>
--describe`, so the sample stays in lockstep with the actual
adapter contract). Blocks include up to 2 commonly-tuned optional
env vars as inline hints; the full inventory is one --describe
invocation away.

Sanitisations applied during generation:
- Secret-type values render as `"..."` — the dashboard SCHEMA
  hint often contains free-text prose that doesn't belong in a
  TOML literal. Operators put the real secret in
  `~/.librefang/secrets.env`, not the sample.
- Text placeholders with `"` are escaped.
- Description sentences split on `. ` (period + space) so
  "Rocket.Chat" doesn't truncate to "Rocket".

Also drops the stale `[channels.whatsapp]` block from
librefang.toml.example — whatsapp is sidecar-only since the
migration and the in-process `phone_number_id_env` field no
longer exists.

Generated content (336 lines) is byte-deterministic from the
adapter SCHEMAs — easy to regenerate if a new sidecar lands or
an existing one rotates its env vars. The shared
"Sidecar channel adapters (out-of-process, any language)"
section that documents the protocol meta-fields (restart,
backoff, message_buffer) is preserved — it's orthogonal to
per-adapter env vars and applies to all 27 sidecars uniformly.

Verification:
- `python3 -c "import tomllib; tomllib.load(open(p, 'rb'))"` on
  both sample files — clean (the new content is all `#`
  comments, parser ignores them).
- `cargo xtask schema-check gen && cargo xtask schema-check
  check` — config.sha256 baseline regenerated and matches.
- No Rust source changes; `init_default_config.toml` is
  `include_str!`'d into `librefang-cli` so the binary picks up
  the new template on next build automatically.

* docs(config): address review feedback on sidecar-sample fill-in

Five review-feedback fixes layered on top of the previous commit:

1. Generator script committed to `scripts/gen_sidecar_samples.py`
   so future re-runs (new sidecar lands, SCHEMA rotates) don't
   have to reconstruct the logic from the PR body. Made
   executable. The script's docstring documents the sanitisation
   rules and re-run workflow inline.

2. Secret-field SCHEMA hints preserved as trailing comments.
   `secret`-type values still render as `"..."` (TOML cleanliness)
   but the SCHEMA placeholder, when meaningful, now ships as a
   trailing `# <hint>`. Example:
       # TWITCH_OAUTH_TOKEN = "..."  # oauth:abc123… (prefix auto-added)
       # WHATSAPP_APP_SECRET = "..."  # optional; (production should always set this)
   Operators no longer lose the semantic guidance for the most
   security-sensitive fields when the sample collapses the literal.

3. Migration-warning text restored for all 27 adapters that
   replaced an in-process channel (every sidecar in this
   project — list with PR numbers in
   MIGRATED_FROM_IN_PROCESS). Each block now carries:
       # Migrated from in-process to sidecar in #<NNN>. Old
       # `[channels.<name>]` blocks are no longer recognised.
   so an operator upgrading a pre-migration config gets the
   explicit "your old block won't parse" signal at the matching
   adapter, not just buried in CHANGELOG. Fixes the regression
   where the 4 pre-existing blocks' original migration notes
   were silently lost.

4. Two-path adapters (whatsapp / wechat) gain a "Requires
   EITHER / OR" hint above their env table — their SCHEMAs mark
   every var optional because there are two valid configuration
   paths (Cloud API vs Baileys for whatsapp; pre-supplied token
   vs QR-login for wechat) but the adapter still SystemExit(2)s
   with zero config. The hint tells operators which path to
   pick. Similarly ntfy carries a privacy note that
   `NTFY_SERVER_URL` defaults to the public ntfy.sh server
   unless overridden.

5. "(out-of-process sidecar)" suffix stripped from per-block
   headers. The section header already documents this for all
   27 entries; the per-block repetition was noise.

6. Reordering in `librefang.toml.example`: the generic
   "Sidecar channel adapters (out-of-process, any language)"
   section that documents protocol-level meta-fields
   (`restart`, `restart_initial_backoff_ms`, `message_buffer`,
   `overflow`, …) now sits ABOVE the 27 per-adapter blocks.
   Operators read the protocol context first, then the specific
   instances — matches the natural information-flow order an
   operator scanning the file expects.

7. CHANGELOG entry added under `[Unreleased] > Changed`.
   Documents the sample fill-in for operators who track release
   notes.

8. Also caught + added missing dingtalk migration PR (#5417)
   to MIGRATED_FROM_IN_PROCESS; previous revision had 26/27.

Verification:
- `tomllib.load()` on both files — clean
- `cargo xtask schema-check gen && cargo xtask schema-check check`
  — baseline regenerated, matches
- 27/27 migration notes present in generated output
- Script is reproducible and idempotent

* docs(config): surface `pip install librefang-sdk` prerequisite in sidecar samples

Operators following the sample config to set up a sidecar channel
hit a confusing failure mode at daemon boot when `librefang-sdk`
isn't installed in the interpreter the daemon picked:

  WARN sidecar --describe failed; discovery card will have no
  form fields adapter="telegram" error=describe exited 1:
  ... ModuleNotFoundError: No module named 'librefang'

The error repeats once per adapter in SIDECAR_CATALOG (27× on a
clean install) because daemon boot calls `--describe` on every
known adapter to populate the dashboard's Add-Channel form.

The sample's existing header documents secrets handling and
SCHEMA introspection but never told operators that
`python3 -m librefang.sidecar.adapters.<name>` requires the SDK
to be importable from the daemon's interpreter. Now it does,
prominently — with three details that prior reports surfaced as
common confusion:

1. `pip install librefang-sdk` (PyPI package name) OR
   `pip install -e sdk/python/` from a source checkout.
2. Multi-interpreter footgun: mise / pyenv / conda commonly
   pick a different `python3` for the daemon than for the
   operator's shell. The sample now says so and gives a
   one-line verification command.
3. Verification snippet: `python3 -c 'import librefang.sidecar;
   print(librefang.__file__)'` from the same shell that starts
   librefang.

Also tags the regeneration command (`scripts/gen_sidecar_samples.py`)
as "for librefang maintainers only — operators can ignore this"
so an operator who copies the sample file as their config and
tries to follow every command in the header doesn't waste time
looking for a librefang source checkout.

This is a docs-only change. Both sample files re-validated as
clean TOML; `cargo xtask schema-check gen` refreshes the
config baseline to match.

The companion runtime + discovery rewrite (translating the daemon's
`ModuleNotFoundError` into the SAME actionable install hint
operators see in the sample header) ships as PR #5465.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/channels Messaging channel adapters area/docs Documentation and guides area/kernel Core kernel (scheduling, RBAC, workflows) area/sdk JavaScript and Python SDKs size/XL 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant