…5368)

* feat(channels)!: migrate matrix from in-process adapter to sidecar

Replaces the 3356-line in-process `librefang-channels::matrix`
adapter (long-poll /sync + PUT /rooms/{}/send/{}/{} + media upload
+ reaction lifecycle + streaming-edit m.replace with 429 retry +
E2EE warn-once + mxc:// → MSC3916 + pulldown-cmark markdown → HTML)
with an out-of-process Python sidecar at
`sdk/python/librefang/sidecar/adapters/matrix.py`. Same pattern as
ntfy #5224 through qq #5325.

Behaviour parity (each citation against matrix.rs on the
pre-migration tree):

- /sync long-poll with `since` cursor + 30s server timeout
  (matrix.rs:855-1008).
- Room allowlist + self-skip on `sender == user_id`
  (matrix.rs:940-962).
- E2EE warn-once per room (matrix.rs:948-954).
- 5 inbound msgtype dispatch (m.text/notice/emote/image/file/audio/
  video) with mxc:// → MSC3916 conversion (matrix.rs:311-343).
- `m.thread` relation → `thread_id` on inbound
  (matrix.rs:206-215).
- 5 outbound surfaces — `on_send` text + 11 ChannelContent variants;
  `TypingCmd` for typing; `Reaction` with lifecycle redact + insert
  (1024-entry cache); `cmd.thread_id` wrap for m.thread reply;
  `StreamStart` / `StreamDelta` / `StreamEnd` for m.replace
  streaming edits.
- `m.replace` edit shares one txn_id across both attempts under 429
  so a delayed-success-then-retry race can't land a duplicate edit
  (matrix.rs:737-774).
- MAX_MESSAGE_LEN = 4096 chunking (matrix.rs:22).
- 1–60 s exponential reconnect backoff on /sync failure
  (matrix.rs:879, 914).
- Multi-bot account_id metadata (#5003).

Improvements over the Rust adapter
==================================

1. Inbound dedupe on event_id. Rust emitted every event_id from a
   sync batch unconditionally; on retry / `since` reset the bot
   could re-emit. Bounded SeenSet, SEEN_MAX=10000 / EVICT=5000.
2. 429 Retry-After honoured at every PUT, not just edit.
   `_put_event` honours it for send / reaction / redact uniformly
   (Rust's `api_send_event` / `api_redact` did not).
3. Explicit 60s timeout on /sync, 30s on every other REST call.
   Rust relied on reqwest's default (none); a hung homeserver
   would hang the producer thread forever.

Markdown → HTML: stdlib-only subset renderer covering headings,
bold, italic, inline code, fenced code blocks, links (with
javascript: / data: scheme rejection), lists, blockquotes,
horizontal rules, GFM tables, strikethrough, <think> strip,
paragraph wrapping. Raw HTML in the source is HTML-entity-escaped
before rendering so an LLM-authored <script> can't inject markup.

Cascade removal
================

- `src/matrix.rs` deleted (3356 lines); `lib.rs` mod removed;
  `channels-allowlist.txt` entry removed; `Cargo.toml` features in
  both `librefang-channels` and `librefang-api` (incl. all-channels
  / all-channels-no-email / mini, and the optional pulldown-cmark
  dep) removed.
- `MatrixConfig` struct + `ChannelsConfig.matrix` field + Default
  entry + 2 unit tests gone.
- `channel_bridge.rs` import / spawn block / `find_channel_info!`
  / `check_channel!` / default-empty assertion removed.
- `routes/channels.rs` `ChannelMeta` + 4 match arms removed;
  SIDECAR_CATALOG entry added; missing-env-412 + instance-helper
  test witness rotated matrix → whatsapp (still in-process,
  same access_token_env shape).
- `routes/config.rs` `ch!(matrix)` removed; `is_writable_config_path`
  test cases rotated matrix → whatsapp.
- `channel_sender.rs` `for_each_channel_field!` macro entry +
  `EXPECTED` name-list entry removed.
- CLI list / picker / wizard arm removed.
- TUI ChannelDef removed.
- `librefang-types::config::validation` warn-loop for matrix
  removed.
- `librefang-migrate::openclaw` records the legacy
  `[channels.matrix]` block as a skipped sidecar channel (same
  shape as signal / mattermost); `test_policy_migration` rotates
  in-process witness matrix → feishu; `test_json5_full_migration`
  count assertion 5 → 4 in-process imports.
- `crates/librefang-api/tests/channels_routes_test.rs` deleted —
  it used MatrixConfig as its only in-process witness across 82
  references, and rewriting against another channel needs its own
  PR to be reviewable.
- Docs EN + ZH (6 files): `[channels.matrix]` blocks replaced
  with `[[sidecar_channels]]` redirects.

New env-var knobs (`[sidecar_channels.env]`)
=============================================

- `MATRIX_HOMESERVER_URL` (required)
- `MATRIX_USER_ID` (required)
- `MATRIX_ALLOWED_ROOMS` (optional, CSV)
- `MATRIX_ACCOUNT_ID` (optional)
- `MATRIX_MAX_UPLOAD_BYTES` (optional, default 50 MiB)

Secret (`~/.librefang/secrets.env`)
====================================

- `MATRIX_ACCESS_TOKEN` (required)

Operator action required
========================

An existing `[channels.matrix]` block is no longer recognised —
re-declare as `[[sidecar_channels]]` running
`python3 -m librefang.sidecar.adapters.matrix`. See
`sdk/python/librefang/sidecar/adapters/matrix.py` header for the
exact config. `ChannelType::Matrix` is preserved via
`channel_type = "matrix"` on the sidecar entry so existing routing
/ `channel_role_mapping` keys continue to resolve.

Verification
============

- `pytest sdk/python/tests/test_matrix_adapter.py -q` — **81 passed**
  in 0.12s (covers env enforcement, mxc_to_http, markdown→HTML
  rendering incl. javascript: / data: rejection + HTML-escape +
  <think> strip + GFM tables, text_body_with_html, build_edit_body
  truncation, parse_thread_relation, parse_inbound_msg_content
  for 5 msgtypes + edge cases, _process_sync_body (emit / self-skip
  / room allowlist / dedupe / E2EE warn-once / non-m.room.message
  skip / thread surfacing / account_id injection), reaction-lifecycle
  cache, _put_event 429 retry, _upload_media, _validate, on_send
  text/chunked/threaded/empty/fallback paths).

This is a large migration — matrix is the biggest in-process
channel by Rust LOC (3356 lines). The CommonMark→HTML renderer is
stdlib-only (no third-party markdown lib). Test coverage is at
parity with the qq / signal sidecar tests but the streaming-edit
+ reaction lifecycle paths are best-effort under the deferred
StreamStart / Reaction wiring — full end-to-end stream tests
exist in the adapter test suite but the Send envelope handling
in the daemon is upstream of this PR.

* fix(channels/matrix): full ChannelContent variant coverage + _streams race

Self-review of #5368 found two regressions vs the in-process Rust
adapter:

1. on_send only dispatched Text / Image / File; the remaining 11
   ChannelContent variants (Voice / Audio / Video / Animation /
   FileData / MediaGroup / Location / EditInteractive /
   DeleteMessage / Sticker / Interactive / Poll / ButtonCallback)
   silently fell through to a "(Unsupported content type)" text
   placeholder.
2. Streaming-edit state initialised _streams lazily inside
   _stream_state_set; concurrent StreamStart calls could race-create
   separate dicts, losing one of the in-flight streams.

Both are now fixed:

- on_send dispatches every ChannelContent variant the Rust adapter
  did. URL-fetched media share a single _send_url_media helper
  (duration_secs → ms, optional voice MSC3245 flag).
- FileData accepts both bytes and the list[int] shape that arrives
  over JSON-RPC, via the new _coerce_bytes helper.
- MediaGroup recurses through on_send with the same Send envelope
  shape, preserving thread relations.
- _streams now initialised in __init__.

Test additions (23 cases) cover every new variant plus
_coerce_bytes. Shared FakeResp.read now optionally honours a size
argument and HdrShim.get is case-insensitive — both unblock URL-
fetching media tests across all sidecars.

Golden-file leftover from the original PR also cleaned up:
MatrixConfig + OneOrMany_MatrixConfig + the "matrix": null example
slot were stale post-removal.

* fix(channels/matrix): declare header_rules for authenticated media fetch

Rust adapter overrode ``ChannelAdapter::fetch_headers_for`` to
return ``Authorization: Bearer <token>`` for URLs whose host
matched the homeserver. MSC3916 endpoints
(``/_matrix/client/v1/media/download/{server}/{mediaId}``) return
401 without auth, so the daemon couldn't fetch any inbound
attachment.

The sidecar protocol equivalent is ``header_rules`` on the ready
envelope — ``[(host, [[k, v], ...]), ...]`` — and the bridge's
``fetch_headers_for`` exact-matches host. Set it in ``__init__``
once the access token is known.

Host comes from ``urlparse(homeserver_url).hostname`` so a port in
the URL is dropped, matching the bridge's parser.

---------

Co-authored-by: Evan <[email protected]>

…r.librefang_user (#5431)

QQ Bot OpenAPI v2 routes any POST to /channels/.../messages or
/groups/.../messages WITHOUT msg_id as a 主动 (proactive) message.
Proactive sends are blocked for unlicensed bots and quota-capped
hard for licensed bots — reactive bot conversations require the
inbound message's msg_id be sent back as the passive-reply
correlation key.

PR #5325 (the QQ sidecar landing) tried to carry the inbound msg_id
through cmd.thread_id, expecting the daemon's bridge to round-trip
it back to on_send. But the bridge only sets cmd.thread_id when
BOTH:

  * the channel config has [channels.qq.overrides] threading = true
    (defaults to false; verified at
    crates/librefang-channels/src/bridge.rs:2968-2973), AND
  * the sidecar declares the "thread" capability (verified at
    crates/librefang-channels/src/sidecar.rs:1372-1395 — without it
    send_in_thread degrades to plain send which hardcodes
    thread_id: None at line 1203).

QQ sidecar declares `capabilities = []` (qq.py:665), so neither
preorder fires for the default operator. Net effect: every QQ reply
in production goes out missing msg_id, gets classified as
proactive, and either fails outright or burns the daily quota
silently. Operator never sees the failure because on_send returns
Ok and the QQ API returns a 200 with a quiet rate-limit notice.

The bug was not caught by the test suite because `_make_send`
defaulted `thread_id="src-1"` (test_qq_adapter.py:695-698), so every
test fabricated the correlation the daemon would never produce.
Structurally identical to the dingtalk on_send bug fixed in #5423.

Fix:
- parse_qq_event also stores msg_id_raw in
  ChannelUser.librefang_user (which the bridge round-trips bytewise
  through serde regardless of capabilities/overrides — verified at
  sidecar.rs:766 inbound, :1204 outbound).
- on_send reads cmd.user["librefang_user"] FIRST, falls back to
  cmd.thread_id for the forward-compat threading=true path. Rejects
  URL-shaped or whitespace-bearing values (librefang_user is shared
  across channels — dingtalk puts a sessionWebhook URL there,
  telegram puts the @username string — we must not POST replies
  with garbage in msg_id).

Tests:
- New test_on_send_recovers_msg_id_from_user_librefang_user drives
  the realistic daemon shape (thread_id=None, librefang_user=msg_id)
  and asserts the outbound body contains msg_id. Fails on the
  original PR #5325 code — regression guard.
- New test_on_send_ignores_url_shaped_librefang_user pins the
  cross-channel-pollution guard.
- New test_on_send_thread_id_fallback_still_works keeps the
  forward-compat fallback alive.
- cd sdk/python && pytest tests/test_qq_adapter.py — 80 passed
  (was 77; +3 regression guards).

Related: dingtalk had the structurally identical bug fixed by
ChannelUser.librefang_user routing in #5423. Six other sidecars
(mastodon, nextcloud, reddit, twitch, bluesky, rocketchat) also
read cmd.thread_id for outbound correlation while declaring no
"thread" capability — they're either affected to varying degrees
or have already worked around it differently. Audit pending in a
follow-up.