Skip to content

feat(sidecar,sdk): telegram full sidecar parity (formatter + full inbound/outbound), stdlib-only#5232

Merged
houko merged 3 commits into
mainfrom
fix/telegram-sidecar-stdlib
May 18, 2026
Merged

feat(sidecar,sdk): telegram full sidecar parity (formatter + full inbound/outbound), stdlib-only#5232
houko merged 3 commits into
mainfrom
fix/telegram-sidecar-stdlib

Conversation

@houko

@houko houko commented May 18, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #5228. Full sidecar parity for telegram so the in-process
crates/librefang-channels/src/telegram.rs can be removed. Three
commits, one tight cluster:

  1. daaa2712 — kill the requests dependency (the fix(sidecar,sdk): move first-party channel adapters out of examples into librefang-sdk #5228 wart).
  2. 134ef485 — promote to SidecarAdapter with capability
    negotiation (typing/reaction/interactive/thread/streaming).
  3. 987fd31afull parity: faithful, function-by-function ports
    of the audited Rust (not re-derived).

What 987fd31a ports (with the Rust oracle it is pinned to)

  • Markdown → Telegram-HTML formatter — byte-exact port of
    formatter::markdown_to_telegram_html (+ inline/fence/heading/list
    helpers). Tests assert the same input→output pairs as
    crate::formatter's own mod tests, so the two implementations
    cannot drift silently. Outbound text now formats + sanitizes +
    chunks and sends parse_mode=HTML (was a naive
    parse_mode=Markdown), with the same plain-text retry on Telegram's
    "can't parse entities" 400.
  • sanitize_telegram_html security pass — tag + <a href>
    scheme allowlist, attribute-injection escaping, unclosed-tag
    balancing (rebuild_safe_tag / parse_attrs / is_safe_href),
    including the exact Rust edge where a dropped unsafe-<a> leaves
    its now-unmatched </a> escaped.
  • message_truncator chunkerutf16_len /
    truncate_to_utf16_limit / split_to_utf16_chunks with the
    HTML-entity boundary guard and the zero-progress guards.
  • Full inbound — text/bot-command, photo, document, audio, voice,
    animation, video, video_note, location, sticker; from /
    sender_chat extraction; callback_queryButtonCallback
    (+ fire-and-forget answerCallbackQuery); edited_message;
    reply-to context; getFile URL resolution with text fallback;
    ALLOWED_USERS by id and username.
  • Full outbound ChannelContent dispatch for every variant
    (Image→sendPhoto, File→sendDocument/sendVoice, Voice/Video/Audio/
    Animation, Sticker, Location, MediaGroup, Poll, Interactive,
    EditInteractive, DeleteMessage), private-URL → multipart upload,
    OGG/Opus voice routing, 429 retry_after retry.

Scope boundary

  • Removing telegram.rs is NOT in this PR. That deletion is a
    breaking, kernel/config/api-cascading change (cf. the ntfy feat(channels)!: migrate ntfy from in-process adapter to sidecar (P7) #5224
    cascade) and is intentionally a separate reviewed PR; this PR makes
    the sidecar a faithful functional replacement so that removal is
    unblocked.
  • No pyproject.toml change; stdlib-only preserved (no requests).
    No Rust touched.

Verification

  • full Python SDK suite (cd sdk/python && pytest) — 43 pass:
    formatter byte-exact vs the Rust oracle; sanitizer (incl. unsafe-href
    / unclosed / unknown-tag / idempotent); UTF-16 + HTML-entity
    chunking; inbound (text/command/media/getFile-fallback/callback/
    edited/reply/allowed-by-id+username); outbound dispatch for every
    variant; plain-text 400 fallback; streaming; reaction map;
    • cross-impl protocol conformance + ntfy/webhook/runtime (no
      regression)

…ests)

Follow-up to #5228. After the first-party adapters moved into
librefang-sdk, `librefang.sidecar.adapters.telegram` shipped as a
hand-rolled stdin/stdout script that `import requests` — a dep never
declared in sdk/python/pyproject.toml (dependencies = [], no extra),
so `pip install librefang-sdk` then `python -m
librefang.sidecar.adapters.telegram` died with
ModuleNotFoundError: requests.

Rewritten as a `SidecarAdapter` subclass on the librefang.sidecar SDK,
identical in shape to the ntfy adapter, using `urllib`. The whole
`librefang.sidecar.adapters` package is now zero-dependency
(ntfy / webhook were already stdlib-only).

Transport behaviour preserved: long-poll getUpdates (server 30s /
client 35s / offset), ALLOWED_USERS numeric whitelist, sender =
first_name / username / "unknown", channel_id = chat id,
platform = "telegram", outbound sendMessage parse_mode Markdown.
Intentional SDK-consistent alignments vs the old raw script (matching
ntfy): missing TELEGRAM_BOT_TOKEN exits (supervisor reports it) rather
than emitting an error event; transport/API/send failures log + drive
the SDK's supervised backoff instead of hand-emitted protocol error
events; ready handshake is the SDK's idempotent ready/ready_ack loop.
Operator config is unchanged.

No pyproject change — the fix removes the need for `requests` entirely
rather than adding a `[telegram]` extra (keeps the adapters package
uniformly zero-dep). No Rust touched; no golden change (telegram is
not in the kernel-config schema).

Verification (clean venv, NO requests installed):
- `python3 -c "import librefang.sidecar.adapters.telegram"` — OK
- `cd sdk/python && pytest` — 32 passed: new tests/test_telegram_adapter.py
  (stdlib-only assertion; `_update_to_event` text/sender/whitelist/
  non-text-skip; `_api_get` ok / HTTPError-body / non-JSON /
  timeout-reraise; `_poll_once` emit+offset+API-error; `on_send`
  text/unsupported/skip), plus existing ntfy + SDK suites green
  (no regression).
@github-actions github-actions Bot added area/docs Documentation and guides area/sdk JavaScript and Python SDKs size/L 250-999 lines changed labels May 18, 2026
…ment 1/N)

Promote the stdlib-only telegram adapter from a plain text relay onto
SidecarAdapter with capability negotiation, matching the in-process
crates/librefang-channels/src/telegram.rs feature surface so the kernel
routes rich commands here instead of degrading to text.

DONE this increment:
* capabilities = [typing, reaction, interactive, thread, streaming]
* reaction emoji map identical to the Rust adapter (map_reaction_emoji),
  incl. optional clear-on-done
* inline keyboards; forum-topic message_thread_id; throttled
  editMessageText streaming with UTF-16-aware 4096 chunking
  (_chunks16, the stdlib equivalent of split_to_utf16_chunks)
* inbound text long-poll, ALLOWED_USERS whitelist (unchanged)

NOT YET (gates removal of the in-process adapter, tracked):
* faithful Markdown -> Telegram-HTML formatter subsystem (this
  increment sends parse_mode=Markdown like the pre-migration adapter,
  not the Rust formatter::TelegramHtml pipeline)
* full inbound parsing of non-text updates into ChannelContent variants

Stdlib-only preserved (no requests). 37 SDK tests pass.
@houko houko changed the title fix(sidecar,sdk): telegram adapter onto the SDK, stdlib-only (no requests) feat(sidecar,sdk): telegram → SidecarAdapter parity, stdlib-only (increment 1/N) May 18, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 134ef485a2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +217 to +219
content=protocol.Content.text(msg["text"]),
channel_id=str(msg.get("chat", {}).get("id", "")),
platform="telegram",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve Telegram message ids for reactions

When a normal Telegram update is converted to a sidecar message, this omits msg["message_id"] even though the adapter now declares the reaction capability. The sidecar reader falls back to a generated UUID when params.message_id is absent (crates/librefang-channels/src/sidecar.rs lines 611-616), and that value is later passed back in send_reaction; _do_reaction then tries to use it as the Telegram message_id. For agents that send lifecycle reactions to inbound Telegram messages, reactions will fail or target the wrong message instead of the original Telegram message.

Useful? React with 👍 / 👎.

await loop.run_in_executor(None, self._do_interactive, cmd)
elif isinstance(cmd, protocol.StreamStart):
self._streams[cmd.stream_id] = {
"chat_id": cmd.channel_id, "thread_id": None,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Carry stream thread ids into Telegram sends

For streamed replies in Telegram forum topics, the daemon serializes the inbound thread id on stream_start (crates/librefang-channels/src/sidecar.rs line 1269), but the adapter discards it here by always storing None. As a result, non-streamed send_in_thread messages go to the topic, while streamed responses for the same threaded conversation are posted as top-level chat messages because _send_text never receives the message_thread_id.

Useful? React with 👍 / 👎.

Comment on lines +311 to +312
self._edit_text(st["chat_id"], st["msg_id"], st["text"][
:TELEGRAM_MSG_LIMIT * 2])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound streaming edits by UTF-16 message chunks

When a streamed response grows past Telegram's 4096 UTF-16-unit limit, this slice can still send more than the limit (8192 BMP characters, or far fewer with emoji) to editMessageText. In those longer streaming replies the edit call starts failing during the stream despite _chunks16 handling the same limit elsewhere; use the first _chunks16(st["text"]) chunk for edits instead of slicing by Python character count.

Useful? React with 👍 / 👎.

…inbound/outbound

Closes every NOT-YET item from increment 1, so the in-process
crates/librefang-channels/src/telegram.rs is now functionally
superseded by the sidecar (its removal is the separate kernel-cascade
change, deliberately not bundled here).

Faithful, function-by-function ports of the audited Rust (not
re-derived):

* Markdown -> Telegram-HTML formatter: byte-exact port of
  formatter::markdown_to_telegram_html (+ render_inline / fence /
  heading / list helpers), pinned by tests to the Rust
  formatter::tests oracle (bold/italic/code/link/heading/list/
  fenced-code/blockquote).
* sanitize_telegram_html security pass: tag + href-scheme allowlist,
  attribute-injection escaping, unclosed-tag balancing, rebuild_safe_tag
  / parse_attrs / is_safe_href — including the Rust edge where a
  dropped unsafe-<a> leaves its </a> escaped.
* message_truncator port: utf16_len / truncate_to_utf16_limit /
  split_to_utf16_chunks with the HTML-entity boundary guard and
  zero-progress guards.
* Outbound text now formats + sanitizes + chunks and sends
  parse_mode=HTML with the same plain-text retry on Telegram's
  "can't parse entities" 400 (was a naive parse_mode=Markdown).
* Full inbound: text/bot-command, photo, document, audio, voice,
  animation, video, video_note, location, sticker; from/sender_chat
  sender extraction; callback_query -> ButtonCallback (+ fire-and-forget
  answerCallbackQuery); edited_message; reply-to context; getFile URL
  resolution with text fallback; ALLOWED_USERS by id and username.
* Full outbound ChannelContent dispatch for every variant
  (Image/File/Voice/Video/Audio/Animation/Sticker/Location/MediaGroup/
  Poll/Interactive/EditInteractive/DeleteMessage), private-URL ->
  multipart upload, OGG/Opus voice routing, 429 retry_after.

Stdlib-only preserved. Full Python SDK suite 43 pass (telegram
parity + cross-impl conformance + ntfy/webhook/runtime, no regression).
@github-actions github-actions Bot added size/XL 1000+ lines changed and removed size/L 250-999 lines changed labels May 18, 2026
@houko houko changed the title feat(sidecar,sdk): telegram → SidecarAdapter parity, stdlib-only (increment 1/N) feat(sidecar,sdk): telegram full sidecar parity (formatter + full inbound/outbound), stdlib-only May 18, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 987fd31a4a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1387 to +1388
"allowed_updates": json.dumps(
["message", "edited_message", "callback_query"])},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include poll_answer in Telegram polling

When users answer a Telegram poll sent by this adapter, Telegram will not deliver those updates because this allowed_updates filter omits poll_answer; even if they arrived, _update_to_event only handles messages and callbacks. The existing in-process Telegram adapter requests poll_answer and converts it to ChannelContent::PollAnswer, and the Python protocol already exposes Content.poll_answer, so poll responses disappear after switching to this sidecar.

Useful? React with 👍 / 👎.

@houko houko left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — self-review log.

Formatter byte-exact port verifiedmarkdown_to_telegram_html faithfully ports crates/librefang-channels/src/formatter.rs:48-178. All 9 oracle cases from formatter::tests (bold/italic/code/link/heading/unordered/ordered/fenced/blockquote) replay verbatim in tests/test_telegram_adapter.py::test_markdown_to_telegram_html_matches_rust_oracle. Subtle bits — italic single-* scan with prev/next lookaround, _rust_lines matching Rust's str::lines trailing-newline behavior, escape ordering &→<→> — all preserved.

Capability negotiation aligns with Rust[typing, reaction, interactive, thread, streaming] matches the in-process surface: telegram.rs:2368 (supports_streaming = true via throttled editMessageText), reactions via setMessageReaction with the same emoji map (map_reaction_emoji_REACTION_MAP), inline keyboards, forum message_thread_id, sendChatAction. Streaming throttle matches (Rust STREAMING_EDIT_INTERVAL = 1000ms ↔ Python STREAM_EDIT_INTERVAL = 1.0).

Scope discipline — Correctly does NOT delete crates/librefang-channels/src/telegram.rs; channels-allowlist.txt still lists telegram. Removal is deferred to a separate kernel-cascade PR (analogous to #5224's ntfy path). Stdlib-only enforced by test_adapter_is_stdlib_only; no pyproject.toml change.

Tests + CI — 17/17 telegram tests pass; all 28 CI checks green; mergeable/clean.

Follow-up to track before in-process removal — Inbound poll_answer is handled by the Rust adapter (telegram.rs:2129-2230, subscribed in allowed_updates) but not by the sidecar (_poll_once subscribes only ["message", "edited_message", "callback_query"]; no _update_to_event branch for poll_answer). PR body's "Full inbound" list does not claim poll_answer, so this is consistent with stated scope. However, before the in-process telegram.rs can be removed, inbound poll_answer parity (and the poll_contexts cache for poll question/options) must land — otherwise operators using polls would silently lose answer events. Flag for the removal PR.

@houko
houko merged commit 80102e4 into main May 18, 2026
30 checks passed
@houko
houko deleted the fix/telegram-sidecar-stdlib branch May 18, 2026 14:44
houko added a commit that referenced this pull request May 18, 2026
… parity) (#5242)

PR #5232 shipped the sidecar Telegram adapter but its long-poll
subscription and update dispatcher missed Telegram's poll_answer
update — the in-process adapter at
crates/librefang-channels/src/telegram.rs:2129-2230 handles it
end-to-end, so the two implementations silently diverged on any
non-anonymous poll vote.

Add a poll_answer branch to _update_to_event and extend
allowed_updates so the long-poll actually receives the update.
Field-by-field mirror of the Rust oracle: sender.platform_id =
user.id, display_name = first_name+last_name (with the same
'Unknown'/empty-last fallbacks), platform_message_id = poll_id,
content = PollAnswer{poll_id, option_ids}, is_group=false,
metadata = {user_id, sender_user_id}. poll_answer fires only in
private chats, so channel_id falls back to user_id — matching the
Rust comment near SENDER_USER_ID_KEY.

New oracle tests pin every mapped field, the empty-poll_id /
disallowed-user filters, and the allowed_updates subscription.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation and guides area/sdk JavaScript and Python SDKs size/XL 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant