feat(sidecar,sdk): telegram full sidecar parity (formatter + full inbound/outbound), stdlib-only#5232
Conversation
…ests) Follow-up to #5228. After the first-party adapters moved into librefang-sdk, `librefang.sidecar.adapters.telegram` shipped as a hand-rolled stdin/stdout script that `import requests` — a dep never declared in sdk/python/pyproject.toml (dependencies = [], no extra), so `pip install librefang-sdk` then `python -m librefang.sidecar.adapters.telegram` died with ModuleNotFoundError: requests. Rewritten as a `SidecarAdapter` subclass on the librefang.sidecar SDK, identical in shape to the ntfy adapter, using `urllib`. The whole `librefang.sidecar.adapters` package is now zero-dependency (ntfy / webhook were already stdlib-only). Transport behaviour preserved: long-poll getUpdates (server 30s / client 35s / offset), ALLOWED_USERS numeric whitelist, sender = first_name / username / "unknown", channel_id = chat id, platform = "telegram", outbound sendMessage parse_mode Markdown. Intentional SDK-consistent alignments vs the old raw script (matching ntfy): missing TELEGRAM_BOT_TOKEN exits (supervisor reports it) rather than emitting an error event; transport/API/send failures log + drive the SDK's supervised backoff instead of hand-emitted protocol error events; ready handshake is the SDK's idempotent ready/ready_ack loop. Operator config is unchanged. No pyproject change — the fix removes the need for `requests` entirely rather than adding a `[telegram]` extra (keeps the adapters package uniformly zero-dep). No Rust touched; no golden change (telegram is not in the kernel-config schema). Verification (clean venv, NO requests installed): - `python3 -c "import librefang.sidecar.adapters.telegram"` — OK - `cd sdk/python && pytest` — 32 passed: new tests/test_telegram_adapter.py (stdlib-only assertion; `_update_to_event` text/sender/whitelist/ non-text-skip; `_api_get` ok / HTTPError-body / non-JSON / timeout-reraise; `_poll_once` emit+offset+API-error; `on_send` text/unsupported/skip), plus existing ntfy + SDK suites green (no regression).
…ment 1/N) Promote the stdlib-only telegram adapter from a plain text relay onto SidecarAdapter with capability negotiation, matching the in-process crates/librefang-channels/src/telegram.rs feature surface so the kernel routes rich commands here instead of degrading to text. DONE this increment: * capabilities = [typing, reaction, interactive, thread, streaming] * reaction emoji map identical to the Rust adapter (map_reaction_emoji), incl. optional clear-on-done * inline keyboards; forum-topic message_thread_id; throttled editMessageText streaming with UTF-16-aware 4096 chunking (_chunks16, the stdlib equivalent of split_to_utf16_chunks) * inbound text long-poll, ALLOWED_USERS whitelist (unchanged) NOT YET (gates removal of the in-process adapter, tracked): * faithful Markdown -> Telegram-HTML formatter subsystem (this increment sends parse_mode=Markdown like the pre-migration adapter, not the Rust formatter::TelegramHtml pipeline) * full inbound parsing of non-text updates into ChannelContent variants Stdlib-only preserved (no requests). 37 SDK tests pass.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 134ef485a2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| content=protocol.Content.text(msg["text"]), | ||
| channel_id=str(msg.get("chat", {}).get("id", "")), | ||
| platform="telegram", |
There was a problem hiding this comment.
Preserve Telegram message ids for reactions
When a normal Telegram update is converted to a sidecar message, this omits msg["message_id"] even though the adapter now declares the reaction capability. The sidecar reader falls back to a generated UUID when params.message_id is absent (crates/librefang-channels/src/sidecar.rs lines 611-616), and that value is later passed back in send_reaction; _do_reaction then tries to use it as the Telegram message_id. For agents that send lifecycle reactions to inbound Telegram messages, reactions will fail or target the wrong message instead of the original Telegram message.
Useful? React with 👍 / 👎.
| await loop.run_in_executor(None, self._do_interactive, cmd) | ||
| elif isinstance(cmd, protocol.StreamStart): | ||
| self._streams[cmd.stream_id] = { | ||
| "chat_id": cmd.channel_id, "thread_id": None, |
There was a problem hiding this comment.
Carry stream thread ids into Telegram sends
For streamed replies in Telegram forum topics, the daemon serializes the inbound thread id on stream_start (crates/librefang-channels/src/sidecar.rs line 1269), but the adapter discards it here by always storing None. As a result, non-streamed send_in_thread messages go to the topic, while streamed responses for the same threaded conversation are posted as top-level chat messages because _send_text never receives the message_thread_id.
Useful? React with 👍 / 👎.
| self._edit_text(st["chat_id"], st["msg_id"], st["text"][ | ||
| :TELEGRAM_MSG_LIMIT * 2]) |
There was a problem hiding this comment.
Bound streaming edits by UTF-16 message chunks
When a streamed response grows past Telegram's 4096 UTF-16-unit limit, this slice can still send more than the limit (8192 BMP characters, or far fewer with emoji) to editMessageText. In those longer streaming replies the edit call starts failing during the stream despite _chunks16 handling the same limit elsewhere; use the first _chunks16(st["text"]) chunk for edits instead of slicing by Python character count.
Useful? React with 👍 / 👎.
…inbound/outbound Closes every NOT-YET item from increment 1, so the in-process crates/librefang-channels/src/telegram.rs is now functionally superseded by the sidecar (its removal is the separate kernel-cascade change, deliberately not bundled here). Faithful, function-by-function ports of the audited Rust (not re-derived): * Markdown -> Telegram-HTML formatter: byte-exact port of formatter::markdown_to_telegram_html (+ render_inline / fence / heading / list helpers), pinned by tests to the Rust formatter::tests oracle (bold/italic/code/link/heading/list/ fenced-code/blockquote). * sanitize_telegram_html security pass: tag + href-scheme allowlist, attribute-injection escaping, unclosed-tag balancing, rebuild_safe_tag / parse_attrs / is_safe_href — including the Rust edge where a dropped unsafe-<a> leaves its </a> escaped. * message_truncator port: utf16_len / truncate_to_utf16_limit / split_to_utf16_chunks with the HTML-entity boundary guard and zero-progress guards. * Outbound text now formats + sanitizes + chunks and sends parse_mode=HTML with the same plain-text retry on Telegram's "can't parse entities" 400 (was a naive parse_mode=Markdown). * Full inbound: text/bot-command, photo, document, audio, voice, animation, video, video_note, location, sticker; from/sender_chat sender extraction; callback_query -> ButtonCallback (+ fire-and-forget answerCallbackQuery); edited_message; reply-to context; getFile URL resolution with text fallback; ALLOWED_USERS by id and username. * Full outbound ChannelContent dispatch for every variant (Image/File/Voice/Video/Audio/Animation/Sticker/Location/MediaGroup/ Poll/Interactive/EditInteractive/DeleteMessage), private-URL -> multipart upload, OGG/Opus voice routing, 429 retry_after. Stdlib-only preserved. Full Python SDK suite 43 pass (telegram parity + cross-impl conformance + ntfy/webhook/runtime, no regression).
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 987fd31a4a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "allowed_updates": json.dumps( | ||
| ["message", "edited_message", "callback_query"])}, |
There was a problem hiding this comment.
Include poll_answer in Telegram polling
When users answer a Telegram poll sent by this adapter, Telegram will not deliver those updates because this allowed_updates filter omits poll_answer; even if they arrived, _update_to_event only handles messages and callbacks. The existing in-process Telegram adapter requests poll_answer and converts it to ChannelContent::PollAnswer, and the Python protocol already exposes Content.poll_answer, so poll responses disappear after switching to this sidecar.
Useful? React with 👍 / 👎.
houko
left a comment
There was a problem hiding this comment.
LGTM — self-review log.
Formatter byte-exact port verified — markdown_to_telegram_html faithfully ports crates/librefang-channels/src/formatter.rs:48-178. All 9 oracle cases from formatter::tests (bold/italic/code/link/heading/unordered/ordered/fenced/blockquote) replay verbatim in tests/test_telegram_adapter.py::test_markdown_to_telegram_html_matches_rust_oracle. Subtle bits — italic single-* scan with prev/next lookaround, _rust_lines matching Rust's str::lines trailing-newline behavior, escape ordering &→<→> — all preserved.
Capability negotiation aligns with Rust — [typing, reaction, interactive, thread, streaming] matches the in-process surface: telegram.rs:2368 (supports_streaming = true via throttled editMessageText), reactions via setMessageReaction with the same emoji map (map_reaction_emoji → _REACTION_MAP), inline keyboards, forum message_thread_id, sendChatAction. Streaming throttle matches (Rust STREAMING_EDIT_INTERVAL = 1000ms ↔ Python STREAM_EDIT_INTERVAL = 1.0).
Scope discipline — Correctly does NOT delete crates/librefang-channels/src/telegram.rs; channels-allowlist.txt still lists telegram. Removal is deferred to a separate kernel-cascade PR (analogous to #5224's ntfy path). Stdlib-only enforced by test_adapter_is_stdlib_only; no pyproject.toml change.
Tests + CI — 17/17 telegram tests pass; all 28 CI checks green; mergeable/clean.
Follow-up to track before in-process removal — Inbound poll_answer is handled by the Rust adapter (telegram.rs:2129-2230, subscribed in allowed_updates) but not by the sidecar (_poll_once subscribes only ["message", "edited_message", "callback_query"]; no _update_to_event branch for poll_answer). PR body's "Full inbound" list does not claim poll_answer, so this is consistent with stated scope. However, before the in-process telegram.rs can be removed, inbound poll_answer parity (and the poll_contexts cache for poll question/options) must land — otherwise operators using polls would silently lose answer events. Flag for the removal PR.
… parity) (#5242) PR #5232 shipped the sidecar Telegram adapter but its long-poll subscription and update dispatcher missed Telegram's poll_answer update — the in-process adapter at crates/librefang-channels/src/telegram.rs:2129-2230 handles it end-to-end, so the two implementations silently diverged on any non-anonymous poll vote. Add a poll_answer branch to _update_to_event and extend allowed_updates so the long-poll actually receives the update. Field-by-field mirror of the Rust oracle: sender.platform_id = user.id, display_name = first_name+last_name (with the same 'Unknown'/empty-last fallbacks), platform_message_id = poll_id, content = PollAnswer{poll_id, option_ids}, is_group=false, metadata = {user_id, sender_user_id}. poll_answer fires only in private chats, so channel_id falls back to user_id — matching the Rust comment near SENDER_USER_ID_KEY. New oracle tests pin every mapped field, the empty-poll_id / disallowed-user filters, and the allowed_updates subscription.
Follow-up to #5228. Full sidecar parity for telegram so the in-process
crates/librefang-channels/src/telegram.rscan be removed. Threecommits, one tight cluster:
daaa2712— kill therequestsdependency (the fix(sidecar,sdk): move first-party channel adapters out of examples into librefang-sdk #5228 wart).134ef485— promote toSidecarAdapterwith capabilitynegotiation (typing/reaction/interactive/thread/streaming).
987fd31a— full parity: faithful, function-by-function portsof the audited Rust (not re-derived).
What
987fd31aports (with the Rust oracle it is pinned to)formatter::markdown_to_telegram_html(+ inline/fence/heading/listhelpers). Tests assert the same input→output pairs as
crate::formatter's ownmod tests, so the two implementationscannot drift silently. Outbound text now formats + sanitizes +
chunks and sends
parse_mode=HTML(was a naiveparse_mode=Markdown), with the same plain-text retry on Telegram's"can't parse entities" 400.
sanitize_telegram_htmlsecurity pass — tag +<a href>scheme allowlist, attribute-injection escaping, unclosed-tag
balancing (
rebuild_safe_tag/parse_attrs/is_safe_href),including the exact Rust edge where a dropped unsafe-
<a>leavesits now-unmatched
</a>escaped.message_truncatorchunker —utf16_len/truncate_to_utf16_limit/split_to_utf16_chunkswith theHTML-entity boundary guard and the zero-progress guards.
animation, video, video_note, location, sticker;
from/sender_chatextraction;callback_query→ButtonCallback(+ fire-and-forget
answerCallbackQuery);edited_message;reply-to context;
getFileURL resolution with text fallback;ALLOWED_USERSby id and username.ChannelContentdispatch for every variant(Image→sendPhoto, File→sendDocument/sendVoice, Voice/Video/Audio/
Animation, Sticker, Location, MediaGroup, Poll, Interactive,
EditInteractive, DeleteMessage), private-URL → multipart upload,
OGG/Opus voice routing, 429
retry_afterretry.Scope boundary
telegram.rsis NOT in this PR. That deletion is abreaking, kernel/config/api-cascading change (cf. the ntfy feat(channels)!: migrate ntfy from in-process adapter to sidecar (P7) #5224
cascade) and is intentionally a separate reviewed PR; this PR makes
the sidecar a faithful functional replacement so that removal is
unblocked.
pyproject.tomlchange; stdlib-only preserved (norequests).No Rust touched.
Verification
cd sdk/python && pytest) — 43 pass:formatter byte-exact vs the Rust oracle; sanitizer (incl. unsafe-href
/ unclosed / unknown-tag / idempotent); UTF-16 + HTML-entity
chunking; inbound (text/command/media/getFile-fallback/callback/
edited/reply/allowed-by-id+username); outbound dispatch for every
variant; plain-text 400 fallback; streaming; reaction map;
regression)