Skip to content

feat(channels): Telegram rich media, polls, interactive commands, and channel_send tool#2356

Merged
houko merged 14 commits into
librefang:mainfrom
leszek3737:feat/telegram
Apr 13, 2026
Merged

feat(channels): Telegram rich media, polls, interactive commands, and channel_send tool#2356
houko merged 14 commits into
librefang:mainfrom
leszek3737:feat/telegram

Conversation

@leszek3737

@leszek3737 leszek3737 commented Apr 12, 2026

Copy link
Copy Markdown
Contributor

Type

  • Channel adapter

Summary

Extend Telegram adapter with rich media delivery, poll creation, interactive inline-keyboard commands (/agents, /models), and wire channel_send as a built-in tool for Messaging/Automation profiles.

Changes

Telegram adapter

  • Rich media: photos, files, audio, video, location, stickers, media captions
  • Polls: regular + quiz mode (correct answer, explanation)
  • Interactive: /agents renders inline buttons, /models provider→model drill-down
  • Auto-registration on /start
  • DeleteMessage, EditInteractive content variants
  • Callback data truncation on UTF-8 boundaries
  • Poll context cleanup (leak fix)

channel_send tool

  • Added to Messaging and Automation tool profiles
  • Poll params: poll_question, poll_options (2–10), poll_is_quiz, poll_correct_option, poll_explanation
  • Validation: quiz requires correct_option, bounds check, mutually exclusive with media
  • KernelHandle::send_channel_poll wired through kernel
  • Prompt builder injects channel_send usage hint when tool available

Hands default model

  • Provider/model default changed from hardcoded "anthropic"/"claude-sonnet-4-20250514" to "default" sentinel
  • Kernel resolves independently — hand can pin one, inherit the other
  • api_key_env/base_url inherited only when hand didn't set them explicitly

Types

  • New ChannelContent variants: Poll, DeleteMessage, EditInteractive
  • InteractiveButton type, truncate_utf8 helper
  • channel_send in tool hints

CI / Docs

  • release-notify.yml split into Docker image + deploy status checks
  • Security docs (EN + ZH): integrity, network-api, operations, sandboxing

Attribution

  • This PR preserves author attribution for any adapted prior work

Testing

  • cargo clippy --workspace --all-targets -- -D warnings passes
  • cargo test --workspace passes
  • Live integration tested (if applicable)

Security

  • No new unsafe code
  • No secrets or API keys in diff
  • User input validated at boundaries

…nds, and auto-registration

Add 7 new ChannelContent variants (DeleteMessage, Audio, Animation, Sticker,
MediaGroup, Poll, PollAnswer) with full send/receive support in the Telegram
adapter. New API methods: sendAudio, sendAnimation, sendSticker, sendMediaGroup,
sendPoll, deleteMessage, setMyCommands, deleteMyCommands, getMyCommands.

Inbound parsing now handles audio files (as Audio, not Voice), animations
(checked before video to avoid misclassification), stickers, and poll answers.

Bot command menu is auto-registered on start() using built-in defaults
(BUILTIN_COMMANDS) when no custom commands are configured, and re-registered
on /start or first incoming message via the polling loop.

Bridge updated with exhaustive match arms in content_to_text and
dispatch_message for all new variants, plus 8 new unit tests.
Agents using Messaging or Automation profiles had no way to send images,
files, or other media to channel users because channel_send was only
available in the Full profile. This made the agent suggest using the API
directly instead of its built-in tools.
The channel awareness section in the system prompt told agents about
formatting and character limits but never mentioned channel_send.
Agents had the tool available but did not know to use it for sending
images or files, proposing raw API calls instead.

Add a channel_send hint to the Channel section when the tool is granted,
including the recipient ID so the agent can use it immediately. Also add
a tool_hint entry so channel_send appears with a description in the
Your Tools section.
Add send_channel_poll to KernelHandle trait and kernel implementation.
Extend channel_send tool with poll_question, poll_options, poll_is_quiz,
poll_correct_option, poll_explanation parameters. Telegram adapter stores
poll context (question + options) keyed by poll_id and attaches it to
inbound PollAnswer messages so agents see readable answers instead of
bare numeric indices.
…ivity

api_send_media_group and api_send_poll previously logged non-429 errors
and returned Ok, making callers believe delivery succeeded. Both now
return Err with status and body. channel_send's poll branch now rejects
requests that also carry image_url/image_path/file_url/file_path,
matching the schema's documented mutual exclusivity.
@github-actions github-actions Bot added area/channels Messaging channel adapters area/runtime Agent loop, LLM drivers, WASM sandbox area/kernel Core kernel (scheduling, RBAC, workflows) labels Apr 12, 2026
@github-actions
github-actions Bot requested a review from houko April 12, 2026 23:30
@leszek3737 leszek3737 changed the title Feat/telegram feat(channels): extend Telegram adapter with rich media, polls, and commands Apr 12, 2026
Intercept /agents in both Command and Text dispatch paths so Telegram
bot-command entities reach the interactive handler. Emits an
InteractiveMessage with one button per agent, truncating names to fit
the 64-byte callback_data limit.
@leszek3737
leszek3737 marked this pull request as draft April 13, 2026 06:33
Replace plain-text /models with a multi-level Telegram inline keyboard:
providers list → provider's models (with Back) → confirmation with
cleared keyboard. Adds ChannelContent::EditInteractive for in-place
editMessageText updates and filters the provider menu to only those
with is_available() auth status.
Slicing agent names, provider IDs, and model IDs by byte index risked
panics when the cut fell inside a multi-byte UTF-8 character. Extract
truncate_utf8 helper in types.rs and use it at every callback_data
truncation site (bridge.rs and telegram.rs).
- Add poll_contexts cleanup task (5min interval, 30min TTL) via
  last_accessed field on PollContext to prevent unbounded growth.
- Remove i==0 guard from media group caption handling so each item
  with a caption uses it instead of silently dropping later ones.
- Validate poll options: 2-10 max, and for quiz mode require
  correct_option_id with bounds checking.
@leszek3737 leszek3737 changed the title feat(channels): extend Telegram adapter with rich media, polls, and commands feat(channels): Telegram rich media, polls, interactive commands, and channel_send tool Apr 13, 2026
@leszek3737
leszek3737 marked this pull request as ready for review April 13, 2026 07:26

@houko houko left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on the feature shape — rich media dispatch, poll creation, interactive /agents and /models drill-down, and channel_send tool wiring are all well-structured and end-to-end. truncate_utf8 extracted cleanly, PollParams struct avoids clippy too-many-arguments, and kernel.set_agent_model(aid, model, None) correctly defers to the catalog for provider resolution on the model-switch button path.

Merging; will push a follow-up commit to address the two things I'd have blocked on otherwise: (1) poll_options silently filtering out non-string entries, and (2) PollContext.last_accessed never being updated on get, which drops poll question/options metadata from answers arriving >30min after the poll was sent.

@houko
houko merged commit edfa73c into librefang:main Apr 13, 2026
11 of 12 checks passed
@leszek3737
leszek3737 deleted the feat/telegram branch April 13, 2026 08:01
houko added a commit that referenced this pull request Apr 13, 2026
Follow-up to #2356 addressing four issues that would bite in production:

1. channel_send: poll_options silently dropped non-string entries.
   The previous filter_map(as_str) pattern turned ["a", 42, "c"] into
   ["a", "c"], slipped past the min-2 check, and sent a poll with the
   user's third option missing. Extract a parse_poll_options helper
   that fails fast with a per-index error when any element is not a
   string, and pin it with eight unit tests covering the type cases,
   the missing-array case, and the 1/11 bounds.

2. telegram: PollContext.last_accessed was never updated on access.
   poll_contexts.get() returned an immutable ref, so the background
   cleanup removed entries 30 minutes after send regardless of whether
   users were still answering — long-running surveys silently lost
   their poll_question / poll_options metadata. Switch to get_mut so
   each answer bumps last_accessed, making the field honest.

3. telegram: poll cleanup task did not respond to shutdown for up to
   5 minutes. The loop had a bare interval.tick().await, so a stop()
   call would block until the next tick. Wrap the tick in a
   tokio::select! against shutdown.changed() and exit immediately when
   the signal flips.

4. telegram: api_send_media_group accepted 1 or 11+ items and let the
   Telegram API reject them with an opaque 400. Validate locally that
   the slice has 2..=10 items before composing the request.
houko added a commit that referenced this pull request Apr 13, 2026
…2364)

Follow-up to #2356 addressing four issues that would bite in production:

1. channel_send: poll_options silently dropped non-string entries.
   The previous filter_map(as_str) pattern turned ["a", 42, "c"] into
   ["a", "c"], slipped past the min-2 check, and sent a poll with the
   user's third option missing. Extract a parse_poll_options helper
   that fails fast with a per-index error when any element is not a
   string, and pin it with eight unit tests covering the type cases,
   the missing-array case, and the 1/11 bounds.

2. telegram: PollContext.last_accessed was never updated on access.
   poll_contexts.get() returned an immutable ref, so the background
   cleanup removed entries 30 minutes after send regardless of whether
   users were still answering — long-running surveys silently lost
   their poll_question / poll_options metadata. Switch to get_mut so
   each answer bumps last_accessed, making the field honest.

3. telegram: poll cleanup task did not respond to shutdown for up to
   5 minutes. The loop had a bare interval.tick().await, so a stop()
   call would block until the next tick. Wrap the tick in a
   tokio::select! against shutdown.changed() and exit immediately when
   the signal flips.

4. telegram: api_send_media_group accepted 1 or 11+ items and let the
   Telegram API reject them with an opaque 400. Validate locally that
   the slice has 2..=10 items before composing the request.
DaBlitzStein pushed a commit to DaBlitzStein/librefang that referenced this pull request Apr 13, 2026
…hardening

Brings in:
- feat(channels): Telegram rich media, polls, interactive commands, channel_send tool (librefang#2356)
- feat(runtime): config-driven session_mode for agent triggers (librefang#2341)
- fix(agents): refuse direct DELETE on hand-spawned agents (librefang#2361)
- fix(channels): harden poll_options parsing and poll context cleanup (librefang#2364)

Conflicts resolved: bridge.rs (new trait methods), prompt_builder.rs
(9-param build_channel_section merging bot_username/group_members with
granted_tools), tool_runner.rs (cron-tz tests + poll_options tests).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/channels Messaging channel adapters area/kernel Core kernel (scheduling, RBAC, workflows) area/runtime Agent loop, LLM drivers, WASM sandbox

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants