Skip to content

TTS in final mode: visible text sent before audio causes Telegram delete-and-resend churn #83511

Description

@danielt69

Symptom

When messages.tts.mode = "final" is enabled with a Telegram channel, every assistant turn produces two outbound messages: text first, then a follow-up voice note. Some surfaces (e.g. ours) then have to delete or replace the now-redundant text message, causing visible UI churn for the user.

Root cause

In src/auto-reply/reply/dispatch-acp.ts, finalizeAcpTurnOutput() awaits delivery.settleVisibleText() (line 218) before maybeApplyTtsToPayload() is invoked (line ~239). So the visible text is always flushed to the channel before the audio is even synthesized. The TTS-supplement reply is then queued behind it with visibleTextAlreadyDelivered: true.

The channel layer already supports the cleaner path: Telegram's sendVoice accepts a caption, and splitTelegramCaption() exists for the overflow case.

Proposed fix

When ttsMode === "final", hasAccumulatedBlockText, canAttemptFinalTts, and the channel supports voice-with-caption, defer settleVisibleText() until after the TTS attempt. If TTS succeeds, deliver a single combined { mediaUrl, audioAsVoice, text (as caption) } payload. If TTS fails, fall through to the existing text-flush path.

Rough sketch:

const hasTtsCandidate = ttsMode === "final" && accumulatedBlockTtsText.trim().length > 0 && canAttemptFinalTts;
const channelSupportsCaptionedVoice = supportsCaptionedVoice(params.ttsChannel);
const deferTextForTts = hasTtsCandidate && channelSupportsCaptionedVoice;

if (!deferTextForTts) {
  await params.delivery.settleVisibleText();
}

let finalMediaDelivered = false;
if (hasTtsCandidate) {
  // ... maybeApplyTtsToPayload as today, but include caption text in payload when deferTextForTts ...
}

if (deferTextForTts && !finalMediaDelivered) {
  await params.delivery.settleVisibleText(); // TTS failed: flush text now
}

Scope

Telegram is the obvious first target (caption + voice supported natively). Discord, WhatsApp, Signal could be added behind the same channelSupportsCaptionedVoice capability flag.

Workaround

None clean. messages.tts.mode = "all" makes the symptom worse. Patching dist/ survives only until the next brew upgrade openclaw.

Happy to send a PR if the direction is acceptable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions