fix(openai): mark mp3 TTS voice output compatible by HemantSudarshan · Pull Request #83227 · openclaw/openclaw

HemantSudarshan · 2026-05-17T18:23:46Z

Summary

Fixes OpenAI-compatible TTS voice-note compatibility for configured responseFormat: "mp3".
Reuses the shared media runtime voice-compatibility contract instead of hard-coding OpenAI response formats.
Adds provider and speech-core regressions so provider-compatible MP3 output continues to route as Telegram voice delivery.

Problem summary

The OpenAI speech provider only marked voice-note synthesis as voice-compatible when responseFormat === "opus". That made OpenAI-compatible local TTS servers configured for MP3 produce an audio file result even though the shared media contract and Telegram send path already allow MP3 voice delivery.

Root cause

extensions/openai/speech-provider.ts used a provider-local equality check for opus, while the rest of the TTS/channel path uses isVoiceCompatibleAudio() to decide whether a produced audio asset can be delivered as a voice note. Microsoft/Edge TTS already follows that shared helper path for MP3 output, so OpenAI-compatible MP3 output diverged from the existing runtime contract.

Architectural reasoning

This keeps the behavior at the provider boundary: the OpenAI provider still reports compatibility only for a voice-note synthesis target, but it now asks the shared media runtime whether the selected container extension is voice-compatible. The change does not alter provider selection, gateway/session flow, async cleanup, request timing, or Telegram transport behavior. It only corrects the metadata emitted by OpenAI-compatible synthesis results so the existing speech-core and Telegram delivery contracts can do what they already support.

Testing performed

Local proof on refreshed head 7d6773b18c4679697f5ca65c9d1fa7b1ac382b5b:

pnpm test extensions/openai/speech-provider.test.ts extensions/speech-core/src/tts.test.ts -- --reporter=verbose passed: OpenAI provider 10/10, speech-core 28/28.
pnpm test extensions/telegram/src/send.test.ts -- --reporter=verbose -t "routes audio media to sendAudio/sendVoice based on voice compatibility" passed: 1/1 selected, 102 skipped.
Local OpenAI-compatible HTTP proof using production buildOpenAISpeechProvider() captured POST /v1/audio/speech with response_format: "mp3"; the returned result was outputFormat: "mp3", fileExtension: ".mp3", and voiceCompatible: true.
pnpm format:check -- extensions/openai/speech-provider.ts extensions/openai/speech-provider.test.ts extensions/speech-core/src/tts.test.ts
pnpm check:changed
git diff --check upstream/main..HEAD

GitHub Actions on 7d6773b18c4679697f5ca65c9d1fa7b1ac382b5b:

Workflow Sanity #201563: success
OpenGrep PR Diff Models: suppress repeated vLLM/Ollama discovery warnings (#21037) #21199: success
CI #207277: success
CodeQL Critical Quality [Bug]: hooks.mappings channel validation mismatch — config schema rejects plugin channel IDs (e.g. bluebubbles), runtime rejects legacy channel IDs (e.g. imessage) #17155: success

Real behavior proof

Behavior addressed: OpenAI-compatible TTS configured with responseFormat: "mp3" should mark voice-note synthesis output as voice-compatible so Telegram delivery can use the voice path instead of falling back to audio-file delivery.

Real environment tested: Local Windows checkout on branch Hemant/fix-openai-tts-mp3-voice, Node v22.15.0 with the repository's existing engine warning. The branch was rebased onto upstream/main 6baa2b38b2712384d7de460d90b127cd78dbb3a5; upstream moved to bc4f27c89a3c5aa40b7fb9e507a91b6e6c753307 while checks ran, and GitHub reported the PR mergeable against the live base after push. The extra proof used the production OpenAI speech provider against a real local HTTP OpenAI-compatible speech endpoint with redacted auth.

Exact steps or command run after this patch: Started a local HTTP server, invoked buildOpenAISpeechProvider().synthesize() with baseUrl pointed at that server, responseFormat: "mp3", and target: "voice-note", then captured the outbound OpenAI-compatible request and provider result.

Evidence after fix:

{
  "capturedRequest": {
    "method": "POST",
    "url": "/v1/audio/speech",
    "responseFormat": "mp3",
    "model": "local-compatible-tts",
    "voice": "alloy"
  },
  "outputFormat": "mp3",
  "fileExtension": ".mp3",
  "voiceCompatible": true,
  "audioBytes": 6
}

Observed result after fix: The production provider requests MP3 from the OpenAI-compatible speech endpoint and returns MP3 voice-note synthesis as voice-compatible. The speech-core regression keeps that provider-compatible MP3 result on Telegram voice delivery, and the Telegram send regression confirms compatible MP3 voice delivery maps to sendVoice.

What was not tested: I did not send a live Telegram message or call a credentialed third-party OpenAI-compatible TTS server. The local HTTP proof exercises the same production provider request/result path without secrets.

Risk analysis

Runtime behavior: Narrow metadata correction for OpenAI-compatible TTS results only.
Concurrency/session/gateway flows: No changes.
Cleanup/lifecycle handling: No changes.
API behavior: No public config or SDK shape changes.
Backward compatibility: Existing Opus voice-note behavior stays compatible; WAV remains non-voice-compatible through the shared helper.
Developer ergonomics: Removes duplicated provider-local format policy and aligns OpenAI with the shared media contract already used by other providers.

Backward compatibility

Backward compatible. Users who explicitly configured responseFormat: "mp3" for OpenAI-compatible TTS now get the expected voice-compatible metadata for voice-note targets. Non-voice targets and formats that are not accepted by the shared media helper keep their previous fallback behavior.

clawsweeper · 2026-05-17T18:24:55Z

Codex review: needs changes before merge. Reviewed July 1, 2026, 1:58 AM ET / 05:58 UTC.

Summary
The branch changes the OpenAI speech provider to use shared media voice compatibility for configured MP3 voice-note output and adds OpenAI plus speech-core regressions.

PR surface: Source +7, Tests +39. Total +46 across 3 files.

Reproducibility: yes. from source inspection rather than a live run. Current main still returns voiceCompatible=false for configured OpenAI MP3 voice-note synthesis, while speech-core and Telegram already use the voice path when that flag is true.

Review metrics: 1 noteworthy metric.

Current branch repairability: mergeable=CONFLICTING; maintainerCanModify=false. The implementation is focused, but maintainers need an author rebase or a replacement branch before this can land.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #80317
Summary: This PR is the focused open fix candidate for the canonical OpenAI-configured MP3 TTS voice-compatibility bug; older Telegram voice-note reports covered broader already-closed behavior.

Members:

canonical: TTS OpenAI provider: MP3 responseFormat not voice-compatible for Telegram, unlike Edge TTS #80317 - The PR body uses closing syntax for this open issue, and the issue describes the same OpenAI responseFormat MP3 voice-note compatibility bug.
superseded: fix(openai): treat mp3 responseFormat as voice-compatible for Telegram voice notes #80326 - This earlier PR attempted the same behavior but is closed unmerged and is not the current viable landing path.
partial_overlap: feat(tts): send Telegram voice messages as OGG/Opus (sendVoice) instead of audio files #43072 - That closed issue covered broader Telegram TTS voice-note delivery, while this PR addresses the remaining OpenAI-provider MP3 metadata gap.
partial_overlap: TTS tool on Telegram renders as audio file instead of native voice memo (sendAudio vs sendVoice) #44459 - That closed issue covered generic Telegram TTS sendVoice routing and MP3 send tests, which are prerequisites but not the OpenAI responseFormat-specific bug.
partial_overlap: [Feature]: Feature Request: Allow configurable file extension override (e.g., .ogg) for OpenAI TTS provider to support native Telegram Voice Notes #21704 - That closed request proposed a configurable file-extension workaround for OpenAI TTS voice notes; this PR has a narrower provider metadata fix.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🦞 diamond lobster
Patch quality: 🦐 gold shrimp
Result: needs maintainer review before merge.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P2] Rebase or apply the focused fix on current main, or enable maintainer edits if the contributor branch should remain the landing path.
Move the speech-core regression to packages/speech-core/src/tts.test.ts.
Run the focused OpenAI provider, speech-core, and Telegram send-path tests after the rebase.

Mantis proof suggestion
Native Telegram proof would directly show the user-visible MP3 TTS voice-note outcome after the provider metadata fix. A maintainer can ask Mantis to capture proof by posting this exact PR comment:

@openclaw-mantis telegram desktop proof: verify OpenAI-compatible TTS with responseFormat mp3 sends a Telegram voice note rather than an audio file.

Risk before merge

[P1] GitHub reports this contributor fork as conflicting and maintainerCanModify=false, so maintainers cannot directly repair this exact branch before landing.
[P1] The speech-core regression is added under extensions/speech-core, which current main no longer contains; it needs to move to packages/speech-core to protect the current package.

Maintainer options:

Decide the mitigation before merge
Land or replace the provider-boundary fix on current main, using shared media compatibility and keeping the speech-core regression under packages/speech-core.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P2] The remaining blocker is mechanical: the contributor fork is conflicting and not maintainer-editable, and the regression needs to move to the current speech-core package path.

Security
Cleared: The diff changes provider metadata logic and focused tests only; it does not touch dependencies, workflows, secrets, package scripts, permissions, or downloaded code.

Review findings

[P2] Move the speech-core regression to packages/speech-core — extensions/speech-core/src/tts.test.ts:265

Review details

Best possible solution:

Land or replace the provider-boundary fix on current main, using shared media compatibility and keeping the speech-core regression under packages/speech-core.

Do we have a high-confidence way to reproduce the issue?

Yes, from source inspection rather than a live run. Current main still returns voiceCompatible=false for configured OpenAI MP3 voice-note synthesis, while speech-core and Telegram already use the voice path when that flag is true.

Is this the best way to solve the issue?

Yes for the provider-boundary shape, but not for the current branch state. Reusing the shared media helper is the narrow fix; the branch still needs a rebase or replacement and the speech-core test must move to the current package path.

Full review comments:

[P2] Move the speech-core regression to packages/speech-core — extensions/speech-core/src/tts.test.ts:265
Current main no longer has extensions/speech-core; the speech-core test suite lives under packages/speech-core, so this added regression is on a stale path and cannot land or protect the current package after rebase. Move this case to packages/speech-core/src/tts.test.ts.
Confidence: 0.95

Overall correctness: patch is incorrect
Overall confidence: 0.88

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 2c7e989686ba.

Label changes

Label justifications:

P2: This is a bounded OpenAI-provider to Telegram voice-delivery bug with a clear repair path and limited blast radius.
rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body includes after-fix local production-provider HTTP proof showing an OpenAI-compatible MP3 request returning voiceCompatible=true; live Telegram UI proof would add visible confidence but is not required for the provider proof gate.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix local production-provider HTTP proof showing an OpenAI-compatible MP3 request returning voiceCompatible=true; live Telegram UI proof would add visible confidence but is not required for the provider proof gate.
mantis: telegram-visible-proof: Mantis should capture Telegram visible proof. The PR changes whether OpenAI-compatible MP3 TTS appears as a native Telegram voice note, which is visible in a short Telegram Desktop recording.

Evidence reviewed

PR surface:

Source +7, Tests +39. Total +46 across 3 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	8	1	+7
Tests	2	40	1	+39
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	3	48	2	+46

Acceptance criteria:

[P1] node scripts/run-vitest.mjs extensions/openai/speech-provider.test.ts packages/speech-core/src/tts.test.ts.
[P1] node scripts/run-vitest.mjs extensions/telegram/src/send.test.ts -t "routes audio media to sendAudio/sendVoice based on voice compatibility".
[P1] git diff --check.

What I checked:

Repository policy read: Root AGENTS.md, extensions/AGENTS.md, src/plugin-sdk/AGENTS.md, and the Telegram maintainer note were read and applied to the plugin-boundary, SDK import, and Telegram-visible-proof review. (AGENTS.md:1, 2c7e989686ba)
Current main still has the OpenAI MP3 bug: Current main resolves the configured responseFormat but only marks voice-note synthesis compatible when the format is Opus, so configured MP3 remains non-voice-compatible before Telegram delivery sees it. (extensions/openai/speech-provider.ts:371, 2c7e989686ba)
Latest release still has the same behavior: v2026.6.11 contains the same Opus-only voiceCompatible condition, so this PR is not obsolete due to a shipped fix. (extensions/openai/speech-provider.ts:371, e085fa1a3ffd)
Shared media contract accepts MP3 voice audio: The shared helper accepts .mp3 plus audio/mpeg and audio/mp3 for voice-message-compatible media, supporting the PR's shared-helper approach. (src/media/audio.ts:6, 2c7e989686ba)
Speech-core depends on provider voice metadata: Speech-core delivers a voice-note target as native voice only when provider metadata has voiceCompatible=true or the channel transcodes, so the OpenAI false value causes fallback routing. (packages/speech-core/src/tts.ts:1000, 2c7e989686ba)
Telegram already has MP3 sendVoice coverage: The Telegram send tests include an asVoice MP3 case routed through sendVoice, so the missing piece is provider metadata rather than Telegram transport behavior. (extensions/telegram/src/send.test.ts:2453, 2c7e989686ba)

Likely related people:

neeravmakwana: GitHub commit metadata shows this author added the OpenAI response-format handling that left the voice-note compatibility check Opus-only. (role: introduced current compatibility gate; confidence: high; commits: eb4bc200d788; files: extensions/openai/speech-provider.ts, extensions/openai/speech-provider.test.ts)
obviyus: GitHub commit metadata lists this account as committer for the commit that added the current OpenAI response-format compatibility gate. (role: committer of implicated OpenAI change; confidence: medium; commits: eb4bc200d788; files: extensions/openai/speech-provider.ts, extensions/openai/speech-provider.test.ts)
azade-c: This account authored the commit that added MP3 and M4A to the shared voice-compatible audio helper reused by the proposed fix. (role: introduced shared media compatibility behavior; confidence: high; commits: 1b95220a99e0; files: src/media/audio.ts, src/media/audio.test.ts)
steipete: Recent history shows this account centralized voice compatibility policy and speech voice-note routing around the helper and speech-core paths involved here. (role: speech and media routing contributor; confidence: medium; commits: 6ebf503fa8d2, f621fb4abaf3; files: src/media/audio.ts, packages/speech-core/src/tts.ts, packages/speech-core/src/tts.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

HemantSudarshan · 2026-05-17T21:36:08Z

@clawsweeper re-review

Rebased onto latest main, reran the OpenAI-compatible local HTTP proof on b7d0f62cb9e636b908daacee0f340274aa2594b2, and updated the PR body with the green GitHub Actions run numbers.

clawsweeper · 2026-05-17T21:37:42Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Complete
Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
Run: https://github.com/openclaw/clawsweeper/actions/runs/26003426113
Updated: 2026-05-17T21:44:45.804Z

steipete · 2026-07-06T07:34:34Z

Landed the cleaned-up fix in 9302c80 via #100715.

The maintained version uses the shared voice-message compatibility helper, keeps WAV on the audio-file path, adds focused WAV/MP3 regression coverage, and preserves your contributor credit. I recreated it because this fork branch was not editable by maintainers and had conflicts with current main.

Thank you, @HemantSudarshan. Enabling Allow edits by maintainers on future pull requests lets us update the original branch directly.

openclaw-barnacle Bot added extensions: openai size: S triage: mock-only-proof Candidate: PR proof only shows tests, mocks, snapshots, lint, typecheck, or CI. labels May 17, 2026

clawsweeper Bot added mantis: telegram-visible-proof Mantis should capture Telegram visible proof. P2 Normal backlog priority with limited blast radius. labels May 17, 2026

HemantSudarshan force-pushed the Hemant/fix-openai-tts-mp3-voice branch 3 times, most recently from e8aa071 to 04a13bc Compare May 17, 2026 19:23

clawsweeper Bot added the impact:message-loss Channel message delivery can be lost, duplicated, or misrouted. label May 17, 2026

HemantSudarshan force-pushed the Hemant/fix-openai-tts-mp3-voice branch from 04a13bc to d67894a Compare May 17, 2026 20:41

openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: mock-only-proof Candidate: PR proof only shows tests, mocks, snapshots, lint, typecheck, or CI. labels May 17, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 17, 2026

HemantSudarshan force-pushed the Hemant/fix-openai-tts-mp3-voice branch from d67894a to b7d0f62 Compare May 17, 2026 21:27

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 17, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 17, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 17, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 17, 2026

HemantSudarshan force-pushed the Hemant/fix-openai-tts-mp3-voice branch from b7d0f62 to 46953d2 Compare May 18, 2026 02:04

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 18, 2026

clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels May 18, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 18, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 18, 2026

fix(openai): mark mp3 tts voice-compatible

7d6773b

HemantSudarshan force-pushed the Hemant/fix-openai-tts-mp3-voice branch from 46953d2 to 7d6773b Compare May 18, 2026 02:35

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 18, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 18, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 18, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 18, 2026

clawsweeper Bot mentioned this pull request May 25, 2026

TTS OpenAI provider: MP3 responseFormat not voice-compatible for Telegram, unlike Edge TTS #80317

Closed

clawsweeper Bot mentioned this pull request Jun 11, 2026

[Feature] Configurable thinking/reasoning block format per channel #8913

Open

steipete self-assigned this Jul 6, 2026

steipete mentioned this pull request Jul 6, 2026

fix(openai): route MP3 TTS through voice delivery #100715

Merged

steipete closed this Jul 6, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(openai): mark mp3 TTS voice output compatible#83227

fix(openai): mark mp3 TTS voice output compatible#83227
HemantSudarshan wants to merge 1 commit into
openclaw:mainfrom
HemantSudarshan:Hemant/fix-openai-tts-mp3-voice

HemantSudarshan commented May 17, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented May 17, 2026 •

edited

Loading

Uh oh!

HemantSudarshan commented May 17, 2026

Uh oh!

clawsweeper Bot commented May 17, 2026 •

edited

Loading

Uh oh!

steipete commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

HemantSudarshan commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem summary

Root cause

Architectural reasoning

Testing performed

Real behavior proof

Risk analysis

Backward compatibility

Uh oh!

clawsweeper Bot commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HemantSudarshan commented May 17, 2026

Uh oh!

clawsweeper Bot commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

HemantSudarshan commented May 17, 2026 •

edited

Loading

clawsweeper Bot commented May 17, 2026 •

edited

Loading

clawsweeper Bot commented May 17, 2026 •

edited

Loading