Skip to content

bug(ios/talk): Gateway speech TTS provider is treated as realtime and falls back to iOS system voice #98153

Description

@Tony-ooo

Summary

iOS Talk Mode appears to treat a Gateway speech/TTS provider such as xiaomi as a realtime voice configuration. With the iOS Provider picker set to Gateway Default, the settings UI shows Voice Mode: Realtime Voice and Transport: Native WebRTC. Starting Talk then attempts a realtime client session and falls back to iOS Speech/System TTS instead of using the Gateway-configured talk.speak provider.

This prevents a configured Gateway TTS provider, verified here with Xiaomi MiMo TTS, from being used by iOS Talk Mode.

Environment

  • OpenClaw CLI/Gateway: OpenClaw 2026.6.10 (aa69b12)
  • iOS app source inspected at commit: 6cb82eaab8
  • Gateway Talk provider: xiaomi
  • iOS settings:
    • Provider: Gateway Default
    • Realtime Voice: Gateway Default
    • UI shows Voice Mode: Realtime Voice
    • UI shows Transport: Native WebRTC

Gateway evidence

talk.config returns only the speech/TTS provider as Xiaomi; there is no talk.realtime block in the local config:

{
  "talk": {
    "interruptOnSpeech": true,
    "silenceTimeoutMs": 1200,
    "speechLocale": "zh-CN",
    "provider": "xiaomi",
    "providers": {
      "xiaomi": {
        "apiKey": "__OPENCLAW_REDACTED__",
        "baseUrl": "https://token-plan-cn.xiaomimimo.com/v1",
        "model": "mimo-v2.5-tts",
        "speakerVoice": "mimo_default",
        "format": "mp3"
      }
    },
    "resolved": {
      "provider": "xiaomi",
      "config": {
        "apiKey": "__OPENCLAW_REDACTED__",
        "baseUrl": "https://token-plan-cn.xiaomimimo.com/v1",
        "model": "mimo-v2.5-tts",
        "speakerVoice": "mimo_default",
        "format": "mp3",
        "voice": "mimo_default",
        "voiceName": "mimo_default"
      }
    }
  }
}

talk.catalog also reports:

{
  "speech": {
    "activeProvider": "xiaomi",
    "providers": [
      {
        "id": "xiaomi",
        "label": "Xiaomi MiMo",
        "configured": true,
        "modes": ["stt-tts"]
      }
    ]
  },
  "realtime": {
    "providers": [
      {
        "id": "openai",
        "configured": false,
        "defaultModel": "gpt-realtime-2"
      }
    ]
  }
}

The Gateway-side talk.speak path for Xiaomi was also tested successfully: it returned provider: xiaomi, outputFormat: mp3, mimeType: audio/mpeg, and non-empty audioBase64.

Actual behavior

  1. Configure Gateway Talk provider as Xiaomi.
  2. In iOS settings, set Talk Provider to Gateway Default.
  3. iOS settings still display Voice Mode: Realtime Voice and Transport: Native WebRTC.
  4. Starting Talk attempts realtime (talk.client.create) and can show an error like:
talk.client.create: [UNAVAILABLE] Error: Realtime voice provider "openai" is not configured
model: gpt-realtime-2
transport: webrtc
  1. Talk falls back to Using iOS Speech fallback. The assistant can answer, but playback uses iOS system voice rather than the configured Xiaomi TTS provider.

Expected behavior

When talk.provider resolves to a speech/TTS provider such as xiaomi with mode stt-tts, iOS native Talk should stay on the native STT -> Gateway chat -> talk.speak TTS path. It should not be classified as realtime merely because the provider is not ElevenLabs.

If realtime setup fails or is not configured, the fallback playback path should prefer Gateway talk.speak for the active Gateway Talk provider before falling back to app-local ElevenLabs or iOS system TTS.

This also matches the docs in docs/nodes/talk.md, which say native iOS/macOS/Android Talk uses local speech recognition, Gateway chat, and talk.speak TTS.

Suspected source-level cause

In apps/ios/Sources/Voice/TalkModeManager.swift, the iOS default provider is hard-coded as ElevenLabs:

https://github.com/openclaw/openclaw/blob/6cb82eaab8/apps/ios/Sources/Voice/TalkModeManager.swift#L36-L38

Then any active Gateway provider that is not this default is treated as realtime:

https://github.com/openclaw/openclaw/blob/6cb82eaab8/apps/ios/Sources/Voice/TalkModeManager.swift#L2654-L2684

Specifically:

let usesRealtimeConfig = activeProvider != Self.defaultTalkProvider || executionMode != .native
self.realtimeWebRTCEnabled = usesRealtimeConfig

For activeProvider = "xiaomi", this makes usesRealtimeConfig = true, which explains the Realtime Voice / Native WebRTC UI and the realtime start attempt.

A second issue is that the iOS native fallback playback function appears to only try app-local ElevenLabs and then iOS system speech. I did not find a Gateway talk.speak call in playAssistant(text:):

https://github.com/openclaw/openclaw/blob/6cb82eaab8/apps/ios/Sources/Voice/TalkModeManager.swift#L1517-L1630

Suggested fix

  • Determine realtime mode from talk.realtime.mode == "realtime" / transport, or from the provider catalog mode, not from activeProvider != "elevenlabs".
  • Treat speech.providers[*].modes containing stt-tts as native Talk playback providers.
  • In iOS native/fallback playback, prefer Gateway talk.speak for the active Gateway speech provider and only then fall back to app-local ElevenLabs or iOS system TTS.

Notes

No secrets are included here; API keys are redacted as __OPENCLAW_REDACTED__.

Metadata

Metadata

Assignees

Labels

P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.no-staleExclude from stale automation

Type

No type

Fields

Priority

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions