Summary
OpenClaw's bundled xAI provider supports buffered TTS through the REST /v1/tts path, but the provider does not currently expose xAI's native streaming TTS WebSocket contract to OpenClaw streaming playback callers.
This leaves streaming-capable surfaces, such as Discord voice paths that use textToSpeechStream, unable to use xAI's lower-latency streamed audio path even when xAI credentials and normal buffered xAI TTS are configured.
Desired behavior
Add native xAI streaming TTS support for OpenClaw's shared speech provider streaming contract:
- Use xAI's documented WebSocket streaming TTS endpoint:
wss://api.x.ai/v1/tts.
- Send
text.delta followed by text.done.
- Read
audio.delta chunks until audio.done.
- Preserve the existing buffered
synthesize path for normal TTS attachments and talk.speak.
- Keep this separate from xAI realtime voice / Talk gateway-relay, which uses
wss://api.x.ai/v1/realtime and has a different session contract.
Scope
In scope:
messages.tts.provider: "xai" streaming support for textToSpeechStream callers.
- Existing OpenClaw TTS controls: language, voice, codec/response format, and speed.
- Native xAI WebSocket endpoint only, to avoid silently applying the xAI streaming contract to arbitrary REST-compatible base URLs.
- Focused unit tests for URL shape, event protocol, lifecycle/release behavior, timeout behavior, and
streamSynthesize integration.
Out of scope:
- xAI realtime voice / Talk gateway-relay provider.
- STT changes.
- Voice Call realtime/managed-room behavior.
- New
sample_rate / bit_rate config knobs. xAI supports them, but this first pass uses provider defaults rather than extending the OpenClaw config surface.
Why this matters
For real voice use, buffered TTS adds avoidable latency because playback waits for a complete synthesized buffer. xAI's streaming TTS API can start delivering audio chunks earlier, which is important for lower-cost and lower-latency voice setups.
This also lets OpenClaw use xAI for streaming playback without overloading the separate realtime voice feature scope.
Proof target
Minimum acceptable proof for a PR:
- Focused tests for
extensions/xai/tts.ts and extensions/xai/speech-provider.ts.
- Docs clearly distinguishing streaming TTS from realtime voice.
- Security posture explaining why streaming WebSocket is restricted to the native xAI host.
- If live credentials are available, a redacted smoke showing
text.delta / text.done -> audio.delta / audio.done and non-empty audio bytes.
Related context
- Existing xAI TTS provider support exists for buffered REST TTS.
- xAI realtime voice / Talk gateway-relay is tracked separately and should not be conflated with this feature.
Summary
OpenClaw's bundled xAI provider supports buffered TTS through the REST
/v1/ttspath, but the provider does not currently expose xAI's native streaming TTS WebSocket contract to OpenClaw streaming playback callers.This leaves streaming-capable surfaces, such as Discord voice paths that use
textToSpeechStream, unable to use xAI's lower-latency streamed audio path even when xAI credentials and normal buffered xAI TTS are configured.Desired behavior
Add native xAI streaming TTS support for OpenClaw's shared speech provider streaming contract:
wss://api.x.ai/v1/tts.text.deltafollowed bytext.done.audio.deltachunks untilaudio.done.synthesizepath for normal TTS attachments andtalk.speak.wss://api.x.ai/v1/realtimeand has a different session contract.Scope
In scope:
messages.tts.provider: "xai"streaming support fortextToSpeechStreamcallers.streamSynthesizeintegration.Out of scope:
sample_rate/bit_rateconfig knobs. xAI supports them, but this first pass uses provider defaults rather than extending the OpenClaw config surface.Why this matters
For real voice use, buffered TTS adds avoidable latency because playback waits for a complete synthesized buffer. xAI's streaming TTS API can start delivering audio chunks earlier, which is important for lower-cost and lower-latency voice setups.
This also lets OpenClaw use xAI for streaming playback without overloading the separate realtime voice feature scope.
Proof target
Minimum acceptable proof for a PR:
extensions/xai/tts.tsandextensions/xai/speech-provider.ts.text.delta/text.done->audio.delta/audio.doneand non-empty audio bytes.Related context