Summary

Adds Telnyx Media Streaming (PCMU 8 kHz μ-law) support to @openclaw/voice-call so realtime voice providers (OpenAI Realtime, etc.) can drive Telnyx calls the same way they already drive Twilio calls. Until now, realtime.enabled was rejected at config time for any non-Twilio provider, and the Telnyx provider had zero streaming code — extensions/voice-call/src/config.ts gated realtime to Twilio explicitly.

This closes that parity gap for the realtime path.

Why

The voice-call plugin's realtime bridge in extensions/voice-call/src/webhook/realtime-handler.ts is provider-agnostic in shape (it consumes a RealtimeVoiceProviderPlugin from openclaw/plugin-sdk/realtime-voice for the model side), but its WebSocket frame parsing and outbound serialization were inlined and Twilio-shaped. The Twilio provider attaches Media Streams via TwiML; Telnyx had no equivalent kickoff and no frame plumbing. Result: anyone configuring provider: "telnyx" with realtime.enabled: true hit a config-time error and never got past the gate.

Closes the gap by:

1. Extracting WebSocket frame parsing/serialization into a small StreamFrameAdapter interface with one implementation per carrier (Twilio retains streamSid; Telnyx omits it).
2. Generalizing the audio pacer to accept any serializer.
3. Wiring Telnyx Media Streaming kickoff into the provider's dial and answer-action payloads per Telnyx's documented canonical pattern (no actions/streaming_start call needed).
4. Threading a stream session issuer from the realtime handler through the manager so carriers that attach streaming at dial/answer time can embed the URL + token in their carrier API payloads.
5. Widening the realtime config gate from provider === "twilio" to allow telnyx as well.

PCMU-only for the first cut. L16 / wideband is out of scope (libsamplerate-js resampling is a separate addition).

Design decisions (locked at plan time)

• Codec: PCMU 8 kHz μ-law only. 20 ms / 160-byte frames, base64 passthrough between Telnyx and OpenAI Realtime (which accepts audio/pcmu natively per extensions/openai/realtime-voice-provider.ts:198). No resampling on the hot path.
• Kickoff: dial-time inline for outbound (POST /v2/calls payload includes stream_url, stream_codec: "PCMU", stream_bidirectional_* fields, stream_auth_token), and answer-action inline for inbound (POST /calls/{id}/actions/answer includes the same fields). Mirrors Telnyx's documented "AI agent" pattern and avoids the "one streaming operation per call" constraint that an explicit actions/streaming_start would impose.
• Frame layer: per-provider StreamFrameAdapter in extensions/voice-call/src/webhook/stream-frame-adapter.ts. Realtime handler stays single-implementation; new providers can drop in by writing one adapter.
• Provider boundary: the Telnyx provider only embeds streaming params into carrier API payloads — it never composes the stream URL itself. The realtime handler owns URL composition via the new issueStreamSession method; manager calls it via a thin streamSessionIssuer hook (Twilio path is unchanged since Twilio learns the URL from TwiML).
• Scope: outbound + inbound realtime. Streaming-transcription path (streaming.enabled) for Telnyx is a follow-up; the new frame adapter makes it a smaller diff.

Files of interest

• extensions/voice-call/src/webhook/stream-frame-adapter.ts (new) — StreamFrame union + StreamFrameAdapter interface + Twilio/Telnyx implementations
• extensions/voice-call/src/webhook/realtime-handler.ts — issueStreamSession extracted, WS upgrade dispatches via adapter
• extensions/voice-call/src/webhook/realtime-audio-pacer.ts — generalized to accept a serializer (was Twilio-shaped)
• extensions/voice-call/src/providers/telnyx.ts — initiateCall + answerCall embed buildTelnyxStreamingFields(streamUrl, token) when streaming is requested; new webhook events call.streaming.failed → non-retryable call.error (streaming.started/stopped acknowledged and dropped to avoid duplicate manager signal)
• extensions/voice-call/src/providers/base.ts — optional setPublicUrl?(url) on the provider interface; streamUrl? and streamAuthToken? added to InitiateCallInput / AnswerCallInput in extensions/voice-call/src/types.ts
• extensions/voice-call/src/manager/context.ts + extensions/voice-call/src/manager.ts — streamSessionIssuer hook on the manager context
• extensions/voice-call/src/manager/outbound.ts + extensions/voice-call/src/manager/events.ts — call the issuer before delegating to the provider when realtime is enabled and the provider is Telnyx
• extensions/voice-call/src/runtime.ts — wires provider.setPublicUrl?.(publicUrl) generically (was Twilio-cast), and connects manager.streamSessionIssuer to realtimeHandler.issueStreamSession
• extensions/voice-call/src/config.ts — realtime gate at lines 871-874 widened to accept telnyx alongside twilio

Real behavior proof

Behavior or issue addressed: Realtime voice mode (realtime.enabled: true) previously required provider: "twilio" — validateProviderConfig in extensions/voice-call/src/config.ts:871-874 explicitly rejected any non-Twilio provider, and the Telnyx provider had zero Media-Streams code (no stream_url plumbing on dial/answer, no inbound WebSocket frame parser, no outbound frame serializer). Users on Telnyx had no path to sub-second voice latency over a carrier call. This PR wires Telnyx Media Streaming (PCMU 8 kHz μ-law) into the existing RealtimeCallHandler bridge that Twilio already uses by adding a per-provider StreamFrameAdapter (Twilio retains streamSid, Telnyx omits it), generalizing the audio pacer to accept any serializer, embedding the streaming params on the Telnyx dial / answer-action payload per Telnyx's documented "AI agent" canonical pattern, and widening the realtime config gate to accept telnyx.

Real environment tested: OpenClaw gateway 2026.5.10-beta.1 running locally on macOS (Darwin 24.6.0) via pnpm gateway:watch (Node 24.14.1, pnpm via Corepack). Patched voice-call plugin with provider: "telnyx", realtime.enabled: true, realtime.provider: "openai", realtime.providers.openai: {} (OpenAI Realtime audio format pinned to audio/pcmu on both legs per extensions/openai/realtime-voice-provider.ts:198), realtime.toolPolicy: "owner", realtime.consultPolicy: "substantive", realtime.consultFastMode: true, realtime.consultThinkingLevel: "low", outbound.defaultMode: "conversation", serve.port: 3421, serve.path: "/webhook". Bidirectional WebSocket tunneled through a reserved ngrok domain (noc-telnyx-eu.ngrok.io) wired by tunnel.provider: "ngrok". Telnyx Voice API Application webhook URL set to https://<ngrok-host>/webhook. Outbound call placed from a real Telnyx-managed number on my account to a real PSTN mobile destination I own (numbers redacted). Secrets sourced from ~/.secrets.zsh via ~/.zshenv: TELNYX_API_KEY, TELNYX_CONNECTION_ID, TELNYX_PUBLIC_KEY, OPENAI_API_KEY, NGROK_AUTHTOKEN.

Exact steps or command run after this patch:

1. Restart the gateway watch session so the patched plugin loads cleanly: tmux kill-session -t openclaw-gateway-watch-main && OPENCLAW_TRACE_SYNC_IO=0 pnpm gateway:watch.
2. Confirm realtime is wired: tmux capture-pane -t openclaw-gateway-watch-main:0.0 -p -S -100 | grep -E "Realtime voice provider|ngrok tunnel active|Runtime initialized" should show the ngrok tunnel active line, [plugins] [voice-call] Realtime voice provider: openai, and [plugins] [voice-call] Runtime initialized.
3. Place a live outbound call to the test destination: pnpm openclaw voicecall call --message "Hi, can you hear me?".
4. Pick up the destination phone and speak — the bot greets, listens, replies; speaking over the bot exercises barge-in.
5. Tail the gateway during and after the call: tmux capture-pane -t openclaw-gateway-watch-main:0.0 -p -S -3000 | grep -E "voice-call|realtime|streaming".
6. Cross-reference Telnyx Mission Control Portal call detail and the ngrok inspector at http://127.0.0.1:4040 for the matching call.initiated → call.answered → streaming.started → call.hangup webhook sequence and the inbound WebSocket upgrade to /stream/realtime/<token>.
7. Confirm pre-PR gates: pnpm test:extension voice-call (45 files / 396 tests), pnpm test:contracts (62 files / 834 tests), pnpm tsgo:extensions, pnpm tsgo:extensions:test, all three import-boundary scripts under scripts/, pnpm exec oxfmt --check on touched files, pnpm exec node scripts/run-oxlint.mjs extensions/voice-call/src, pnpm build — all green locally.

Evidence after fix: Redacted runtime logs captured live from pnpm gateway:watch stdout during a real outbound Telnyx call (callId 2eabfaa1-3416-4677-96c7-71a131ba0e15, providerCallId v3:kPoU84kZj-..., destination redacted). The Realtime bridge starting line is the existing RealtimeCallHandler.handleCall accepting Telnyx's start.call_control_id-shaped frame through the new TelnyxStreamFrameAdapter (extensions/voice-call/src/webhook/stream-frame-adapter.ts). The realtime input transcript lines confirm OpenAI Realtime is transcribing inbound PCMU audio from Telnyx live during the call. The realtime outbound audio cleared by barge-in lines confirm OpenAI Realtime produced outbound audio AND the speech-start detector fired on caller barge-in, clearing the outbound queue (which writes {event:"clear"} to Telnyx via the new adapter).

21:34:25 [voice-call] Outbound call initiated: callId=2eabfaa1-3416-4677-96c7-71a131ba0e15 providerCallId=v3:kPoU84kZj-... mode=conversation preConnectDtmf=no initialMessage=yes
21:34:25 [ws] res voicecall.initiate 1077ms
21:34:48 [voice-call] Starting max duration timer (300s) for call 2eabfaa1-3416-4677-96c7-71a131ba0e15
21:34:48 [voice-call] Realtime bridge starting for call 2eabfaa1-3416-4677-96c7-71a131ba0e15 (providerCallId=v3:kPoU84kZj-..., initialGreeting=queued)
21:34:58 [voice-call] realtime input transcript callId=2eabfaa1-... providerCallId=v3:kPoU84kZj-... final=false chars=14 aggregateChars=14
21:35:04 [voice-call] realtime outbound audio cleared by barge-in callId=2eabfaa1-... providerCallId=v3:kPoU84kZj-... queuedBytes=1280
21:35:04 [voice-call] realtime outbound audio clear requested callId=2eabfaa1-... providerCallId=v3:kPoU84kZj-... queuedBytes=0
21:35:06 [voice-call] realtime outbound audio cleared by barge-in callId=2eabfaa1-... providerCallId=v3:kPoU84kZj-... queuedBytes=2400
21:35:07 [voice-call] realtime input transcript callId=2eabfaa1-... providerCallId=v3:kPoU84kZj-... final=true chars=39 aggregateChars=39

The Telnyx-side webhook sequence confirmed end-to-end during the same call: call.initiated → call.answered → streaming.started (bidirectional WebSocket opened by the carrier to our wss endpoint) → media frames flowing both ways → call.hangup on call end. A separate live test with realtime.toolPolicy: "owner" plus a substantive caller question exercised the agent-consult round trip — openclaw_agent_consult fired and runEmbeddedPiAgent ran on the carrier leg.

Observed result after fix: Outbound Telnyx phone calls connect to the realtime bridge over the bidirectional PCMU WebSocket. Destination hears OpenAI Realtime's spoken greeting within ~1 second of pickup. Conversation runs full-duplex with sub-second voice-turn latency over the carrier leg. Barge-in works: when the caller speaks over the bot, the speech-start detector clears the outbound audio queue, writing {event:"clear"} to Telnyx via the new adapter. Final transcripts land via the existing call.speech event path and flow through to openclaw_agent_consult when the model decides to consult. Twilio realtime path was not regressed during this session — the only Twilio-side surface changed is the audio-pacer rename plus the frame-adapter extraction; both have round-trip tests asserting the Twilio envelope shape is byte-identical to pre-PR.

What was not tested: Inbound Telnyx realtime call to a Telnyx-owned number — inboundPolicy: "allowlist" config plus the answer-action streaming-field embedding are wired and unit-tested, but a real PSTN inbound call into the test number was not exercised this session; L16 / wideband codec (PCMU only here; the team-telnyx/realtime-ai-demo reference shows L16 needs libsamplerate-js resampling 16↔24 kHz which is intentionally out of scope, and the adapter is shaped so L16 is a drop-in follow-up); Plivo or Mock realtime (Plivo provider does not implement Media Streaming and Mock has no streaming at all; the config gate continues to reject realtime for any provider that is not twilio or telnyx, so misconfiguration fails fast); high-concurrency / load (single-call manual smoke only; stale-call reaper and maxConcurrentCalls unchanged from upstream and not stress-tested); Telnyx {event:"error"} WS frame real-world trigger (handled in the new TelnyxStreamFrameAdapter and unit-tested with a synthesized payload, but a real failure such as a revoked auth token or mid-stream network drop was not reproduced live this session).

Automated checks

CI=true pnpm test:extension voice-call           # 45 files / 394 tests pass
CI=true pnpm tsgo:extensions                     # exit 0
CI=true pnpm tsgo:extensions:test                # exit 0

New / extended tests cover:

• extensions/voice-call/src/webhook/stream-frame-adapter.test.ts (new) — Twilio + Telnyx frame round-trips (parse + serialize), ignored frames, malformed JSON
• extensions/voice-call/src/webhook/realtime-audio-pacer.test.ts — adapter-shaped serializer paths for both Twilio and Telnyx envelope shapes; barge-in clear; backpressure
• extensions/voice-call/src/providers/telnyx.test.ts — streaming fields present/absent in initiateCall and answerCall; call.streaming.failed → call.error; call.streaming.started/stopped acknowledged
• extensions/voice-call/src/manager/outbound.test.ts — streamSessionIssuer invoked for Telnyx + realtime; not invoked for Twilio; not invoked when realtime is disabled
• extensions/voice-call/src/config.test.ts — realtime + telnyx accepted; realtime + other providers rejected with the updated error message

Test plan (reviewer checklist)

• pnpm test:extension voice-call green locally
• pnpm tsgo:extensions + pnpm tsgo:extensions:test green
• Confirm stream-frame-adapter.ts Twilio adapter produces wire-identical output to the previous Twilio path (key assertion: outbound media/clear/mark frames still include streamSid)
• Smoke an outbound Telnyx realtime call against a real number (the path proven in this PR)
• Optional: smoke an inbound Telnyx realtime call by dialing INTO a Telnyx number associated with the configured Voice API Application

Known limitations / out of scope

• PCMU only. L16 / 16 kHz wideband and the libsamplerate-js resampler from the team-telnyx/realtime-ai-demo are deliberately deferred. The adapter design leaves L16 as a drop-in.
• Telnyx + streaming.enabled transcription path is not wired here. That's a separate carrier-shape job that reuses the same adapter abstraction; tracked as follow-up.
• stream_track: "inbound_track" + bidirectional via stream_bidirectional_mode: "rtp" is the only flow exercised. Other Telnyx track modes (outbound_track, both_tracks) aren't tested in this PR.
• stream_bidirectional_target_legs: "self" is hardcoded. Per Telnyx-side notes from the reference demo, "opposite" silently drops audio — "self" is the safe canonical choice for a single carrier leg. Re-evaluate when bridge / conference flows arrive.

Backwards compatibility

• Twilio realtime path is unchanged. Twilio learns its stream URL from TwiML exactly as before; RealtimeCallHandler.buildTwiMLPayload now delegates URL composition to the extracted issueStreamSession helper but the resulting TwiML is byte-identical.
• The RealtimeTwilioAudioPacer class is renamed to RealtimeAudioPacer (generalized). External consumers were checked — no references outside extensions/voice-call/ (verified via grep).
• The VoiceCallProvider.setPublicUrl? method is optional — providers that don't implement it (Telnyx, mock) get a no-op.

Resources

• team-telnyx/realtime-ai-demo — Telnyx-published reference that bridges Telnyx Media Streaming to OpenAI Realtime; the PCMU profile in this PR mirrors its frame envelope, codec choice, and dial-time inline kickoff.
• Telnyx — Media Streaming over WebSockets — canonical frame format documentation; top-level stream_id, start.call_control_id, {event:"error"} WS frames.
• Telnyx — Dial Call API — stream_url + stream_codec + stream_bidirectional_* fields this PR embeds at dial time.
• Telnyx — Streaming Start API — actions/streaming_start reference for the late-attach case; relevant context for feat(voice-call): bidirectional Telnyx Media Streams for realtime voice #79575.
• Telnyx — Bi-directional streaming release notes — explains stream_bidirectional_mode: "rtp" plus the stream_bidirectional_target_legs semantics used here.
• Twilio — Media Streams and <Stream> TwiML — Twilio side of the same generic bridge for reference.

AI-assisted disclosure

This PR was drafted with assistance from Claude (Anthropic). I reviewed every code change line-by-line, ran tests locally, and performed the live verification described above. The exact session prompts and design decisions are available on request.