Skip to content

[voice-call] OpenAI realtime: outbound calls greet caller twice — triggerGreeting + server_vad.create_response both fire response.create #85846

Description

@jnikolaidis

Summary

On outbound calls using plugins.entries.voice-call.config.realtime.provider: "openai", the assistant greets the caller twice. Reproducible on every outbound call that passes a non-empty --message (or any initialMessage).

The model emits two distinct responses back-to-back:

  1. First: a generic "Hi, how may I help you today?" / "Hi John, how can I help?" — generated from the model's own auto-response on detected user audio
  2. Second: a Milly-flavored greeting derived from the initialGreetingInstructions payload — generated from the plugin's explicit triggerGreeting() call

Result is jarring and breaks the caller's mental model (they answered ONE call, but the agent introduces itself twice with different wording).

Reproduction

OpenClaw 2026.5.20, @openclaw/[email protected], provider: openai, model: gpt-realtime-2, Twilio Voice as carrier, outbound CLI call:

openclaw voicecall call -t "+15551234567" --mode conversation \
  -m "Hi! Just a quick test of the voice setup. How are you doing today?"

Caller picks up, says "hello". The model then speaks both greetings back-to-back (~1s gap). Confirmed across 5 consecutive test calls with varying -m content. Disappears when initialGreetingInstructions is set to undefined at extensions/voice-call/src/webhook/realtime-handler.ts:1176 (local patch — model then only emits one greeting on caller's first audio).

Root cause

Two independent code paths fire response.create on call start when both an initialMessage is queued AND OpenAI's server VAD is enabled (the default):

Path 1 — OpenAI server VAD auto-response

extensions/openai/realtime-voice-provider.ts:763,782:

const autoRespondToAudio = cfg.autoRespondToAudio ?? true;  // default TRUE
...
turn_detection: {
  type: "server_vad",
  ...
  create_response: autoRespondToAudio,    // OpenAI side will auto-fire response.create on detected turn end
  interrupt_response: interruptResponseOnInputAudio,
}

When the caller says "hello", OpenAI's server VAD detects turn end and OpenAI automatically generates a response without us asking — that's the first greeting.

Path 2 — Plugin's explicit triggerGreeting

extensions/voice-call/src/webhook/realtime-handler.ts:497 and :665:

triggerGreetingOnReady: Boolean(initialGreetingInstructions),
...
bridge.triggerGreeting(instructions);

triggerGreeting() in the OpenAI provider (extensions/openai/realtime-voice-provider.ts:414-418) sends a conversation.item.create + response.create to make the model speak the supplied greeting — that's the second greeting.

Both responses are independently generated; the model just emits them back-to-back as the user perceives them.

Suggested fix

Pick one source of truth for the initial greeting in the OpenAI provider when an initialMessage is queued:

Option A (lowest blast radius): in realtime-handler.ts, when provider === "openai" AND autoRespondToAudio !== false, skip triggerGreeting. Rely on the server-VAD auto-response on caller's first audio — model uses system instructions for an appropriate opener.

Option B: when triggerGreeting will fire, send a session.update immediately before it with turn_detection.create_response: false, then re-enable after the first response completes. Keeps the plugin in charge of opening line, suppresses OpenAI's auto-response just for the first turn.

Option C (cleanest, biggest change): expose autoRespondToAudio in the validated config schema at extensions/openai/realtime-voice-provider.ts config parser so operators can set autoRespondToAudio: false in plugins.entries.voice-call.config.realtime.providers.openai. Currently the field is read by the provider but dropped by the schema validator (asObjectRecord(...) doesn't preserve it), so setting it in config has no effect.

Gemini provider (extensions/google/realtime-voice-provider.ts) doesn't have this issue because Google Live's activityHandling is start-of-activity-interrupts, not auto-respond.

Environment

  • OpenClaw 2026.5.20 (gateway)
  • @openclaw/[email protected]
  • macOS 25.5.0 (Mac Mini M4 Pro)
  • Twilio Voice (carrier) → Cloudflare Tunnel → plugin on 127.0.0.1:3334
  • Tested with gpt-realtime-2 model, voice: shimmer, silenceDurationMs: 250, prefixPaddingMs: 200

Workaround currently in production

Local patch at realtime-handler.ts:1176: replace

initialGreetingInstructions: buildGreetingInstructions(this.config.instructions, initialGreeting),

with

initialGreetingInstructions: undefined,

This breaks Google realtime greetings (Google needs the explicit trigger), so it's not a portable fix — needs the provider-aware solution above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions