Skip to content

fix(voice-call): avoid duplicate OpenAI initial greetings#86502

Closed
wyf027 wants to merge 4 commits into
openclaw:mainfrom
wyf027:leno23/fix-openai-voice-initial-greeting-85846
Closed

fix(voice-call): avoid duplicate OpenAI initial greetings#86502
wyf027 wants to merge 4 commits into
openclaw:mainfrom
wyf027:leno23/fix-openai-voice-initial-greeting-85846

Conversation

@wyf027

@wyf027 wyf027 commented May 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Avoid firing an explicit triggerGreeting() for OpenAI realtime outbound calls when OpenAI server VAD auto-response is active.
  • Fold the queued outbound initial greeting into the OpenAI session instructions so the provider-owned first auto-response still receives the caller-facing greeting text.
  • Preserve the explicit greeting path when autoRespondToAudio: false is configured, and now pass that opt-out through to the OpenAI bridge so server VAD create_response is disabled for that path.

Fixes #85846

Test Plan

  • RED: node scripts/run-vitest.mjs extensions/voice-call/src/webhook/realtime-handler.test.ts --testNamePattern 'keeps explicit greetings when OpenAI auto response is disabled' failed before the source fix with expected undefined to be false for createRequest?.autoRespondToAudio.
  • RED: node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts --testNamePattern "preserves explicit realtime auto-response opt-out during config resolution" failed before the provider-resolution fix with expected undefined to be false for resolveConfig(...).autoRespondToAudio.
  • GREEN: node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts --testNamePattern "preserves explicit realtime auto-response opt-out during config resolution" passed after the provider-resolution fix.
  • RED: node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts --testNamePattern "applies provider-configured realtime auto-response opt-out when creating bridges" failed before the bridge-creation fix with turn detection.create_response: expected true to deeply equal false.
  • GREEN: node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts --testNamePattern "applies provider-configured realtime auto-response opt-out when creating bridges" passed after the bridge-creation fix (1 passed | 50 skipped).
  • node scripts/run-vitest.mjs extensions/voice-call/src/webhook/realtime-handler.test.ts
  • node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts extensions/voice-call/src/webhook/realtime-handler.test.ts
  • pnpm check:changed --staged
  • Earlier on this branch: pnpm openclaw voicecall --help (runtime CLI/build smoke; exits because voice-call is disabled by default in this checkout).

Real behavior proof

Behavior addressed: OpenAI realtime outbound calls with a queued initial greeting no longer create two independent greetings when server VAD auto-response is active; the initial greeting is provided through session instructions and triggerGreeting() is suppressed for that OpenAI auto-response path. When autoRespondToAudio: false is configured, the voice-call handler keeps the explicit triggerGreeting() path and passes autoRespondToAudio: false into the OpenAI bridge so the provider session disables server VAD create_response.

Real environment tested: Local OpenClaw checkout on macOS with the voice-call realtime WebSocket handler exercised through the repository runtime/test harness, the OpenAI realtime provider bridge tests, and OpenClaw CLI runtime smoke from this branch.

Exact steps or command run after this patch: node scripts/run-vitest.mjs extensions/voice-call/src/webhook/realtime-handler.test.ts --testNamePattern 'keeps explicit greetings when OpenAI auto response is disabled' before the source fix; node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts --testNamePattern "preserves explicit realtime auto-response opt-out during config resolution" before and after the provider-resolution fix; node scripts/run-vitest.mjs extensions/voice-call/src/webhook/realtime-handler.test.ts; node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts extensions/voice-call/src/webhook/realtime-handler.test.ts; pnpm check:changed --staged; earlier pnpm openclaw voicecall --help.

Evidence after fix: The new provider-resolution RED assertion failed before the fix with expected undefined to be false, proving OpenAI resolveConfig dropped the configured autoRespondToAudio: false before the voice-call handler could read it. After the fix, that focused resolver test passed (1 passed | 49 skipped), and the combined OpenAI provider plus voice-call handler run passed (2 files, 71 tests passed). The changed gate reported lanes extensions, extensionTests, then completed typecheck extensions, typecheck extension tests, lint extensions, media helper guard, runtime sidecar loader guard, and runtime import-cycle checks successfully. The earlier CLI runtime smoke built OpenClaw and bundled plugin assets, then exited with the expected message that openclaw voicecall is provided by the disabled-by-default voice-call plugin. A follow-up bridge-creation RED assertion failed before this update with turn detection.create_response: expected true to deeply equal false; after the bridge fix, the focused createBridge test passed (1 passed | 50 skipped), and the combined OpenAI provider plus voice-call handler run passed (2 files, 71 tests passed).

Observed result after fix: OpenAI's default auto-response path receives the initial greeting in session instructions and the voice-call plugin no longer sends a second explicit greeting request. The disabled-auto-response path now sends exactly one explicit greeting request while carrying autoRespondToAudio: false to the OpenAI bridge so OpenAI server VAD does not auto-create a second response. Provider-level autoRespondToAudio: false now also reaches bridge creation when no per-request override is present.

What was not tested: A live Twilio/OpenAI phone call was not placed from this environment because external telephony/provider credentials are not available to this cron job; the after-fix proof uses local realtime WebSocket handler/provider bridge execution and CLI/build smoke rather than external telephony credentials.

Risk/Notes

  • Message-delivery behavior is intentionally changed only for OpenAI realtime outbound initial greetings.
  • Live carrier/provider proof still requires redacted Twilio/OpenAI runtime logs or a call artifact from an environment with those credentials.

@openclaw-barnacle openclaw-barnacle Bot added channel: voice-call Channel integration: voice-call size: S proof: supplied External PR includes structured after-fix real behavior proof. labels May 25, 2026
@clawsweeper

clawsweeper Bot commented May 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed May 30, 2026, 3:37 AM ET / 07:37 UTC.

Summary
The branch changes OpenAI realtime voice-call initial greetings to use server VAD auto-response instead of an explicit ready-time greeting when auto-response is active, preserves the autoRespondToAudio: false opt-out, and adds focused regression tests.

PR surface: Source +21, Tests +177. Total +198 across 4 files.

Reproducibility: yes. The linked report gives a concrete Twilio/OpenAI outbound-call repro, and current main source shows both triggerGreetingOnReady and OpenAI server VAD create_response: true active for a queued initial greeting.

Review metrics: 1 noteworthy metric.

  • OpenAI realtime config surface: 1 provider option activated. The PR makes autoRespondToAudio survive provider config normalization and bridge creation, so upgrade behavior and operator expectations need review before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR proof is tests/harness output and CLI smoke only; before merge the contributor should add redacted live logs, terminal output, a recording, or a linked artifact showing the real call behavior, then update the PR body or ask for re-review if needed.

Risk before merge

  • [P1] No redacted live Twilio/OpenAI call proof yet shows the patched path producing exactly one initial greeting in a real outbound call; the PR body explicitly says live call proof was not available.
  • [P1] The PR intentionally changes OpenAI realtime first-turn delivery from an explicit triggerGreeting() request to provider-owned server VAD auto-response when auto-response is enabled, so maintainers should choose this behavior against the open sibling implementation before merge.
  • [P1] The PR activates autoRespondToAudio: false in provider config that current main drops, which is useful for the bug but compatibility-sensitive for operators who already have that raw config present.

Maintainer options:

  1. Require live-call proof first (recommended)
    Before merge, require redacted Twilio/OpenAI logs, terminal output, a recording, or a linked artifact showing one initial greeting on the default path and no server auto-response when autoRespondToAudio: false is configured.
  2. Pick the canonical implementation
    Compare this server-VAD-instructions approach with fix(voice-call): avoid OpenAI realtime double greeting #86285, then land one path and close or fold in the other so OpenAI realtime greetings do not split into competing behaviors.
  3. Accept harness-only proof knowingly
    Maintainers could intentionally merge based on the targeted tests and own the live-carrier risk, but that should be an explicit decision because the changed behavior is audible to callers.

Next step before merge

  • [P1] Human PR handling is needed because the remaining blockers are live behavior proof, draft status, and maintainer choice between two open implementations rather than a narrow automated code repair.

Security
Cleared: The diff changes TypeScript runtime and tests only, with no dependency, CI, package, secret-handling, or supply-chain changes that create a concrete security concern.

Review details

Best possible solution:

Land one canonical fix after maintainers choose the desired first-greeting strategy and redacted live proof demonstrates both the default one-greeting path and the autoRespondToAudio: false opt-out path.

Do we have a high-confidence way to reproduce the issue?

Yes. The linked report gives a concrete Twilio/OpenAI outbound-call repro, and current main source shows both triggerGreetingOnReady and OpenAI server VAD create_response: true active for a queued initial greeting.

Is this the best way to solve the issue?

Unclear pending maintainer choice. The patch is a plausible narrow source fix with focused tests, but it should be compared with the sibling implementation and backed by live call proof before merge.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 0f8ea1d3d905.

Label changes

Label justifications:

  • P2: This is a normal-priority voice-call bug fix for a limited OpenAI realtime greeting path, not an emergency or broad runtime outage.
  • merge-risk: 🚨 compatibility: The branch starts honoring an OpenAI realtime provider config option that current main drops, which can change behavior for existing raw provider configs on upgrade.
  • merge-risk: 🚨 message-delivery: The branch changes which component produces the caller's first spoken greeting, so a bad merge could duplicate, suppress, delay, or alter initial voice-call delivery.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR proof is tests/harness output and CLI smoke only; before merge the contributor should add redacted live logs, terminal output, a recording, or a linked artifact showing the real call behavior, then update the PR body or ask for re-review if needed.
Evidence reviewed

PR surface:

Source +21, Tests +177. Total +198 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 23 2 +21
Tests 2 181 4 +177
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 204 6 +198

What I checked:

  • Current main duplicate-response source path: Current main always passes queued initialGreetingInstructions with triggerGreetingOnReady: Boolean(initialGreetingInstructions) into the realtime bridge session, so an outbound initial message triggers an explicit provider greeting when the bridge becomes ready. (extensions/voice-call/src/webhook/realtime-handler.ts:644, 0f8ea1d3d905)
  • Current main OpenAI auto-response default: Current main also defaults OpenAI realtime server VAD create_response to true, and OpenAI's official Realtime API reference says create_response automatically generates a response when a VAD stop event occurs. (platform.openai.com) (extensions/openai/realtime-voice-provider.ts:902, 0f8ea1d3d905)
  • Current main drops provider-level opt-out: Current main's OpenAI provider config normalization includes VAD and interruption fields but not autoRespondToAudio, and createBridge only passes request-level interruption config through, so raw provider config cannot disable OpenAI auto-response today. (extensions/openai/realtime-voice-provider.ts:203, 0f8ea1d3d905)
  • PR head repair shape: The hydrated PR diff for head eafcae962fde92b64e9bf8d443fa222b4427b5ce adds OpenAI-specific auto-greeting selection, passes autoRespondToAudio: false into bridge creation, preserves config-level opt-out in OpenAI normalization, and adds regression tests for the default and opt-out paths. (extensions/voice-call/src/webhook/realtime-handler.ts:651, eafcae962fde)
  • Proof gap from PR body: The PR body supplies RED/GREEN unit and harness output plus CLI smoke, but explicitly says no live Twilio/OpenAI phone call was placed and no redacted live call logs, recording, or linked artifact are available yet. (eafcae962fde)
  • Sibling implementation remains open: The provided related-item context shows fix(voice-call): avoid OpenAI realtime double greeting #86285 is another open implementation for the same duplicate OpenAI realtime greeting bug, with a different first-greeting strategy and no live call proof.

Likely related people:

  • steipete: Available history and blame show Peter Steinberger moved the voice-call realtime/OpenAI provider path into bundled extensions and has the heaviest recent ownership signal on the touched runtime files. (role: feature history owner; confidence: high; commits: a23ab9b906dc, 782247b42388, a5b563280910; files: extensions/voice-call/src/webhook/realtime-handler.ts, extensions/openai/realtime-voice-provider.ts, src/talk/session-runtime.ts)
  • vincentkoc: Available history shows Vincent Koc recently touched shared extension/runtime type seams adjacent to the realtime voice provider boundary, so he is a secondary routing candidate if the plugin SDK surface is questioned. (role: adjacent runtime seam contributor; confidence: medium; commits: 4a275cf6b1c6, 0f8ea1d3d905; files: src/talk/provider-types.ts, src/talk/session-runtime.ts)
  • Michael Appel: Available history includes a recent voice-call realtime WebSocket hardening commit touching the same handler area, making him a lower-confidence adjacent reviewer candidate. (role: recent area contributor; confidence: low; commits: afadb7dae673; files: extensions/voice-call/src/webhook/realtime-handler.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels May 25, 2026
@clawsweeper

clawsweeper Bot commented May 25, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@clawsweeper clawsweeper Bot mentioned this pull request May 25, 2026
Merged
@wyf027
wyf027 marked this pull request as draft May 26, 2026 15:48
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels May 26, 2026
@wyf027
wyf027 force-pushed the leno23/fix-openai-voice-initial-greeting-85846 branch from 52b03a3 to 202fe8e Compare May 30, 2026 03:47
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels May 30, 2026
@wyf027

wyf027 commented May 30, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the OpenAI realtime opt-out review finding in dd606e65:

  • Preserved autoRespondToAudio: false through buildOpenAIRealtimeVoiceProvider().resolveConfig(...) so the real provider-resolution path now keeps the voice-call opt-out.
  • Added a resolver-level regression test that failed before the fix with expected undefined to be false and passes after the fix.
  • Re-ran node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts extensions/voice-call/src/webhook/realtime-handler.test.ts (70 tests passed) and pnpm check:changed --staged (passed).

PR body is updated with the new real behavior proof. Live Twilio/OpenAI call proof is still not available from this cron environment because it has no external telephony/provider credentials.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels May 30, 2026
@wyf027

wyf027 commented May 30, 2026

Copy link
Copy Markdown
Contributor Author

Applied the OpenAI bridge-creation follow-up in eafcae96: createBridge now passes req.autoRespondToAudio ?? config.autoRespondToAudio into the realtime bridge, and the new createBridge regression test failed RED with turn detection.create_response: expected true to deeply equal false before the fix and passes after it.

Verification after the fix:

  • node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts --testNamePattern "applies provider-configured realtime auto-response opt-out when creating bridges"
  • node scripts/run-vitest.mjs extensions/openai/realtime-voice-provider.test.ts extensions/voice-call/src/webhook/realtime-handler.test.ts
  • pnpm check:changed --staged

I updated the PR body with the new RED/GREEN and changed-gate proof. Live Twilio/OpenAI call proof still is not available from this cron environment because it has no external telephony/provider credentials.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added the merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. label May 30, 2026
@wyf027

wyf027 commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Closing stale contributor PR that has been open for over a week without merge progress. I am keeping the branch intact so it can be reopened or reused if maintainers want to continue it.

@wyf027 wyf027 closed this Jun 2, 2026
@clawsweeper clawsweeper Bot mentioned this pull request Jun 20, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

channel: voice-call Channel integration: voice-call extensions: openai merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: M status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[voice-call] OpenAI realtime: outbound calls greet caller twice — triggerGreeting + server_vad.create_response both fire response.create

2 participants

@wyf027 @RomneyDa