Skip to content

refactor(talk): add realtime voice session harness and adopt in google-meet and voice-call#109497

Merged
steipete merged 2 commits into
mainfrom
refactor/talk-realtime-harness
Jul 17, 2026
Merged

refactor(talk): add realtime voice session harness and adopt in google-meet and voice-call#109497
steipete merged 2 commits into
mainfrom
refactor/talk-realtime-harness

Conversation

@steipete

@steipete steipete commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Every realtime voice consumer re-plumbed the same session wiring around the shared bridge/talk primitives: talk-session controller creation and event ring capture, output-activity tracking with echo-suppression window bookkeeping, transcript recording with assistant-echo filtering, agent talkback queue construction, forced-consult coordination, and barge-in-to-output-flush coupling. Google Meet carried this twice (agent/STT and bidi engines), voice-call once, Discord once — four hand-maintained copies of the same invariants.

Why This Change Was Made

Phase 3 of the voice-stack consolidation (follows #109413, #109466). New src/talk/realtime-session-harness.ts (createRealtimeVoiceSessionHarness, exported via openclaw/plugin-sdk/realtime-voice) owns the transport-agnostic session shape: talk lifecycle helpers with injected payload builders, input/output audio recording with echo-suppression windows and suppressed-byte accounting, bounded transcript + bridge-event capture, talkback queue, forced-consult coordinator, health snapshot assembly, and handleBargeIn with flush-generation fallback (generalizing Meet's clear-count pattern).

Adopted by Google Meet (both engines) and voice-call's realtime handler; their local wiring is deleted. Consumers keep audio I/O, format conversion, provider resolution, prompts, and pacing.

Discord adoption is deliberately a follow-up: the mapping was verified against extensions/discord/src/voice/realtime.ts (bridge creation, talkback, forced-consult state, output tracker, barge-in paths all map onto the harness; speaker attribution, wake-name policy, Opus conversion, preroll, and player lifecycle stay Discord-side) and no harness API gap was found.

User Impact

None intended. Talk-event shapes and payloads, health fields, echo-suppression timings, log lines, and barge-in behavior are unchanged per consumer.

Evidence

  • node scripts/run-vitest.mjs src/talk extensions/google-meet extensions/voice-call: 1009+ tests pass (rerun after rebase onto current main).
  • New harness unit tests (talk-event ordering, echo-suppression window, talkback delegation, barge-in flush fallback); the voice-call forced-consult cancellation regression test was adapted to drive the harness-owned coordinator (review-found, restored — late native consult is terminally answered as cancelled without invoking the consult handler).
  • pnpm plugin-sdk:surface:check green (+2 exports / +1 callable, standard annotations); node scripts/check-deadcode-exports.mjs all scans 0; API baseline regenerated; SDK docs updated.
  • Autoreview (Codex gpt-5.6-sol, xhigh): one finding (the deleted regression test) fixed; final round clean.
  • Non-test LOC net −3; the four wiring copies collapse to one owner.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation channel: voice-call Channel integration: voice-call scripts Repository scripts plugin: google-meet size: XL maintainer Maintainer-authored PR labels Jul 17, 2026
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 17, 2026
@steipete
steipete force-pushed the refactor/talk-realtime-harness branch 2 times, most recently from 825005d to e539c21 Compare July 17, 2026 03:22
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jul 17, 2026
@clawsweeper

clawsweeper Bot commented Jul 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 17, 2026, 12:10 AM ET / 04:10 UTC.

Summary
Adds a transport-independent realtime voice session harness to the public plugin SDK and migrates Google Meet and voice-call realtime session wiring to it.

PR surface: Source -1, Tests +168, Docs 0, Generated 0, Other +2. Total +169 across 10 files.

Reproducibility: not applicable. This PR is presented as a behavior-neutral refactor, so review requires equivalence proof across real Google Meet and voice-call sessions rather than reproduction of a reported bug.

Review metrics: 1 noteworthy metric.

  • Public SDK surface: 2 exports added, 1 callable added. The aggregate harness creates a compatibility-sensitive plugin contract rather than only refactoring bundled implementation.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add redacted live Google Meet and voice-call evidence covering talkback, echo suppression, forced consult, and barge-in behavior.
  • Resolve whether the aggregate session harness is a public plugin SDK contract or an internal bundled helper.
  • Refresh the dirty head against current main and re-review the resolved result.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The evidence is limited to tests and static gates; add redacted after-fix live Google Meet and voice-call recordings, logs, terminal output, or linked artifacts, update the PR body, and ask a maintainer for @clawsweeper re-review if review does not refresh automatically.

Risk before merge

  • [P1] The exported aggregate harness becomes a shipped plugin API contract even though its permanent abstraction boundary has not received explicit maintainer-visible approval.
  • [P1] Tests do not prove that live audio, echo suppression, forced-consult cancellation, talkback pacing, and barge-in behavior remain equivalent in actual Google Meet and telephony sessions.
  • [P1] The head is not cleanly mergeable with current main, so the reviewed behavior may change during conflict resolution or rebase.

Maintainer options:

  1. Internalize Before Merge (recommended)
    Remove the public SDK exports and surface-budget additions while retaining the shared helper for bundled Google Meet and voice-call consumers.
  2. Accept the SDK Contract
    Merge the public harness only after explicit owner approval and focused compatibility documentation and tests define its supported contract.
  3. Pause the Consolidation
    Hold or close the PR if the aggregate harness is too broad to stabilize while the voice architecture is still evolving.

Next step before merge

  • [P1] A maintainer must decide the permanent public SDK boundary, the contributor must supply real-session proof, and the dirty branch must be refreshed; these are not safe automated repairs.

Maintainer decision needed

  • Question: Should createRealtimeVoiceSessionHarness become a permanent public plugin-sdk/realtime-voice contract, or remain internal while the voice consolidation settles?
  • Rationale: The factory exposes a broad orchestration abstraction whose lifecycle and type shape become compatibility obligations; tests cannot determine the intended long-term external plugin contract.
  • Likely owner: steipete — The strongest available ownership signal is the same contributor's merged work across the preceding voice-stack consolidation phases.
  • Options:
    • Keep It Internal First (recommended): Adopt the harness in bundled consumers without exporting it publicly until an external plugin use case proves the API shape.
    • Approve Public SDK: Accept the exported factory and type now with explicit ownership, documentation, and compatibility expectations.
    • Narrow the Export: Expose smaller stable primitives and keep aggregate session orchestration owner-private.

Security
Cleared: The diff changes voice orchestration, SDK exports, tests, docs, and surface baselines without adding dependencies, permissions, secret access, downloaded code, or new execution sources.

Review details

Best possible solution:

Keep the harness internal while the consolidation settles, or explicitly approve and document its stable SDK contract; then land a refreshed branch only after redacted live Google Meet and voice-call evidence confirms behavioral equivalence.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR is presented as a behavior-neutral refactor, so review requires equivalence proof across real Google Meet and voice-call sessions rather than reproduction of a reported bug.

Is this the best way to solve the issue?

Unclear: centralizing duplicated invariants is directionally sound, but publishing the aggregate harness before its external API boundary is approved is not yet proven to be the narrowest maintainable solution.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 17c2ce05d80a.

Label changes

Label changes:

  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • remove rating: 🦪 silver shellfish: Current PR rating is rating: 🧂 unranked krab, so this older rating label is no longer current.

Label justifications:

  • P3: This is a behavior-neutral consolidation with no claimed user-facing regression or urgent release impact.
  • merge-risk: 🚨 compatibility: The PR exports an aggregate realtime session harness through the public plugin SDK, making its lifecycle and type shape an upgrade contract.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The evidence is limited to tests and static gates; add redacted after-fix live Google Meet and voice-call recordings, logs, terminal output, or linked artifacts, update the PR body, and ask a maintainer for @clawsweeper re-review if review does not refresh automatically.
Evidence reviewed

PR surface:

Source -1, Tests +168, Docs 0, Generated 0, Other +2. Total +169 across 10 files.

View PR surface stats
Area Files Added Removed Net
Source 4 486 487 -1
Tests 2 180 12 +168
Docs 2 2 2 0
Config 0 0 0 0
Generated 1 2 2 0
Other 1 4 2 +2
Total 10 674 505 +169

What I checked:

  • Aggregate harness introduced: The branch adds a shared owner for talk lifecycle events, transcript and bridge-event capture, echo suppression, talkback, forced consults, health snapshots, and barge-in flushing. (src/talk/realtime-session-harness.ts:1, e539c217d3ac)
  • Public plugin API expanded: The harness factory and type are exported through the realtime-voice SDK, creating a new compatibility-sensitive contract for external plugins. (src/plugin-sdk/realtime-voice.ts:174, e539c217d3ac)
  • Bundled consumers migrated: Google Meet and voice-call replace their local session orchestration with the shared harness while retaining transport and provider-specific behavior. (extensions/google-meet/src/realtime.ts:13, e539c217d3ac)
  • Regression coverage added: New harness tests cover event ordering, echo suppression, talkback delegation, and barge-in fallback; voice-call coverage retains late forced-consult cancellation behavior. (src/talk/realtime-session-harness.test.ts:1, e539c217d3ac)
  • Proof remains synthetic: The PR body reports focused tests, API-surface validation, dead-code checks, and autoreview, but provides no live after-fix Google Meet or voice-call recording, logs, terminal transcript, or linked artifact. (e539c217d3ac)
  • Previous review continuity: The previous completed ClawSweeper cycle found no discrete code defect but required real behavior proof and maintainer confirmation of the new public plugin API boundary; the reviewed head is unchanged. (e539c217d3ac)

Likely related people:

  • steipete: Authored the two immediately preceding merged voice-stack consolidation phases and their central Google Meet, voice-call, and realtime-voice SDK work, establishing relevant merged history beyond this proposal. (role: feature owner; confidence: high; commits: edd052989b5d, 92146f9f803b; files: extensions/google-meet/src/realtime.ts, extensions/voice-call/src/webhook/realtime-handler.ts, src/plugin-sdk/realtime-voice.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (1 earlier review cycle)
  • reviewed 2026-07-17T03:28:13.881Z sha e539c21 :: needs real behavior proof before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jul 17, 2026
@steipete
steipete force-pushed the refactor/talk-realtime-harness branch from e539c21 to 8bb26bf Compare July 17, 2026 04:22
@steipete
steipete merged commit e361845 into main Jul 17, 2026
117 checks passed
@steipete
steipete deleted the refactor/talk-realtime-harness branch July 17, 2026 04:28
@steipete

steipete commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Merged via squash.

@steipete steipete mentioned this pull request Jul 17, 2026
Merged
@github-actions github-actions Bot mentioned this pull request Jul 17, 2026
Open
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 17, 2026
@steipete
refactor(talk): add realtime voice session harness and adopt in googl…
1a00d9b 
…e-meet and voice-call (openclaw#109497)

* refactor(talk): add realtime voice session harness and adopt in google-meet and voice-call

* fix(google-meet): satisfy prefer-const in realtime engine wiring
This was referenced Jul 17, 2026
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

channel: voice-call Channel integration: voice-call docs Improvements or additions to documentation maintainer Maintainer-authored PR merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. plugin: google-meet rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. scripts Repository scripts size: XL status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@steipete