Skip to content

fix(voice-call): persist complete Google Live transcripts#84161

Merged
steipete merged 1 commit into
openclaw:mainfrom
happydog-bot:fix/persist-realtime-assistant-transcript
Jul 7, 2026
Merged

fix(voice-call): persist complete Google Live transcripts#84161
steipete merged 1 commit into
openclaw:mainfrom
happydog-bot:fix/persist-realtime-assistant-transcript

Conversation

@happydog-bot

@happydog-bot happydog-bot commented May 19, 2026

Copy link
Copy Markdown

What Problem This Solves

Completed Google Live voice calls could persist user speech while losing the assistant side of the conversation. The original patch routed assistant text through call.speaking, but that event is also used for classic telephony state transitions and cannot safely double as a transcript payload.

Why This Change Was Made

This rebuild keeps the fix at the owning boundaries:

  • Google Live accumulates the official input/output transcription streams and finalizes them only on the protocol finished marker or permanent bridge close.
  • Resumable reconnects preserve pending hypotheses; fresh sessions discard unfinished hypotheses instead of merging unrelated utterances or triggering a premature consult.
  • Voice Call persists finalized assistant text through a distinct call.assistant-speech event while leaving classic call.speaking behavior unchanged.
  • User final reconciliation supports complete finals, terminal deltas, cumulative hypotheses, and bounded long partials without duplication.
  • Provider close callbacks run before call teardown, with exception-safe cleanup.

The rebuilt commit preserves the original contributor as co-author.

User Impact

Google Live voice-call histories now retain complete user and assistant transcript entries. Transient reconnects no longer turn incomplete speech into an agent request, and the change does not alter classic telephony speaking semantics or add configuration.

Evidence

  • Blacksmith Testbox: 82 focused tests passed across Google Live, realtime voice-call handling, manager persistence, and the mock provider.
  • Blacksmith Testbox: pnpm check:changed passed after rebasing onto current main, including extension typechecks, lint, storage guards, and import-cycle checks.
  • Live Google Developer API trace confirmed output-transcription chunks can be followed by generationComplete and turnComplete without a transcription finished marker; permanent-close flushing covers that observed API behavior.
  • Fresh autoreview: clean, with no accepted or actionable findings.
  • Original contributor live proof: an inbound Telnyx call using Google Gemini Live produced both user and bot transcript entries after the persistence fix.

Known gap: no fresh carrier call was placed for this rebuild; the live Google API trace plus close-order integration coverage exercise the changed contract.

@openclaw-barnacle openclaw-barnacle Bot added channel: voice-call Channel integration: voice-call gateway Gateway runtime extensions: memory-core Extension: memory-core size: L triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 19, 2026
@clawsweeper

clawsweeper Bot commented May 19, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 1, 2026, 6:15 AM ET / 10:15 UTC.

Summary
The PR persists text-bearing voice-call speaking events as bot transcript entries, adds Google realtime transcript aggregation, changes final user transcript buffering, and adds boot-run session classification/reaper behavior.

PR surface: Source +67, Tests +331, Docs +1. Total +399 across 17 files.

Reproducibility: yes. Source inspection on current main and v2026.6.11 shows final assistant realtime text is emitted as call.speaking.text while processEvent does not append that text to the call transcript; I did not run a live carrier call in this read-only review.

Review metrics: 1 noteworthy metric.

  • Session-State Surface: 1 boot-run key shape added, 1 host SDK export added, 1 reaper classifier expanded. These changes affect persisted session classification and cleanup outside the voice-call transcript bug, so maintainers should review upgrade and ownership impact before merge.

Stored data model
Persistent data-model change detected: serialized state: extensions/voice-call/src/webhook/realtime-handler.test.ts, serialized state: packages/memory-host-sdk/src/host/session-files.test.ts, serialized state: packages/memory-host-sdk/src/host/session-files.ts, serialized state: src/cron/session-reaper.test.ts, serialized state: src/cron/session-reaper.ts, serialized state: src/sessions/session-key-utils.ts, and 5 more. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add inspectable redacted live proof showing completed calls with both user and bot transcript entries, then update the PR body to trigger a fresh review or ask a maintainer to comment @clawsweeper re-review.
  • Rebase against current main while preserving BOOT.md echo protections and call.speaking timeout refresh behavior.
  • Remove duplicate transcript paths and split or redesign the boot-session memory work.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body describes live testing but provides no inspectable redacted rows, logs, terminal output, screenshots, recordings, or linked artifacts; contributor action is needed, with private details redacted, before ClawSweeper can clear proof. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The PR body claims live Telnyx/Gemini testing, but it provides no inspectable redacted transcript rows, logs, terminal output, screenshots, recordings, or linked artifacts showing completed calls with both user and bot entries.
  • [P1] Classic Telnyx TTS can store duplicate bot transcript entries because the branch records every text-bearing call.speaking while speak() already appends the bot utterance after provider playback.
  • [P1] Google finished:true output transcription can emit both the final chunk and the aggregate as final assistant callbacks, which the voice-call handler would persist as separate bot turns.
  • [P1] The boot-session changes overlap current BOOT.md prompt-disclosure protections and can leave boot transcripts unclassified after the new boot-run mapping is deleted.
  • [P2] The PR is currently conflicting, so conflict resolution must preserve current call.speaking live-call timeout refresh behavior and current boot prompt safeguards.

Maintainer options:

  1. Rebase And Narrow The Voice Fix (recommended)
    Resolve conflicts against current main, keep the voice transcript persistence focused, preserve live-call timeout and BOOT.md safeguards, prevent duplicate bot turns, and require redacted live proof before merge.
  2. Split Boot-Session Memory Work
    Move the boot-run key, host SDK export, memory ingestion, and reaper changes to a separate maintainer-reviewed PR with targeted security and upgrade proof.
  3. Pause Until Real Proof Exists
    Keep this PR blocked if the contributor cannot provide inspectable redacted live transcript evidence for the changed call path.

Next step before merge

  • [P1] Human review is needed because the branch requires contributor-supplied proof, conflict resolution, and maintainer scope/security decisions rather than a safe automated repair on the current head.

Security
Needs attention: The diff has no dependency or workflow supply-chain change, but the boot rewrite can weaken BOOT.md prompt-disclosure protections and leave boot prompt transcripts available to memory ingestion.

Review findings

  • [P1] Preserve BOOT.md echo protections when rebasing — src/gateway/boot.ts:38-49
  • [P1] Keep boot-run classification evidence after the run — src/gateway/boot.ts:139-142
  • [P2] Avoid recording Telnyx TTS callbacks twice — extensions/voice-call/src/manager/events.ts:280-281
Review details

Best possible solution:

Land a rebased, focused voice-call transcript fix that persists exactly one assistant turn, preserves current timer and BOOT.md safeguards, and splits or redesigns the boot-session memory work with separate security and upgrade proof.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection on current main and v2026.6.11 shows final assistant realtime text is emitted as call.speaking.text while processEvent does not append that text to the call transcript; I did not run a live carrier call in this read-only review.

Is this the best way to solve the issue?

No. Persisting assistant text is the right direction, but this branch records too broad an event source, creates duplicate final paths, and mixes in boot/session-state changes that need a separate or corrected design.

Full review comments:

  • [P1] Preserve BOOT.md echo protections when rebasing — src/gateway/boot.ts:38-49
    The PR head builds the boot prompt with raw BOOT.md content and omits current main's internal-runtime-context wrapper plus boot echo guard. Resolving conflicts by keeping this version can re-open prompt disclosure through final replies or message-tool sends.
    Confidence: 0.9
  • [P1] Keep boot-run classification evidence after the run — src/gateway/boot.ts:139-142
    The new memory classifier only recognizes boot transcripts by finding a boot-run key in sessions.json, but runBootOnce deletes that same key immediately after agentCommand. The transcript file can remain while the classifier loses the only evidence that it was boot-generated, so BOOT.md content can still enter memory ingestion.
    Confidence: 0.9
  • [P2] Avoid recording Telnyx TTS callbacks twice — extensions/voice-call/src/manager/events.ts:280-281
    This records bot text for every text-bearing call.speaking, but speak() already appends the bot utterance after provider.playTts, and Telnyx maps call.speak.started to call.speaking with the same text. Classic Telnyx TTS calls can therefore store duplicate bot transcript entries.
    Confidence: 0.9
  • [P2] Emit one final Google transcript per utterance — extensions/google/realtime-voice-provider.ts:770-771
    When outputTranscription.finished is true, the bridge emits that chunk as a final callback and then flushes the accumulated aggregate as another final callback. The voice-call realtime handler persists every final assistant callback as call.speaking, so this can record both the last chunk and the aggregate as separate bot turns.
    Confidence: 0.88

Overall correctness: patch is incorrect
Overall confidence: 0.91

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 129f1bcf28e8.

Label changes

Label justifications:

  • P2: The PR targets a real bounded voice-call transcript persistence bug, but the remaining blockers are proof, correctness, conflicts, and session/security review rather than an emergency outage.
  • merge-risk: 🚨 session-state: The diff changes persisted call transcripts and also adds boot-run session classification, memory ingestion, host SDK export, and reaper behavior.
  • merge-risk: 🚨 security-boundary: The boot rewrite overlaps current BOOT.md prompt-disclosure protections and may let boot prompt content reach user-visible or memory paths if resolved incorrectly.
  • merge-risk: 🚨 availability: The conflicting PR head lacks current call.speaking live-call timer refresh behavior, so conflict resolution must preserve the current availability guard.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body describes live testing but provides no inspectable redacted rows, logs, terminal output, screenshots, recordings, or linked artifacts; contributor action is needed, with private details redacted, before ClawSweeper can clear proof. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +67, Tests +331, Docs +1. Total +399 across 17 files.

View PR surface stats
Area Files Added Removed Net
Source 10 147 80 +67
Tests 6 430 99 +331
Docs 1 1 0 +1
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 17 578 179 +399

Security concerns:

  • [medium] BOOT.md echo guard can be lost on rebase — src/gateway/boot.ts:38
    Current main wraps BOOT.md content in internal-runtime-context markers and registers a boot echo guard for message-tool sends; PR head lacks both protections on its boot prompt path.
    Confidence: 0.9
  • [medium] Boot prompt transcript can remain unclassified — src/gateway/boot.ts:139
    The PR's memory classifier depends on a boot-run sessions.json key, but the boot runner deletes that key after the run, so BOOT.md prompt content can remain in a transcript without being marked as boot-generated for memory ingestion.
    Confidence: 0.9

What I checked:

Likely related people:

  • steipete: Path history shows repeated recent work on voice-call manager/realtime handling and Google realtime behavior near the central transcript surfaces. (role: voice-call and Google realtime area contributor; confidence: high; commits: 2a39c217c87f, 5dccba7405c4, da7fb64aa485; files: extensions/voice-call/src/manager/events.ts, extensions/voice-call/src/webhook/realtime-handler.ts, extensions/google/realtime-voice-provider.ts)
  • jalehman: Path history shows recent work on boot session mapping, memory transcript corpus metadata, and memory ingestion paths that overlap the PR's boot-session side changes. (role: recent session and memory area contributor; confidence: high; commits: acc2a0ee7297, c24d266b2d09, da50a450d271; files: src/gateway/boot.ts, packages/memory-host-sdk/src/host/session-files.ts, extensions/memory-core/src/dreaming-phases.ts)
  • stainlu: Commit 95b2f9c added the current BOOT.md internal-context wrapping and echo suppression that this PR must preserve. (role: security-boundary contributor; confidence: high; commits: 95b2f9c6f9d4; files: src/gateway/boot.ts, src/gateway/boot-echo-guard.ts)
  • vincentkoc: Path history shows recent voice-call path normalization, voice model catalog, replay-key, and Windows transcript-path work adjacent to the PR's voice and memory surfaces. (role: recent adjacent contributor; confidence: medium; commits: 905c9759a716, c8cee2dce4e3, 27b15a19e84c; files: extensions/voice-call/src/manager/events.ts, extensions/voice-call/src/webhook/realtime-handler.ts, extensions/google/realtime-voice-provider.ts)
  • dynamite-bud: Path history shows Telnyx Media Streaming work adjacent to the PR's Telnyx/realtime transcript behavior. (role: Telnyx realtime feature contributor; confidence: medium; commits: bb1b075118e5; files: extensions/voice-call/src/webhook/realtime-handler.ts, extensions/voice-call/src/providers/telnyx.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels May 19, 2026
@clawsweeper clawsweeper Bot added P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels May 19, 2026
@clawsweeper

clawsweeper Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@clawsweeper clawsweeper Bot added the merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. label Jun 15, 2026
@clawsweeper clawsweeper Bot added the merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. label Jun 22, 2026
@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 22, 2026
@steipete steipete self-assigned this Jul 7, 2026
@steipete
steipete force-pushed the fix/persist-realtime-assistant-transcript branch from 578b502 to e3acf5f Compare July 7, 2026 13:01
@openclaw-barnacle openclaw-barnacle Bot removed gateway Gateway runtime extensions: memory-core Extension: memory-core labels Jul 7, 2026
@steipete steipete changed the title fix(voice-call): persist assistant transcript on call.speaking events fix(voice-call): persist complete Google Live transcripts Jul 7, 2026
@steipete

steipete commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Maintainer rebuild complete at e3acf5f4492ca0a7a08b94d2ea776a35a136d3e9.

What changed:

  • aggregate only Google Live's official input/output transcription streams;
  • finalize on transcription finished or permanent bridge close, not model-turn ordering;
  • preserve pending text only across resumable reconnects;
  • persist assistant finals through call.assistant-speech without changing classic call.speaking semantics;
  • reconcile full finals, terminal deltas, cumulative hypotheses, and bounded long partials;
  • keep close-time transcript callbacks alive through exception-safe teardown.

Proof on rebased head:

  • Blacksmith Testbox focused run: 82 tests passed (37 Google provider + 45 voice-call across realtime handler, manager, and mock provider).
  • Blacksmith Testbox pnpm check:changed: passed (extension typechecks/tests, lint, database/storage guards, runtime sidecar guard, import cycles).
  • Live Google Developer API trace: output-transcription chunks were observed without finished, followed by generationComplete and turnComplete, validating the close-time flush case.
  • Fresh autoreview: clean; no accepted/actionable findings.

Known gap: no fresh carrier call on the rebuilt head. Original contributor carrier proof is retained in the PR body; the rebuild adds live Google protocol proof plus focused close/reconnect integration coverage.

Thanks @happydog-bot for the report, live carrier proof, and original fix.

@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jul 7, 2026
@steipete
steipete merged commit e591dcf into openclaw:main Jul 7, 2026
110 of 115 checks passed
@steipete

steipete commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Merged via squash.

giodl73-repo pushed a commit to giodl73-repo/openclaw that referenced this pull request Jul 8, 2026
Co-authored-by: Peter Steinberger <[email protected]>
Co-authored-by: Happy Dog Support Bot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: voice-call Channel integration: voice-call extensions: google merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: L status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants