fix(ui): keep streamed talk voice-turn transcripts readable#102573
Conversation
Talk voice turns collapsed to one character per line in the macOS app
(WKWebView) while streaming: WebKit never re-runs shrink-to-fit intrinsic
sizing for a grid flex-item as its content grows, so the turn froze at
first-delta width. Convert the turn to a flex row (same visual layout).
Assistant transcript deltas also merged through the user-ASR word-boundary
spacing heuristic, mangling verbatim fragments ("Chat G PT"). Assistant
entries now concatenate fragments verbatim, space only sentence-punctuation
joins, and accept both full-transcript and tail-fragment finals.
Fixes #102556
openclaw-barnacle
Bot
added
app: web-ui
Drop the sentence-punctuation separator: it invented characters in real content (Version 1. 2, docs.openclaw. ai) and Google Live tail-finals could never heal it. The punctuation-spacing heuristic remains user-only.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aab4ab7f9c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (looksLikeTranscriptReplacement(existing, incoming)) { | ||
| return incoming; | ||
| } | ||
| return `${existing}${assistantFragmentSeparator(existing, incoming)}${incoming}`; |
There was a problem hiding this comment.
When the provider-websocket Google Live path emits an assistant outputTranscription and then a final modelTurn.parts[].text on the same turn, as implemented in ui/src/pages/chat/realtime-talk-google-live.ts:242-269 and exercised by the fixture at ui/src/pages/chat/realtime-talk-google-live.test.ts:284-293, the final text part reaches this append branch. Because assistantFragmentSeparator only inserts spaces after sentence punctuation, an existing partial like hi followed by final text there now renders as hithere; before this change the shared merge path inserted the word-boundary space. Keep the verbatim merge limited to actual transcript deltas, or preserve spacing for these final text-part joins.
Useful? React with 👍 / 👎.
|
ClawSweeper status: review started. I am starting a fresh review of this pull request: fix(ui): keep streamed talk voice-turn transcripts readable This is item 1/1 in the current shard. Shard 0/1. This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking. Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted. |
|
Merged via squash.
|
…#102573) * fix(ui): keep streamed talk voice-turn transcripts readable Talk voice turns collapsed to one character per line in the macOS app (WKWebView) while streaming: WebKit never re-runs shrink-to-fit intrinsic sizing for a grid flex-item as its content grows, so the turn froze at first-delta width. Convert the turn to a flex row (same visual layout). Assistant transcript deltas also merged through the user-ASR word-boundary spacing heuristic, mangling verbatim fragments ("Chat G PT"). Assistant entries now concatenate fragments verbatim, space only sentence-punctuation joins, and accept both full-transcript and tail-fragment finals. Fixes openclaw#102556 * fix(ui): keep assistant talk fragments strictly verbatim Drop the sentence-punctuation separator: it invented characters in real content (Version 1. 2, docs.openclaw. ai) and Google Live tail-finals could never heal it. The punctuation-spacing heuristic remains user-only.
1 participant
What Problem This Solves
Fixes #102556.
During a realtime Talk session in the macOS app (Control UI inside WKWebView), the streaming assistant voice turn rendered one character per line: the turn box froze at the width of the first transcript delta (~114px) and grew thousands of pixels tall. Independently, the streamed transcript text itself was corrupted with injected spaces ("Chat G PT", "con vers ational").
Why This Change Was Made
Two independent bugs stacked on the same surface:
WebKit-only layout collapse.
.agent-chat__voice-turnwas a CSS grid sized shrink-to-fit as a flex item of the voice-turns column. WebKit (Mac/iOS WKWebView) never re-runs the grid flex item's intrinsic sizing while streamed text grows, so the turn kept the width computed for the first one-character delta and every subsequent character wrapped to its own line. Chromium recomputes correctly, which is why existing browser tests never caught it. Converting the turn to a flex row (same visual: speaker | text | streaming dot, baseline-aligned, hugging content up to 780px) sidesteps the WebKit invalidation bug in both engines; probed grid variants (autotrack, plain1fr) stayed collapsed. Side effect: the fixed 12px third grid column is no longer reserved on finished turns that have no streaming dot.Assistant transcript merge mangling (all engines).
mergeRealtimeTranscriptTextapplied a word-boundary spacing heuristic designed for user ASR re-statements to assistant audio-transcript deltas. Assistant deltas are verbatim fragments that split inside words (OpenAIresponse.output_audio_transcript.delta, Google LiveoutputTranscription; the server-side Google provider concatenates them verbatim inextensions/google/realtime-voice-provider.ts), so the heuristic injected a space between letter-adjacent fragments. Assistant entries now merge as strictly verbatim fragment streams (no synthesized characters, which would corrupt content like "Version 1.2" or "docs.openclaw.ai" at punctuation boundaries), and finals handle both provider shapes: OpenAI-style full-transcript finals replace, Google-Live-style tail-fragment finals append. User ASR merge heuristics are unchanged; user transcripts from shipped providers arrive with leading whitespace or as re-statements, which the existing path handles.User Impact
Talk mode transcripts in the macOS/iOS app and all browsers render as normal wrapped text while streaming, and words are no longer broken apart ("I'm ChatGPT, a conversational AI model ..." instead of a one-character-per-line column of "I ' m C h a t G P T ...").
Before (WebKit): https://raw.githubusercontent.com/steipete/openclaw-pr-assets/main/talk-voice-turn-wrapping/before-webkit-collapsed.png
After (WebKit): https://raw.githubusercontent.com/steipete/openclaw-pr-assets/main/talk-voice-turn-wrapping/after-webkit-fixed.png
Evidence
gateway-relayTalk session, per-character assistant transcript deltas) in Playwright WebKit 26.5: assistant turn measured 114x5238px with a 3px text column before the fix; 780x56px after. Chromium measures identically after the fix (780x56) and the user turn keeps hugging its content (273px).["I","'m"," Chat","G","PT",...]previously merged to"I'm Chat G PT, a con vers ational ...", now"I'm ChatGPT, a conversational AI".ui/src/pages/chat/realtime-talk-conversation.test.ts: verbatim per-token and per-character assistant delta merging (including punctuation boundaries like "Version 1.2 is on docs.openclaw.ai"), OpenAI-style authoritative final replacement, Google-Live-style tail-fragment final append (9/9 pass).ui/src/e2e/browser-talk-start-stop.e2e.test.ts: drives a fullgateway-relayTalk session through the mock gateway with per-character assistant deltas and asserts the rendered voice-turn text is unmangled plus sane turn geometry (3/3 pass).