Skip to content

feat(mac): unified Liquid Glass pairing approval panel#102601

Merged
steipete merged 2 commits into
mainfrom
claude/mac-dialog-redesign-4f4fea
Jul 9, 2026
Merged

feat(mac): unified Liquid Glass pairing approval panel#102601
steipete merged 2 commits into
mainfrom
claude/mac-dialog-redesign-4f4fea

Conversation

@steipete

@steipete steipete commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes #102556.

During a realtime Talk session in the macOS app (Control UI inside WKWebView), the streaming assistant voice turn rendered one character per line: the turn box froze at the width of the first transcript delta (~114px) and grew thousands of pixels tall. Independently, the streamed transcript text itself was corrupted with injected spaces ("Chat G PT", "con vers ational").

Why This Change Was Made

Two independent bugs stacked on the same surface:

  1. WebKit-only layout collapse. .agent-chat__voice-turn was a CSS grid sized shrink-to-fit as a flex item of the voice-turns column. WebKit (Mac/iOS WKWebView) never re-runs the grid flex item's intrinsic sizing while streamed text grows, so the turn kept the width computed for the first one-character delta and every subsequent character wrapped to its own line. Chromium recomputes correctly, which is why existing browser tests never caught it. Converting the turn to a flex row (same visual: speaker | text | streaming dot, baseline-aligned, hugging content up to 780px) sidesteps the WebKit invalidation bug in both engines; probed grid variants (auto track, plain 1fr) stayed collapsed. Side effect: the fixed 12px third grid column is no longer reserved on finished turns that have no streaming dot.

  2. Assistant transcript merge mangling (all engines). mergeRealtimeTranscriptText applied a word-boundary spacing heuristic designed for user ASR re-statements to assistant audio-transcript deltas. Assistant deltas are verbatim fragments that split inside words (OpenAI response.output_audio_transcript.delta, Google Live outputTranscription; the server-side Google provider concatenates them verbatim in extensions/google/realtime-voice-provider.ts), so the heuristic injected a space between letter-adjacent fragments. Assistant entries now merge as strictly verbatim fragment streams (no synthesized characters, which would corrupt content like "Version 1.2" or "docs.openclaw.ai" at punctuation boundaries), and finals handle both provider shapes: OpenAI-style full-transcript finals replace, Google-Live-style tail-fragment finals append. User ASR merge heuristics are unchanged; user transcripts from shipped providers arrive with leading whitespace or as re-statements, which the existing path handles.

User Impact

Talk mode transcripts in the macOS/iOS app and all browsers render as normal wrapped text while streaming, and words are no longer broken apart ("I'm ChatGPT, a conversational AI model ..." instead of a one-character-per-line column of "I ' m C h a t G P T ...").

Before (WebKit): https://raw.githubusercontent.com/steipete/openclaw-pr-assets/main/talk-voice-turn-wrapping/before-webkit-collapsed.png
After (WebKit): https://raw.githubusercontent.com/steipete/openclaw-pr-assets/main/talk-voice-turn-wrapping/after-webkit-fixed.png

Evidence

  • Reproduced against the real Control UI (vite dev server + mock gateway, gateway-relay Talk session, per-character assistant transcript deltas) in Playwright WebKit 26.5: assistant turn measured 114x5238px with a 3px text column before the fix; 780x56px after. Chromium measures identically after the fix (780x56) and the user turn keeps hugging its content (273px).
  • Transcript merge unit-proofed: per-token deltas ["I","'m"," Chat","G","PT",...] previously merged to "I'm Chat G PT, a con vers ational ...", now "I'm ChatGPT, a conversational AI".
  • New regression tests:
    • ui/src/pages/chat/realtime-talk-conversation.test.ts: verbatim per-token and per-character assistant delta merging (including punctuation boundaries like "Version 1.2 is on docs.openclaw.ai"), OpenAI-style authoritative final replacement, Google-Live-style tail-fragment final append (9/9 pass).
    • ui/src/e2e/browser-talk-start-stop.e2e.test.ts: drives a full gateway-relay Talk session through the mock gateway with per-character assistant deltas and asserts the rendered voice-turn text is unmangled plus sane turn geometry (3/3 pass).
  • Existing talk conversation + talk e2e suites pass; the punctuation-spacing heuristic test now exercises the user ASR path, which keeps that behavior.

@openclaw-barnacle openclaw-barnacle Bot added app: macos App: macos size: XL maintainer Maintainer-authored PR labels Jul 9, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 650c9c674e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

return
}

self.queue.removeAll { $0.requestId == request.requestId }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard node decisions against stale list snapshots

When a node.pair.list reconcile request is already in flight while the user approves or rejects a card, this path removes the card but does not invalidate that older list response. reconcileOnce still applies whatever it fetched (apps/macos/Sources/OpenClaw/NodePairingApprovalPrompter.swift:612-613), and apply enqueues every pending row (apps/macos/Sources/OpenClaw/NodePairingApprovalPrompter.swift:209-212), so a pre-decision snapshot can re-add the just-approved/rejected request and leave a stale approval card until the next poll or a failed click. The device prompter bumps listFetchGeneration before removing for this exact race; the node path needs the same guard.

Useful? React with 👍 / 👎.

steipete added 2 commits July 9, 2026 02:21
Replace the serial node/device pairing NSAlerts with one floating
approval panel. Both prompters feed request cards into a shared
PairingApprovalCenter; each card shows a hardware icon, platform and
model, app/core version, source IP, a copyable short id, request age,
a trust line (first connection / already-paired-id caution / repair
token rotation), and a friendly access summary with an elevated
warning for system.run-class commands. Requests resolved elsewhere
disappear live; Not Now snoozes without resolving (gateway TTL
applies) and the menu-bar pending line reopens the panel. Liquid
Glass surface on macOS 26+, material fallback on macOS 15. Deletes
the NSAlert + invisible-host-window machinery, dead node isRepair
handling, and text-only describe() body.

Closes #102535
@steipete
steipete force-pushed the claude/mac-dialog-redesign-4f4fea branch from 650c9c6 to 9dd1afd Compare July 9, 2026 09:23
@steipete
steipete merged commit 4cc9200 into main Jul 9, 2026
74 checks passed
@steipete
steipete deleted the claude/mac-dialog-redesign-4f4fea branch July 9, 2026 09:31
@steipete

steipete commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 10, 2026
* feat(mac): unified Liquid Glass pairing approval panel

Replace the serial node/device pairing NSAlerts with one floating
approval panel. Both prompters feed request cards into a shared
PairingApprovalCenter; each card shows a hardware icon, platform and
model, app/core version, source IP, a copyable short id, request age,
a trust line (first connection / already-paired-id caution / repair
token rotation), and a friendly access summary with an elevated
warning for system.run-class commands. Requests resolved elsewhere
disappear live; Not Now snoozes without resolving (gateway TTL
applies) and the menu-bar pending line reopens the panel. Liquid
Glass surface on macOS 26+, material fallback on macOS 15. Deletes
the NSAlert + invisible-host-window machinery, dead node isRepair
handling, and text-only describe() body.

Closes openclaw#102535

* fix(mac): satisfy swiftformat conditionalAssignment and resync native i18n inventory
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

app: macos App: macos maintainer Maintainer-authored PR size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Talk voice-turn transcript collapses to one character per line in the macOS app while streaming

1 participant