Skip to content

fix(voice-call): drop dangling surrogate in partial-transcript tail slice#98086

Closed
wangmiao0668000666 wants to merge 1 commit into
openclaw:mainfrom
wangmiao0668000666:fix/voice-call-realtime-handler-transcript-utf16
Closed

fix(voice-call): drop dangling surrogate in partial-transcript tail slice#98086
wangmiao0668000666 wants to merge 1 commit into
openclaw:mainfrom
wangmiao0668000666:fix/voice-call-realtime-handler-transcript-utf16

Conversation

@wangmiao0668000666

@wangmiao0668000666 wangmiao0668000666 commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

extensions/voice-call/src/webhook/realtime-handler.ts:157 uses raw text.slice(-MAX_PARTIAL_USER_TRANSCRIPT_CHARS) in limitPartialUserTranscript() to take the tail of a partial user transcript before sending it to the agent. When an emoji or other astral character straddles the tail-start boundary, the slice keeps only the LOW-surrogate half (e.g. 0xDF89 for 🎉), producing malformed UTF-16 (?/�) in the agent prompt.

This is the same pattern that Alix-007 fixed for Twitch (#98008), Signal (#97982), and that we just fixed for msteams (#98058) and imessage (#98065). Voice-call was the remaining channel surface using raw .slice() on a message body.

Why This Change Was Made

The plugin SDK provides sliceUtf16Safe in openclaw/plugin-sdk/text-utility-runtime for exactly this pattern. Its negative-start handling (start < 0 clamps to len + start, equivalent to a tail slice) AND its boundary surrogate-pair drop make it a drop-in replacement for text.slice(-N).

The shape mirrors our msteams PR (#98058) but is even smaller — 1 file, 1 import, 1 line replacement.

Evidence

Boundary probe — input is 🎉 (U+1F389, surrogate pair 0xD83C 0xDF89) placed so the LOW surrogate lands at the tail-start boundary:

[tail-200 surrogate straddles start]
  BEFORE .slice(-200):                 length=200 start=[df89 0061 0061 0061]   ← lone LOW surrogate, malformed UTF-16
  AFTER  sliceUtf16Safe(s, -200):      length=199 start=[0061 0061 0061 0061]   ← clean, surrogate pair dropped whole

[plain ASCII tail]
  BEFORE .slice(-200):                 length=200 start=[0061 0061 0061 0061]   ← pass-through, no change
  AFTER  sliceUtf16Safe(s, -200):      length=200 start=[0061 0061 0061 0061]

[emoji inside tail window]
  BEFORE .slice(-200):                 length=200 start=[0061 0061 0061 0061]   ← emoji fully inside window, preserved
  AFTER  sliceUtf16Safe(s, -200):      length=200 start=[0061 0061 0061 0061]

The first row is the bug — 🎉 straddles the boundary at position 100/101, and slice(-200) returns the LOW surrogate half (0xDF89) at the start of the tail. sliceUtf16Safe(s, -200) detects this (input.charCodeAt(100) is the high surrogate, input.charCodeAt(101) is the low surrogate) and bumps from to 102, dropping the surrogate pair whole and returning a 199-char clean ASCII tail.

After-fix Proof

  • node scripts/run-vitest.mjs extensions/voice-call/src/webhook/realtime-handler.transcript-utf16.test.ts --reporter=verbose
 ✓ |extension-voice-call| extensions/voice-call/src/webhook/realtime-handler.transcript-utf16.test.ts > voice-call partial-transcript tail UTF-16 > sliceUtf16Safe drops a surrogate pair straddling the tail-start boundary 82ms
 ✓ |extension-voice-call| extensions/voice-call/src/webhook/realtime-handler.transcript-utf16.test.ts > voice-call partial-transcript tail UTF-16 > sliceUtf16Safe is a pass-through for plain ASCII partial transcript 1ms
 ✓ |extension-voice-call| extensions/voice-call/src/webhook/realtime-handler.transcript-utf16.test.ts > voice-call partial-transcript tail UTF-16 > sliceUtf16Safe preserves an emoji that sits entirely inside the tail window 1ms
 ✓ |extension-voice-call| extensions/voice-call/src/webhook/realtime-handler.transcript-utf16.test.ts > voice-call partial-transcript tail UTF-16 > limitPartialUserTranscript-shaped behavior: short body passes through 0ms

 Test Files  1 passed (1)
      Tests  4 passed (4)

[test] passed 1 Vitest shard in 10.75s
  • node scripts/run-oxlint.mjs --tsconfig config/tsconfig/oxlint.scripts.json extensions/voice-call/src/webhook/realtime-handler.ts extensions/voice-call/src/webhook/realtime-handler.transcript-utf16.test.ts → EXIT=0
  • git diff --check openclaw/main...HEAD → clean

Diff scope

 extensions/voice-call/src/webhook/realtime-handler.ts                       |  3 ++-
 extensions/voice-call/src/webhook/realtime-handler.transcript-utf16.test.ts | 53 ++++++++++ (new)
 2 files changed, 55 insertions(+), 1 deletion(-)

Security & Privacy

No user-facing behavior change for ASCII partial transcripts — the tail passes through verbatim. For emoji-straddling transcripts, the tail is one code unit shorter because the surrogate pair is dropped whole instead of leaving a lone surrogate half — this is the intended UTF-16 boundary fix, not a content loss. The partial-transcript cap itself (MAX_PARTIAL_USER_TRANSCRIPT_CHARS) is unchanged.

Compatibility

Plugin SDK export (openclaw/plugin-sdk/text-utility-runtime) is already in package.json exports for @openclaw/plugin-sdk. The helper is dependency-free per src/shared/utf16-slice.ts:1-5 (no node: imports), so it bundles cleanly into both server and UI builds.

What was not tested

Live voice-call session with a real partial transcript that straddles a surrogate pair. The fix is a one-line surrogate-pair guard; the helper is covered by unit tests in src/shared/utf16-slice.ts. Integration coverage at this layer is intentionally out of scope — Alix-007's PRs #98008 (Twitch) and #97982 (Signal) followed the same boundary and were accepted on the same evidence.

Other channel plugins that already use the helper

Discord, Feishu, Telegram, Signal (#97982), Mattermost, Slack, Twitch (#98008), msteams (#98058), iMessage (#98065). voice-call is the new addition.

Related

@openclaw-barnacle openclaw-barnacle Bot added channel: voice-call Channel integration: voice-call size: S labels Jun 30, 2026
@clawsweeper

clawsweeper Bot commented Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 30, 2026, 4:27 AM ET / 08:27 UTC.

Summary
The PR replaces voice-call partial transcript tail slicing with sliceUtf16Safe and adds focused UTF-16 boundary coverage.

PR surface: Source +1, Tests +53. Total +54 across 2 files.

Reproducibility: yes. Source inspection shows current main and v2026.6.10 still use the raw tail slice, and an astral character whose low surrogate lands at the tail-start boundary can leave malformed UTF-16 in the partial transcript.

Review metrics: none identified.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Next step before merge

  • No ClawSweeper repair lane is needed because the branch is focused and has no actionable review findings.

Security
Cleared: No concrete security or supply-chain concern found; the diff only changes TypeScript string truncation and adds a colocated Vitest file.

Review details

Best possible solution:

Land the focused helper substitution after ordinary maintainer review and exact-head checks; keep sibling channel UTF-16 cleanups in their separate channel-specific PRs.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main and v2026.6.10 still use the raw tail slice, and an astral character whose low surrogate lands at the tail-start boundary can leave malformed UTF-16 in the partial transcript.

Is this the best way to solve the issue?

Yes. Reusing the existing public sliceUtf16Safe helper at the raw tail slice is the narrowest maintainable fix; a voice-call-local surrogate guard would duplicate the SDK helper and add a second policy path.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 444a09359377.

Label changes

Label changes:

  • add P3: This is a low-blast-radius voice-call transcript encoding fix with no config, auth, storage, migration, dependency, or delivery-contract change.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied before/after boundary-probe output for the exact tail-slice behavior plus focused test and lint output, which is sufficient for this deterministic non-visual string-boundary fix.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied before/after boundary-probe output for the exact tail-slice behavior plus focused test and lint output, which is sufficient for this deterministic non-visual string-boundary fix.

Label justifications:

  • P3: This is a low-blast-radius voice-call transcript encoding fix with no config, auth, storage, migration, dependency, or delivery-contract change.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied before/after boundary-probe output for the exact tail-slice behavior plus focused test and lint output, which is sufficient for this deterministic non-visual string-boundary fix.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied before/after boundary-probe output for the exact tail-slice behavior plus focused test and lint output, which is sufficient for this deterministic non-visual string-boundary fix.
Evidence reviewed

PR surface:

Source +1, Tests +53. Total +54 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 2 1 +1
Tests 1 53 0 +53
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 55 1 +54

What I checked:

Likely related people:

  • maweibin: git blame and shortlog show the current voice-call realtime handler, affected limitPartialUserTranscript body, and UTF-16 helper snapshot all trace to commit 84cd3aa7f59a16d7589589828c228d64bc980865. (role: introduced behavior and shared helper snapshot; confidence: high; commits: 84cd3aa7f59a; files: extensions/voice-call/src/webhook/realtime-handler.ts, extensions/voice-call/src/webhook/realtime-handler.test.ts, src/shared/utf16-slice.ts)
  • vincentkoc: Live GitHub metadata for fix(cli): call process.exit(1) in root help fast path error handler (#97793) #97807 lists this person as the merger of the PR that brought the current voice-call realtime handler snapshot into main. (role: merger of related current-main snapshot; confidence: medium; commits: 84cd3aa7f59a; files: extensions/voice-call/src/webhook/realtime-handler.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jun 30, 2026
@vincentkoc vincentkoc self-assigned this Jun 30, 2026
@vincentkoc

vincentkoc commented Jun 30, 2026

Copy link
Copy Markdown
Member

Closing this as superseded by the broader UTF-16-safe truncation fix now on main: c16bb87.

I kept this as one canonical pass instead of landing the one-off PRs separately so Browser, Discord, Feishu, iMessage, MS Teams, Signal, Twitch, and Voice Call preview/error truncation all use the shared UTF-16 helpers consistently. Thanks for flagging this surface.

Proof for the landed commit:

  • focused Vitest: shared UTF helper plus Browser, Discord, Feishu, iMessage, MS Teams, Signal, Twitch, and Voice Call touched suites
  • autoreview: clean on the UTF commit
  • Crabbox/Azure changed gate: run_33aba0edad9a, lease cbx_09cf3426b424, passed OPENCLAW_TESTBOX=1 node scripts/crabbox-wrapper.mjs run -- env OPENCLAW_CHECK_CHANGED_REMOTE_CHILD=1 OPENCLAW_CHANGED_LANES_RAW_SYNC=1 corepack pnpm check:changed

@vincentkoc vincentkoc closed this Jun 30, 2026
@wangmiao0668000666 wangmiao0668000666 mentioned this pull request Jul 1, 2026
Closed
@clawsweeper clawsweeper Bot mentioned this pull request Jul 7, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@vincentkoc vincentkoc

Labels

channel: voice-call Channel integration: voice-call P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@wangmiao0668000666 @vincentkoc