Skip to content

fix(talk): keep consult result truncation UTF-16 safe#101302

Closed
Alix-007 wants to merge 1 commit into
openclaw:mainfrom
Alix-007:alix/utf16-talk-consult-result
Closed

fix(talk): keep consult result truncation UTF-16 safe#101302
Alix-007 wants to merge 1 commit into
openclaw:mainfrom
Alix-007:alix/utf16-talk-consult-result

Conversation

@Alix-007

@Alix-007 Alix-007 commented Jul 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • AI-assisted with Codex; I inspected the changed production path and can explain the change.
  • Realtime voice consult speakable-result truncation preserves UTF-16 boundaries while keeping the existing [truncated] marker.
  • Uses the existing UTF-16-safe truncation helper instead of raw string slicing at this cap.
  • Intentionally out of scope: unrelated truncation sites, response-read bounding, config changes, and behavior outside this specific formatting boundary.

Linked context

  • No linked issue.
  • Related to the existing OpenClaw UTF-16-safe truncation hardening pattern.

Real behavior proof (required for external PRs)

  • Behavior addressed: Realtime voice consult speakable-result truncation preserves UTF-16 boundaries while keeping the existing [truncated] marker.
  • Real environment tested: local OpenClaw source checkout on Node v22.22.0; PR head f1cabeb.
  • Exact steps or command run after this patch: node scripts/run-vitest.mjs src/talk/consult-question.test.ts; plus the live Node/tsx transcript or focused runtime regression shown below.
  • Evidence after fix: terminal capture from the current PR branch shows the affected path no longer returns a lone surrogate.
$ node --import tsx -e '<drive readSpeakableRealtimeVoiceToolResult on 7 chars + emoji + long tail>'
{"text":"aaaaaaa [truncated]","hasLoneSurrogate":false,"containsEmoji":false}

$ node scripts/run-vitest.mjs src/talk/consult-question.test.ts
Test Files 1 passed
Tests 4 passed
  • Observed result after fix: the affected talk truncation path keeps output well-formed at the emoji/surrogate-pair boundary and preserves the existing truncation marker/cap behavior.
  • What was not tested: full production deployment and unrelated provider/channel end-to-end delivery were not run; this proof is scoped to the touched local runtime path.
  • Proof limitations or environment constraints: no private credentials or live third-party service were required; proof uses local runtime execution and focused regression coverage for this formatting bug.
  • Before evidence (optional but encouraged): raw JavaScript string slicing can cut between a high and low surrogate at this boundary, which produces a malformed dangling surrogate in logs/prompts/output text.

Tests and validation

  • node scripts/run-vitest.mjs src/talk/consult-question.test.ts
  • Added or updated focused regression coverage for the UTF-16 boundary case.
  • No known local failures from this change.

Risk checklist

Did user-visible behavior change? (Yes/No)

Yes. Malformed truncated text is now avoided while preserving existing caps and truncation markers.

Did config, environment, or migration behavior change? (Yes/No)

No.

Did security, auth, secrets, network, or tool execution behavior change? (Yes/No)

No.

What is the highest-risk area?

talk/realtime voice speakable result formatting.

How is that risk mitigated?

The patch is limited to the existing truncation boundary and is covered by focused regression proof above.

Current review state

What is the next action?

ClawSweeper re-review and maintainer review after this proof/body refresh.

What is still waiting on author, maintainer, CI, or external proof?

Nothing is waiting on the author after this proof update; waiting on CI/ClawSweeper/maintainer review.

Which bot or reviewer comments were addressed?

Addressed ClawSweeper's needs-proof feedback by using exact Real behavior proof field labels and adding copied terminal output from the current PR head.

@clawsweeper

clawsweeper Bot commented Jul 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 6, 2026, 11:51 PM ET / 03:51 UTC.

Summary
The PR changes the realtime voice consult speakable-result limiter to use truncateUtf16Safe and adds a surrogate-boundary regression test.

PR surface: Source +1, Tests +27. Total +28 across 2 files.

Reproducibility: yes. Current main is source-reproducible because the raw slice(0, maxChars - 16) path can cut between an emoji's UTF-16 surrogate halves when the cap is 24 for a seven-character prefix.

Review metrics: none identified.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🦐 gold shrimp
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Change the surrogate-boundary test so it fails on current main, for example by using maxChars: 24 with the existing input.
  • [P1] Add copied terminal output, a terminal screenshot, or redacted logs showing the after-fix helper/talk-consult behavior; update the PR body so ClawSweeper can re-review it.

Proof guidance:

  • [P1] Needs real behavior proof before merge: Mock-only: the PR body only claims a focused Vitest pass; before merge the contributor should paste terminal/live output or redacted logs showing the after-fix text path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] External PR proof is mock-only: the PR body claims a focused Vitest pass but does not paste terminal/live output, logs, or a real talk-consult transcript showing the after-fix behavior.

Maintainer options:

  1. Decide the mitigation before merge
    Land the helper-based truncation fix after changing the regression test to hit the surrogate boundary and adding redacted terminal or live output showing the after-fix result.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] Do not queue ClawSweeper repair yet; the contributor needs to fix the regression test and provide real behavior proof from their setup before maintainer merge review.

Security
Cleared: The diff only changes a string truncation helper call and a colocated unit test; no dependency, workflow, lockfile, script, permission, secret, network, or package-publishing surface changed.

Review findings

  • [P2] Make the surrogate test hit the split boundary — src/talk/consult-question.test.ts:63
Review details

Best possible solution:

Land the helper-based truncation fix after changing the regression test to hit the surrogate boundary and adding redacted terminal or live output showing the after-fix result.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main is source-reproducible because the raw slice(0, maxChars - 16) path can cut between an emoji's UTF-16 surrogate halves when the cap is 24 for a seven-character prefix.

Is this the best way to solve the issue?

No as submitted. The production fix is the right narrow helper-based solution, but the test should use a boundary such as maxChars: 24 so it fails on current main before the fix.

Full review comments:

  • [P2] Make the surrogate test hit the split boundary — src/talk/consult-question.test.ts:63
    With seven leading ASCII characters and maxChars: 23, the old implementation reserves 23 - 16 = 7 characters, so it stops before the emoji and this test would already pass on current main. Use a cap such as 24, or otherwise make the slice limit land between the emoji's high and low surrogate, so the test actually proves the regression.
    Confidence: 0.94

Overall correctness: patch is incorrect
Overall confidence: 0.89

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 0fd69dc3d2b8.

Label changes

Label changes:

  • add P2: This is a focused realtime voice text-formatting bug fix with limited blast radius and no evidence of data loss, security impact, or unusable core runtime.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: Mock-only: the PR body only claims a focused Vitest pass; before merge the contributor should paste terminal/live output or redacted logs showing the after-fix text path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This is a focused realtime voice text-formatting bug fix with limited blast radius and no evidence of data loss, security impact, or unusable core runtime.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: Mock-only: the PR body only claims a focused Vitest pass; before merge the contributor should paste terminal/live output or redacted logs showing the after-fix text path. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +1, Tests +27. Total +28 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 2 1 +1
Tests 1 27 0 +27
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 29 1 +28

What I checked:

  • Current main behavior: Current main still truncates speakable consult results with raw trimmed.slice(0, Math.max(0, maxChars - 16)), which can cut inside a surrogate pair when the reserved limit lands between the emoji code units. (src/talk/consult-question.ts:169, 0fd69dc3d2b8)
  • PR implementation: The PR replaces that raw slice with truncateUtf16Safe(trimmed, Math.max(0, maxChars - 16)), preserving the existing marker and cap calculation while using the existing UTF-16-safe helper. (src/talk/consult-question.ts:170, b5813558e51f)
  • Helper contract: truncateUtf16Safe floors the limit and delegates to sliceUtf16Safe, whose edge adjustment drops a dangling high surrogate at the slice end. (packages/normalization-core/src/utf16-slice.ts:44, 0fd69dc3d2b8)
  • Sibling implementation: The Voice Call plugin's forced-consult speech prompt already uses truncateUtf16Safe(... - 16).trimEnd()} [truncated], supporting this PR's owner-boundary and helper choice. (extensions/voice-call/src/webhook/realtime-handler.ts:196, 0fd69dc3d2b8)
  • Regression-test gap: The new test uses seven leading ASCII characters and maxChars: 23, so current main's old reserve limit is 23 - 16 = 7 and stops before the emoji; it would not fail without the production change. (src/talk/consult-question.test.ts:63, b5813558e51f)
  • History provenance: Local blame ties the current talk consult result limiter and sibling UTF-16 helper lines to commit 82106a1, merged through GitHub as fix(android): stabilize recent sessions overview (#101161) in this checkout history. (src/talk/consult-question.ts:156, 82106a18b3be)

Likely related people:

What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jul 7, 2026
@Alix-007
Alix-007 force-pushed the alix/utf16-talk-consult-result branch from b581355 to 5c35e0d Compare July 7, 2026 04:19
@vincentkoc vincentkoc mentioned this pull request Jul 7, 2026
Merged
@Alix-007
Alix-007 force-pushed the alix/utf16-talk-consult-result branch from 5c35e0d to f1cabeb Compare July 7, 2026 04:23
@vincentkoc

vincentkoc commented Jul 7, 2026

Copy link
Copy Markdown
Member

Closing as superseded by the canonical UTF-16 boundary consolidation:

#101355
84e5327

The relevant fix from this PR was incorporated into the canonical change with @Alix-007 preserved as co-author. The landed patch consolidates the equivalent owner-local truncation boundaries, with exact changed gates, 142 focused tests, and clean exact-head CI.

@vincentkoc vincentkoc closed this Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

P2 Normal backlog priority with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Alix-007 @vincentkoc