fix(shared): use UTF-16 safe truncation in assistant error formatting by zenglingbiao · Pull Request #97289 · openclaw/openclaw

zenglingbiao · 2026-06-27T17:40:34Z

Summary

Replace String.prototype.slice(0, 600) with truncateUtf16Safe() in two assistant error formatting functions — formatRawAssistantErrorForUi and formatAssistantErrorText (plus its companion comparison in isRawAssistantErrorPassthrough) — to prevent broken Unicode from surrogate-pair splitting when error messages contain emoji or non-BMP characters.

What Problem This Solves

When formatRawAssistantErrorForUi receives an error message containing characters outside the Basic Multilingual Plane (emoji like 🎉, rare CJK characters, etc.), the truncation at line 249 can split a UTF-16 surrogate pair in half. This produces a dangling surrogate code unit — a malformed Unicode character displayed as � (replacement character) in TUI and chat channels.

Code evidence:

In src/shared/assistant-error-format.ts:250, formatRawAssistantErrorForUi uses trimmed.slice(0, 600) for the fallback raw error preview
In src/agents/embedded-agent-helpers/errors.ts:1543, formatAssistantErrorText uses the same pattern for its raw fallback, and its companion comparison in isRawAssistantErrorPassthrough at line 1563 also uses .slice(0, 600)

Both operate on UTF-16 code units, not code points. Characters outside BMP (U+10000+) are encoded as two code units (surrogate pairs); slicing at an arbitrary index can cut between them.

Root Cause

Introduced by: commit 397b0d85f57 (2026-03-19, refactor: split assistant error formatting)
Violated invariant: String.prototype.slice(n) assumes all characters are single-code-unit BMP characters. In reality, JavaScript strings are UTF-16 — supplementary-plane characters occupy 2 code units.
Trigger condition: An LLM API error response containing an emoji or non-BMP character at the 600-character truncation boundary. For example, a 601-code-unit message with an emoji starting at position 599.

Why This Fix

Chosen approach: Use the existing truncateUtf16Safe utility from src/shared/utf16-slice.ts (same directory, zero new dependencies).
Alternative considered: Regex-based approach or manual surrogate checking — rejected because the utility already exists and is well-tested across the codebase.
Lines changed: 5 (+3 imports, +2 call changes across 2 files), minimal surface area.

User Impact

Affected: Error messages displayed in TUI (src/tui/), auto-reply channels, and agent payload builders — any path that calls formatRawAssistantErrorForUi.
Before: Error text could end with a broken character � if the original message contained an emoji near the 600-char boundary.
After: Error text is always truncated on a valid code-point boundary.

Evidence

Terminal proof script demonstrating the bug and fix using a 601-code-unit error message containing a 🎉 emoji at the truncation boundary:

Before (on main, String.prototype.slice(0, 600)):

Result length: 601 code units (600 truncated + … ellipsis)
FAIL: Dangling high surrogate at position 599 — the emoji surrogate pair was split in half
Users see garbled � in error messages

After (with fix, truncateUtf16Safe):

Result length: 600 code units (599 safe chars + … ellipsis)
PASS: The surrogate pair is preserved intact, truncation occurs at the nearest safe code-point boundary

Real Behavior Proof

Before (on main)

$ node --import tsx proof.mjs
=== PROOF: UTF-16 safe truncation in formatRawAssistantErrorForUi ===

Input length: 601 UTF-16 code units
Input ends with emoji: true

Truncated result length: 601

FAIL: Dangling surrogate detected at positions: 599
The truncation split a surrogate pair, producing malformed Unicode.
Users would see garbled characters (�) in error messages.

After (with fix)

$ node --import tsx proof.mjs
=== PROOF: UTF-16 safe truncation in formatRawAssistantErrorForUi ===

Input length: 601 UTF-16 code units
Input ends with emoji: true

Truncated result length: 600

PASS: No dangling surrogate — truncation is UTF-16 safe.

Tests and Validation

pnpm check passed
TypeScript compilation verified (git diff --check clean)
Existing formatRawAssistantErrorForUi unit tests continue to pass
New regression test added: formatRawAssistantErrorForUi with non-BMP character at the 600-char truncation boundary
Manual proof script confirms the fix prevents surrogate-pair splitting in both formatting functions

Use truncateUtf16Safe from the existing utf16-slice utility instead of String.prototype.slice(0, 600), which can split a surrogate pair and produce malformed Unicode when the error message contains characters outside the Basic Multilingual Plane (emoji, rare CJK, etc.).

clawsweeper · 2026-06-27T17:41:35Z

Codex review: needs maintainer review before merge. Reviewed June 27, 2026, 2:25 PM ET / 18:25 UTC.

Summary
The branch replaces raw 600-code-unit assistant error truncation with the existing UTF-16-safe helper in shared and embedded-agent error formatters, and adds a focused regression test.

PR surface: Source +2, Tests +27. Total +29 across 3 files.

Reproducibility: yes. Current main uses raw UTF-16 slice(0, 600) at the reported formatter fallbacks, and a focused node check shows the same boundary split leaves a dangling surrogate.

Review metrics: none identified.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Next step before merge

[P2] No repair lane is needed; the remaining action is ordinary maintainer review and exact-head validation.

Security
Cleared: The diff only reuses an existing local string helper and adds a unit test; it does not touch dependencies, workflows, permissions, secrets, package resolution, or code-execution surfaces.

Review details

Best possible solution:

Land the narrow helper reuse after normal exact-head gates; leave broader truncation sweeps to separate surface-specific PRs.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main uses raw UTF-16 slice(0, 600) at the reported formatter fallbacks, and a focused node check shows the same boundary split leaves a dangling surrogate.

Is this the best way to solve the issue?

Yes. Reusing the existing dependency-free truncateUtf16Safe helper at the two assistant fallback truncation sites is the narrowest maintainable fix; duplicating surrogate logic or adding a new utility would be worse.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 98ed83f848de.

Label changes

Label justifications:

P2: This is a normal-priority user-visible assistant error formatting bugfix with limited blast radius and a narrow patch.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes copied before/after terminal output showing the current dangling-surrogate failure and the helper-based passing result after the fix.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied before/after terminal output showing the current dangling-surrogate failure and the helper-based passing result after the fix.

Evidence reviewed

PR surface:

Source +2, Tests +27. Total +29 across 3 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	2	5	3	+2
Tests	1	27	0	+27
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	3	32	3	+29

What I checked:

Repository policy read: Read the full root AGENTS.md and the scoped src/agents/AGENTS.md; the review applied the repo's requirement to inspect the changed module, callers, sibling helper usage, tests, and history. (AGENTS.md:1, 98ed83f848de)
Current main raw shared truncation: formatRawAssistantErrorForUi still truncates fallback raw assistant errors with trimmed.slice(0, 600) on current main, matching the reported surrogate-pair split surface. (src/shared/assistant-error-format.ts:249, 98ed83f848de)
Current main raw agent fallback truncation: formatAssistantErrorText and isRawAssistantErrorPassthrough still compare against raw slice(0, 600) fallback text on current main. (src/agents/embedded-agent-helpers/errors.ts:1543, 98ed83f848de)
UTF-16 helper contract: truncateUtf16Safe delegates to sliceUtf16Safe, which adjusts slice edges away from paired surrogate halves and stays dependency-free for shared/browser-safe imports. (src/shared/utf16-slice.ts:46, 98ed83f848de)
Patch covers reported assistant formatter sites: The PR imports truncateUtf16Safe and changes both fallback formatter returns plus the passthrough comparison to use the safe helper. (src/agents/embedded-agent-helpers/errors.ts:1541, 2e292ec05d63)
Regression coverage: The added test builds a 601-code-unit raw error with an emoji straddling the 600-unit boundary and checks that the formatted result has no dangling surrogate before the ellipsis. (src/agents/embedded-agent-helpers.formatassistanterrortext.test.ts:650, 2e292ec05d63)

Likely related people:

vincentkoc: Commit 397b0d8 introduced src/shared/assistant-error-format.ts, and live PR metadata shows vincentkoc merged the related UTF-16 helper PR fix(agents): truncate tool-display detail on code-point boundaries #96958. (role: introduced formatter seam and recent merger for related helper work; confidence: high; commits: 397b0d85f571, e09b9dfc1ba9; files: src/shared/assistant-error-format.ts, src/tui/tui-formatters.ts, src/shared/utf16-slice.ts)
Bartok9: Live PR metadata for fix(agents): truncate tool-display detail on code-point boundaries #96958 shows Bartok9 authored the merged commit that added src/shared/utf16-slice.ts and related surrogate-safe truncation tests. (role: adjacent helper contributor; confidence: high; commits: e09b9dfc1ba9; files: src/shared/utf16-slice.ts, src/agents/tool-display-common.ts, src/agents/tool-display-common.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Verify formatRawAssistantErrorForUi does not produce dangling surrogates when truncating a fallback raw error message with a non-BMP character (emoji) straddling the 600-code-unit boundary.

zenglingbiao · 2026-06-27T17:54:40Z

@clawsweeper re-review

clawsweeper · 2026-06-27T17:54:43Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Use truncateUtf16Safe instead of String.prototype.slice(0,600) in formatAssistantErrorText and its companion comparison in isRawAssistantErrorPassthrough, preventing dangling surrogates when non-BMP characters straddle the 600-code-unit truncation boundary.

…openclaw#97289) * fix(shared): truncate assistant error text on UTF-16 boundary Use truncateUtf16Safe from the existing utf16-slice utility instead of String.prototype.slice(0, 600), which can split a surrogate pair and produce malformed Unicode when the error message contains characters outside the Basic Multilingual Plane (emoji, rare CJK, etc.). * test(shared): add regression test for UTF-16 safe error truncation Verify formatRawAssistantErrorForUi does not produce dangling surrogates when truncating a fallback raw error message with a non-BMP character (emoji) straddling the 600-code-unit boundary. * fix(agents): truncate assistant error text fallback on UTF-16 boundary Use truncateUtf16Safe instead of String.prototype.slice(0,600) in formatAssistantErrorText and its companion comparison in isRawAssistantErrorPassthrough, preventing dangling surrogates when non-BMP characters straddle the 600-code-unit truncation boundary.

openclaw-barnacle Bot added size: XS triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jun 27, 2026

openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 27, 2026

test(shared): add regression test for UTF-16 safe error truncation

c004a62

Verify formatRawAssistantErrorForUi does not produce dangling surrogates when truncating a fallback raw error message with a non-BMP character (emoji) straddling the 600-code-unit boundary.

openclaw-barnacle Bot added the agents Agent runtime and tooling label Jun 27, 2026

zenglingbiao changed the title ~~fix(shared): truncate assistant error text on UTF-16 boundary~~ fix(shared): use UTF-16 safe truncation in assistant error formatting Jun 27, 2026

This was referenced Jun 27, 2026

fix(cron): truncate failure alert error text on UTF-16 boundary #97298

Merged

fix(auto-reply): truncate user-facing text on UTF-16 boundary #97299

Merged

vincentkoc merged commit e445d61 into openclaw:main Jun 28, 2026
101 of 105 checks passed

github-actions Bot mentioned this pull request Jun 28, 2026

📡 Upstream Digest — 2026-06-28 19:05 UTC curtismercier/openclaw-mods#1168

Open

This was referenced Jul 9, 2026

fix(agents): keep error log truncation UTF-16 safe #102645

Closed

fix(agents-errors): use truncateUtf16Safe for error log truncation #102679

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(shared): use UTF-16 safe truncation in assistant error formatting#97289

fix(shared): use UTF-16 safe truncation in assistant error formatting#97289
vincentkoc merged 3 commits into
openclaw:mainfrom
zenglingbiao:fix/utf16-safe-assistant-error-truncation

zenglingbiao commented Jun 27, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading

Uh oh!

zenglingbiao commented Jun 27, 2026

Uh oh!

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

zenglingbiao commented Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What Problem This Solves

Root Cause

Why This Fix

User Impact

Evidence

Real Behavior Proof

Before (on main)

After (with fix)

Tests and Validation

Uh oh!

clawsweeper Bot commented Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zenglingbiao commented Jun 27, 2026

Uh oh!

clawsweeper Bot commented Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

zenglingbiao commented Jun 27, 2026 •

edited

Loading

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading