fix(agents): keep tool/command display truncation UTF-16 safe by lsr911 · Pull Request #101613 · openclaw/openclaw

lsr911 · 2026-07-07T11:37:45Z

What Problem This Solves

Four agent-module call sites truncate text with raw .slice(0, N) where the boundary can land mid-surrogate-pair — producing U+FFFD in:

tool-search: directory descriptions (177-code-unit boundary)
gateway-tool: restart reason parameter (200-code-unit boundary)
openai-transport-stream: redacted tool-argument/event payloads (8000/2000)
command/attempt-execution: ACP diagnostic text (240-code-unit boundary)

All of these truncate user-visible or agent-generated text that may contain emoji or CJK supplementary characters.

Why This Change Was Made

truncateUtf16Safe from @openclaw/normalization-core/utf16-slice is the canonical leaf helper.

Evidence

Branch prerun (commit `eb05889`)

=== gateway-tool reason @200 ===
Raw .slice ends: "rrrr\ud83d" (dangling high surrogate)
truncateUtf16Safe: "rrr" | len: 199 | no U+FFFD

=== transport payload @8000 ===
Raw .slice ends: "pp\ud83c"
truncateUtf16Safe: len: 7999 | no U+FFFD

=== ACP diagnostic @240 ===
Raw .slice ends: "dd\ud83d"
truncateUtf16Safe: len: 239 | no U+FFFD

=== tool-search directory @177 ===
Raw .slice ends: "dd\ud83d"
truncateUtf16Safe: len: 176 | no U+FFFD

Files Changed

File	Change
`src/agents/tool-search.ts`	`compactDirectoryDescription` @177: `.slice(0, 177)` → `truncateUtf16Safe`
`src/agents/tools/gateway-tool.ts`	Restart `reason` @200: `?.slice(0, 200)` → `truncateUtf16Safe`
`src/agents/openai-transport-stream.ts`	Redacted payload @8000 + event @2000: `.slice()` → `truncateUtf16Safe`
`src/agents/command/attempt-execution.ts`	ACP diagnostic @240: `.slice(0, 240)` → `truncateUtf16Safe`

@240

Replace raw .slice(0, N) with truncateUtf16Safe from @openclaw/normalization-core/utf16-slice across 4 agent modules: - tool-search: compactDirectoryDescription @177-code-unit boundary - gateway-tool: restart reason @200-code-unit boundary - openai-transport-stream: redacted payload/event @8000/2000 - command/attempt-execution: ACP diagnostic text @240 All sites truncate user-visible or agent-generated text where emoji/CJK supplementary characters can be split mid-surrogate.

clawsweeper · 2026-07-07T11:54:03Z

Codex review: needs maintainer review before merge. Reviewed July 7, 2026, 7:59 AM ET / 11:59 UTC.

Summary
The PR replaces raw fixed-length truncation in four agent display/diagnostic paths with truncateUtf16Safe and includes formatter reflow in the touched modules.

PR surface: Source +149. Total +149 across 4 files.

Reproducibility: yes. source-reproducible: current main uses raw .slice(0, N) at the reported boundaries, and JavaScript slicing can leave a dangling surrogate when the limit lands inside an emoji pair. The PR body also includes copied Node.js output exercising each boundary.

Review metrics: none identified.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P2] Add targeted regression tests for the four changed boundaries if maintainers want stronger long-term guard coverage.

Risk before merge

[P1] No focused regression tests were added for the four changed agent call sites, so future guard coverage depends on the shared helper tests plus reviewer attention.

Maintainer options:

Decide the mitigation before merge
Land the focused helper-based truncation cleanup after maintainer review and CI/proof acceptance; small callsite-level regression tests would make the invariant more durable.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

No automated repair is indicated because the branch is focused and no blocking patch defect was found; remaining action is ordinary maintainer review.

Security
Cleared: The diff only imports an existing internal normalization helper and changes redacted/display truncation; no supply-chain, secret-handling, permission, or execution-path regression was found.

Review details

Best possible solution:

Land the focused helper-based truncation cleanup after maintainer review and CI/proof acceptance; small callsite-level regression tests would make the invariant more durable.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: current main uses raw .slice(0, N) at the reported boundaries, and JavaScript slicing can leave a dangling surrogate when the limit lands inside an emoji pair. The PR body also includes copied Node.js output exercising each boundary.

Is this the best way to solve the issue?

Yes, the proposed fix uses the repository's existing truncateUtf16Safe helper at the leaf truncation sites. A bespoke truncator or broader protocol/config change would be less maintainable for this bug.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 7c0a7c8be210.

Label changes

Label changes:

add P3: The PR is a focused display/diagnostic text cleanup with limited blast radius and no evidence of a blocking runtime regression.
add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied Node.js boundary output showing the before-fix dangling surrogate and after-fix surrogate-safe result for each changed limit.
add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied Node.js boundary output showing the before-fix dangling surrogate and after-fix surrogate-safe result for each changed limit.

Label justifications:

P3: The PR is a focused display/diagnostic text cleanup with limited blast radius and no evidence of a blocking runtime regression.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied Node.js boundary output showing the before-fix dangling surrogate and after-fix surrogate-safe result for each changed limit.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied Node.js boundary output showing the before-fix dangling surrogate and after-fix surrogate-safe result for each changed limit.

Evidence reviewed

PR surface:

Source +149. Total +149 across 4 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	4	193	44	+149
Tests	0	0	0	0
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	4	193	44	+149

What I checked:

Current main still has the reported raw slices: Current main truncates tool-directory descriptions, gateway restart reasons, OpenAI transport redacted payload/event previews, and ACP diagnostic text with raw .slice(0, N), so the PR is not already implemented on main. (src/agents/openai-transport-stream.ts:465, 7c0a7c8be210)
Shared helper contract matches the fix: truncateUtf16Safe clamps the requested limit and delegates to sliceUtf16Safe, whose implementation backs off a dangling high surrogate before returning the slice. (packages/normalization-core/src/utf16-slice.ts:44, 7c0a7c8be210)
Existing helper tests cover surrogate-boundary behavior: The normalization-core tests cover regular truncation, zero/negative/decimal limits, and truncation at a surrogate-pair boundary returning no dangling half. (packages/normalization-core/src/utf16-slice.test.ts:56, 7c0a7c8be210)
PR diff uses the existing helper at all reported call sites: The PR head imports truncateUtf16Safe and replaces the four reported raw truncations in agent modules without changing config, storage, auth, dependency, or protocol surfaces. (src/agents/tools/gateway-tool.ts:470, eb05889e6248)
Source-reproducible behavior and copied proof: The PR body includes copied Node.js output showing each raw boundary leaves a dangling surrogate while the helper-backed result avoids U+FFFD at the 177, 200, 240, 2000, and 8000 code-unit limits. (eb05889e6248)
History and current-main provenance: git blame ties the current raw slices and helper to recent main history, and git merge-base --is-ancestor returned 1 for the PR head against current main, confirming this branch is not already contained. (src/agents/tool-search.ts:1135, 240487179b69)

Likely related people:

Alix-007: Blame on the current raw truncation lines and the shared UTF-16 helper points to commit 2404871, which recently touched these central files and introduced the helper surface. (role: recent area contributor; confidence: high; commits: 240487179b69; files: src/agents/tool-search.ts, src/agents/openai-transport-stream.ts, src/agents/tools/gateway-tool.ts)
lsr911: The same contributor has recent merged OpenClaw UTF-16 truncation cleanup in adjacent ACP code via pull request fix(acp): use truncateUtf16Safe for event mapper error text truncation #101535. (role: adjacent contributor; confidence: medium; commits: 42fc9e8a92f9; files: src/acp/event-mapper.ts, src/acp/event-mapper.test.ts)
steipete: The latest current-main commit touching src/agents/tool-search.ts is 7c0a7c8, so this is a useful routing signal for that agent tool-search surface. (role: recent adjacent owner; confidence: medium; commits: 7c0a7c8be210; files: src/agents/tool-search.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

steipete · 2026-07-07T12:45:58Z

Thanks @lsr911. The useful behavior from this PR was consolidated with the sibling UTF-16 boundary fixes into #101654 and landed on main in a9582a1bb62aa70bb0b0ceb72536d0d76f08eab8.

The landed fix uses the existing truncateUtf16Safe helper, preserves this caller's prior code-unit limit and output shape, and adds caller-level surrogate-boundary coverage. Closing this PR as superseded by the canonical landed change.

openclaw-barnacle Bot added agents Agent runtime and tooling size: M triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jul 7, 2026

openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jul 7, 2026

steipete mentioned this pull request Jul 7, 2026

fix: keep bounded text truncation UTF-16 safe #101654

Merged

steipete closed this Jul 7, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(agents): keep tool/command display truncation UTF-16 safe#101613

fix(agents): keep tool/command display truncation UTF-16 safe#101613
lsr911 wants to merge 1 commit into
openclaw:mainfrom
lsr911:fix/agents-utf16-safe-truncation

lsr911 commented Jul 7, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

lsr911 commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

Evidence

Branch prerun (commit eb05889)

Files Changed

Uh oh!

clawsweeper Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lsr911 commented Jul 7, 2026 •

edited

Loading

Branch prerun (commit `eb05889`)

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading