fix(agents): keep tool-result truncation UTF-16 safe by chengzhichao-xydt · Pull Request #102087 · openclaw/openclaw

chengzhichao-xydt · 2026-07-08T07:01:04Z

Summary

AI-assisted with Codex; I inspected the changed production path and can explain the change.
Tool-result truncation now preserves UTF-16 boundaries when capping single results, detecting important error tails, and producing aggregate elision markers.
Uses the existing @openclaw/normalization-core/utf16-slice helpers (sliceUtf16Safe / truncateUtf16Safe) instead of raw string slicing.
Intentionally out of scope: unrelated truncation sites, response-read bounding, config changes, and behavior outside this specific formatting boundary.

Linked context

No linked issue.
Related to the existing OpenClaw UTF-16-safe truncation hardening pattern (follow-up to Alix-007's merged UTF-16 campaign).

Real behavior proof (required for external PRs)

Behavior addressed: Tool-result truncation preserves UTF-16 boundaries when capping single results, detecting important error tails, and producing aggregate elision markers.
Real environment tested: local OpenClaw source checkout on Node v22.22.0; PR head e050e01.
Exact steps or command run after this patch: node --import tsx directly drives the exported truncation functions and checks for lone surrogates; plus node scripts/run-vitest.mjs src/agents/embedded-agent-runner/tool-result-truncation.test.ts.
Evidence after fix: terminal capture from the current PR branch shows the affected paths no longer return a lone surrogate.

$ node --import tsx -e '<drive truncateToolResultText + truncateToolResultMessage on emoji-crossing-boundary inputs>'
simple truncation lone surrogate: false
head+tail truncation lone surrogate: false
head+tail truncation keeps error: true
message truncation lone surrogate: false

$ node scripts/run-vitest.mjs src/agents/embedded-agent-runner/tool-result-truncation.test.ts
[test] starting test/vitest/vitest.agents.config.ts
...
Test Files 1 passed (1)
Tests 59 passed (59)

Observed result after fix: the affected tool-result truncation paths keep output well-formed at the emoji/surrogate-pair boundary and preserve existing truncation markers, caps, and error-tail detection.
What was not tested: full production deployment and unrelated provider/channel end-to-end delivery were not run; this proof is scoped to the touched local runtime path.
Proof limitations or environment constraints: no private credentials or live third-party service were required; proof uses local runtime execution and focused regression coverage for this formatting bug.
Before evidence (optional but encouraged): raw JavaScript string slicing can cut between a high and low surrogate at these boundaries, which produces a malformed dangling surrogate in tool output/prompts.

Tests and validation

node scripts/run-vitest.mjs src/agents/embedded-agent-runner/tool-result-truncation.test.ts
Added focused regression coverage for the UTF-16 boundary case (simple truncation and head+tail error preservation).
No known local failures from this change.

Risk checklist

Did user-visible behavior change? (Yes/No)

Yes. Malformed truncated tool output is now avoided while preserving existing caps and truncation markers.

Did config, environment, or migration behavior change? (Yes/No)

No.

Did security, auth, secrets, network, or tool execution behavior change? (Yes/No)

No.

What is the highest-risk area?

agent tool-result formatting.

How is that risk mitigated?

The patch is limited to the existing truncation boundaries and is covered by focused regression proof above.

Current review state

What is the next action?

ClawSweeper re-review and maintainer review after this proof/body refresh.

What is still waiting on author, maintainer, CI, or external proof?

Nothing is waiting on the author after this proof update; waiting on CI/ClawSweeper/maintainer review.

Which bot or reviewer comments were addressed?

N/A — new PR.

Replace raw .slice() calls in tool-result truncation with UTF-16-safe helpers (sliceUtf16Safe/truncateUtf16Safe) so surrogate pairs are not split when capping tool output, error-tail detection, or aggregate elision markers. - hasImportantTail: sliceUtf16Safe(text, -2000) - appendBoundedTruncationSuffix: truncateUtf16Safe + sliceUtf16Safe - truncateToolResultText head/tail/default paths: sliceUtf16Safe - formatAggregateElisionText fallback: truncateUtf16Safe Added regression tests for emoji at char and head+tail boundaries. Co-Authored-By: Claude Opus 4.8 <[email protected]>

clawsweeper · 2026-07-08T07:17:20Z

Codex review: needs maintainer review before merge. Reviewed July 8, 2026, 11:40 PM ET / 03:40 UTC.

Summary
The PR changes agent tool-result truncation to use shared UTF-16-safe slicing helpers at the single-result and head/tail truncation boundaries, with focused emoji-boundary regression tests.

PR surface: Source +3, Tests +26. Total +29 across 2 files.

Reproducibility: yes. source-reproducible: current main slices tool-result strings by UTF-16 code units at the affected truncation boundaries, so an emoji crossing those indices can leave a dangling surrogate. The PR body also includes after-fix terminal output for the changed functions and focused Vitest coverage.

Review metrics: none identified.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Next step before merge

No ClawSweeper repair job is needed because there are no blocking review findings; the remaining path is maintainer review and exact-head required-check gating.

Security
Cleared: The diff only changes TypeScript string slicing in an existing agent-runtime helper and focused tests; it does not touch dependencies, workflows, secrets, auth, network, or package execution surfaces.

Review details

Best possible solution:

Merge the narrow agent formatting fix after ordinary maintainer review and exact-head required checks; keep broader UTF-16 hardening in separate focused PRs.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: current main slices tool-result strings by UTF-16 code units at the affected truncation boundaries, so an emoji crossing those indices can leave a dangling surrogate. The PR body also includes after-fix terminal output for the changed functions and focused Vitest coverage.

Is this the best way to solve the issue?

Yes: using the existing shared UTF-16 helpers at the exact tool-result truncation boundary is the narrowest maintainable fix. The aggregate marker fallback remaining on raw .slice() is acceptable because that marker is ASCII-only.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 003bb8284bea.

Label changes

Label justifications:

P2: This is a bounded agent-runtime formatting bug fix with limited blast radius and no evidence of a current P0/P1 user blocker.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes terminal output from a local OpenClaw checkout directly exercising the changed truncation functions on emoji-boundary inputs and the focused Vitest file after the patch.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal output from a local OpenClaw checkout directly exercising the changed truncation functions on emoji-boundary inputs and the focused Vitest file after the patch.

Evidence reviewed

PR surface:

Source +3, Tests +26. Total +29 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	10	7	+3
Tests	1	26	0	+26
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	36	7	+29

What I checked:

Current main still has raw slicing at the affected boundary: Current main uses raw .slice() in appendBoundedTruncationSuffix, tail detection, head/tail retention, and default retention, so the central change is not already present on main. (src/agents/embedded-agent-runner/tool-result-truncation.ts:142, 003bb8284bea)
PR applies existing UTF-16-safe helpers to the truncation sites: The PR imports sliceUtf16Safe and truncateUtf16Safe, then uses them for suffix-only fallback, kept head/tail slices, default kept text, and important-tail scanning. (src/agents/embedded-agent-runner/tool-result-truncation.ts:140, d361eab4586b)
Focused regression coverage checks exact UTF-16 cuts: The PR adds tests for direct truncation, suffix-only truncation, and head/tail error-preserving truncation where emoji would straddle the cut point. (src/agents/embedded-agent-runner/tool-result-truncation.test.ts:167, d361eab4586b)
Shared helper contract matches the patch: sliceUtf16Safe adjusts start and end away from dangling surrogate halves, and truncateUtf16Safe delegates through that safe slice. (packages/normalization-core/src/utf16-slice.ts:16, 003bb8284bea)
Runtime callers are live prompt paths: The truncation helper feeds prompt-history truncation before model submission and the provider prompt-history transform, so this is a live agent formatting path. (src/agents/embedded-agent-runner/run/attempt.ts:4361, 003bb8284bea)
Codex contract checked directly: Sibling Codex source converts tool outputs into textual function-call output bodies, supporting the need to keep OpenClaw tool-result text well-formed before prompt submission. (../codex/codex-rs/tools/src/tool_output.rs:135, a7c72aee8b38)

Likely related people:

obviyus: Authored recent main commits changing aggregate tool-result budgets, spill pointers, and elision behavior in the same truncation module. (role: recent area contributor; confidence: high; commits: ad826faf0dce, 168a4890bb4b; files: src/agents/embedded-agent-runner/tool-result-truncation.ts)
momothemage: Authored the merged fresh-tool-result aggregate cap work that this truncation path builds on. (role: feature-history contributor; confidence: high; commits: d45b8be939a1; files: src/agents/embedded-agent-runner/tool-result-truncation.ts)
vincentkoc: Carried a series of merged fixes around live tool-result prompt projection, transformed tool results, and cache-stable prompt history in this module. (role: agent prompt-projection contributor; confidence: high; commits: 708c1b31e097, a60947fb3e92, 30925601aef7; files: src/agents/embedded-agent-runner/tool-result-truncation.ts, src/agents/embedded-agent-runner/run/attempt.ts)
steipete: Landed the reusable AI/normalization package extraction and later fixed sliceUtf16Safe semantics, which define the helper contract used by this PR. (role: adjacent helper contributor; confidence: medium; commits: 062f88e3e3af, b22c36f1125c; files: packages/normalization-core/src/utf16-slice.ts)
AdrianIp0204: Authored the merged change that taught this truncation module and its tests to handle Codex-style nested toolResult blocks. (role: Codex tool-result contributor; confidence: medium; commits: dbd4c98b025c; files: src/agents/embedded-agent-runner/tool-result-truncation.ts, src/agents/embedded-agent-runner/tool-result-truncation.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (1 earlier review cycle)

reviewed 2026-07-08T07:21:39.224Z sha e050e01 :: needs maintainer review before merge. :: none

chengzhichao-xydt · 2026-07-08T07:24:01Z

@vincentkoc this is a narrow agent-runtime formatting fix: tool-result truncation was using raw .slice() and could split UTF-16 surrogate pairs, so I switched the affected sites to the existing sliceUtf16Safe/truncateUtf16Safe helpers and added emoji-boundary regression tests. ClawSweeper rated it 🐚 platinum hermit / proof: sufficient / status: 👀 ready for maintainer look; checks are green except for an unrelated actionlint checkout-timeout failure. Does the approach look right to you?

steipete · 2026-07-09T03:49:13Z

Land-ready review complete on exact head d361eab4586b55326dc97ba8225dc6305252dd8d.

Reviewed the shared truncation owner, its live/persisted callers, the normalization-core helper contract, and sibling stderr-tail behavior.
Maintainer fixup removed the false-positive fixed-ASCII marker change and replaced broad assertions with exact production-path outputs for direct, head/tail, retained-text, and suffix-only cuts.
git diff --check and targeted formatting passed.
Fresh autoreview is clean at confidence 0.98.
Exact-head hosted CI run 28992097512 passed after rerunning one unrelated flaky MCP-session test shard.

No remaining findings or proof gaps.

steipete · 2026-07-09T03:49:28Z

Merged via squash.

Prepared head SHA: d361eab4586b55326dc97ba8225dc6305252dd8d
Landed commit: 9e75ce9dd1e10d9a25a4d85efd21a12be955b030

* fix(agents): keep tool-result truncation UTF-16 safe Replace raw .slice() calls in tool-result truncation with UTF-16-safe helpers (sliceUtf16Safe/truncateUtf16Safe) so surrogate pairs are not split when capping tool output, error-tail detection, or aggregate elision markers. - hasImportantTail: sliceUtf16Safe(text, -2000) - appendBoundedTruncationSuffix: truncateUtf16Safe + sliceUtf16Safe - truncateToolResultText head/tail/default paths: sliceUtf16Safe - formatAggregateElisionText fallback: truncateUtf16Safe Added regression tests for emoji at char and head+tail boundaries. Co-Authored-By: Claude Opus 4.8 <[email protected]> * test(agents): cover exact tool-result UTF-16 cuts --------- Co-authored-by: chengzhichao-xydt <[email protected]> Co-authored-by: Claude Opus 4.8 <[email protected]> Co-authored-by: Peter Steinberger <[email protected]>

* fix(agents): keep tool-result truncation UTF-16 safe Replace raw .slice() calls in tool-result truncation with UTF-16-safe helpers (sliceUtf16Safe/truncateUtf16Safe) so surrogate pairs are not split when capping tool output, error-tail detection, or aggregate elision markers. - hasImportantTail: sliceUtf16Safe(text, -2000) - appendBoundedTruncationSuffix: truncateUtf16Safe + sliceUtf16Safe - truncateToolResultText head/tail/default paths: sliceUtf16Safe - formatAggregateElisionText fallback: truncateUtf16Safe Added regression tests for emoji at char and head+tail boundaries. Co-Authored-By: Claude Opus 4.8 <[email protected]> * test(agents): cover exact tool-result UTF-16 cuts --------- Co-authored-by: chengzhichao-xydt <[email protected]> Co-authored-by: Claude Opus 4.8 <[email protected]> Co-authored-by: Peter Steinberger <[email protected]> (cherry picked from commit 9e75ce9)

openclaw-barnacle Bot added agents Agent runtime and tooling size: S labels Jul 8, 2026

test(agents): cover exact tool-result UTF-16 cuts

d361eab

openclaw-barnacle Bot added size: XS and removed size: S labels Jul 9, 2026

steipete merged commit 9e75ce9 into openclaw:main Jul 9, 2026
160 of 163 checks passed

This was referenced Jul 9, 2026

fix(agents): keep tool-result context guard truncation UTF-16 safe #102466

Merged

fix(gateway): re-check session runtime model against current agent defaults after hot-reload #102305

Merged

wangmiao0668000666 mentioned this pull request Jul 9, 2026

fix(text): keep five src-side text truncations UTF-16 safe #102630

Closed

9 tasks

maweibin mentioned this pull request Jul 9, 2026

fix(auto-reply): keep fallback reason truncation UTF-16 safe #102631

Merged

wangmiao0668000666 mentioned this pull request Jul 9, 2026

fix(chat,cli): keep session snippet and CLI output digest truncation UTF-16 safe #102747

Closed

9 tasks

steipete mentioned this pull request Jul 11, 2026

fix(release/2026.7.1): backport tool-result compaction fix stack (#99495/#102610 + #99756 + UTF-16 safety) #104702

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(agents): keep tool-result truncation UTF-16 safe#102087

fix(agents): keep tool-result truncation UTF-16 safe#102087
steipete merged 2 commits into
openclaw:mainfrom
chengzhichao-xydt:fix/tool-result-truncation-utf16

chengzhichao-xydt commented Jul 8, 2026

Uh oh!

clawsweeper Bot commented Jul 8, 2026 •

edited

Loading

Uh oh!

chengzhichao-xydt commented Jul 8, 2026

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

chengzhichao-xydt commented Jul 8, 2026

Summary

Linked context

Real behavior proof (required for external PRs)

Tests and validation

Risk checklist

Current review state

Uh oh!

clawsweeper Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chengzhichao-xydt commented Jul 8, 2026

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jul 8, 2026 •

edited

Loading