fix(agents): keep structured prompt summaries UTF-16 safe by Alix-007 · Pull Request #101311 · openclaw/openclaw

Alix-007 · 2026-07-07T03:11:10Z

Summary

AI-assisted with Codex; I inspected the changed production path and can explain the change.
Embedded agent runner structured media/JSON prompt summaries preserve UTF-16 boundaries when summarizing long structured values.
Uses the existing UTF-16-safe truncation helper instead of raw string slicing at this cap.
Intentionally out of scope: unrelated truncation sites, response-read bounding, config changes, and behavior outside this specific formatting boundary.

Linked context

No linked issue.
Related to the existing OpenClaw UTF-16-safe truncation hardening pattern.

Real behavior proof (required for external PRs)

Behavior addressed: Embedded agent runner structured media/JSON prompt summaries preserve UTF-16 boundaries when summarizing long structured values.
Real environment tested: local OpenClaw source checkout on Node v22.22.0; PR head b35de52.
Exact steps or command run after this patch: node scripts/run-vitest.mjs src/agents/embedded-agent-runner/run/attempt.prompt-helpers.test.ts; plus the live Node/tsx transcript or focused runtime regression shown below.
Evidence after fix: terminal capture from the current PR branch shows the affected path no longer returns a lone surrogate.

$ node --import tsx -e '<drive mergeOrphanedTrailingUserPrompt on image_url/json keys ending at emoji boundary>'
{"merged":true,"promptLength":1429,"hasLoneSurrogate":false,"containsImageUrl":true,"containsCharsMarker":true}

$ node scripts/run-vitest.mjs src/agents/embedded-agent-runner/run/attempt.prompt-helpers.test.ts
Test Files 1 passed
Tests 15 passed

Observed result after fix: the affected agents truncation path keeps output well-formed at the emoji/surrogate-pair boundary and preserves the existing truncation marker/cap behavior.
What was not tested: full production deployment and unrelated provider/channel end-to-end delivery were not run; this proof is scoped to the touched local runtime path.
Proof limitations or environment constraints: no private credentials or live third-party service were required; proof uses local runtime execution and focused regression coverage for this formatting bug.
Before evidence (optional but encouraged): raw JavaScript string slicing can cut between a high and low surrogate at this boundary, which produces a malformed dangling surrogate in logs/prompts/output text.

Tests and validation

node scripts/run-vitest.mjs src/agents/embedded-agent-runner/run/attempt.prompt-helpers.test.ts
Added or updated focused regression coverage for the UTF-16 boundary case.
No known local failures from this change.

Risk checklist

Did user-visible behavior change? (Yes/No)

Yes. Malformed truncated text is now avoided while preserving existing caps and truncation markers.

Did config, environment, or migration behavior change? (Yes/No)

No.

Did security, auth, secrets, network, or tool execution behavior change? (Yes/No)

No.

What is the highest-risk area?

embedded agent prompt summary formatting.

How is that risk mitigated?

The patch is limited to the existing truncation boundary and is covered by focused regression proof above.

Current review state

What is the next action?

ClawSweeper re-review and maintainer review after this proof/body refresh.

What is still waiting on author, maintainer, CI, or external proof?

Nothing is waiting on the author after this proof update; waiting on CI/ClawSweeper/maintainer review.

Which bot or reviewer comments were addressed?

Addressed ClawSweeper's needs-proof feedback by using exact Real behavior proof field labels and adding copied terminal output from the current PR head.

clawsweeper · 2026-07-07T03:53:33Z

Codex review: needs maintainer review before merge. Reviewed July 7, 2026, 12:53 AM ET / 04:53 UTC.

Summary
The branch changes embedded-agent structured media/JSON prompt summary truncation to use truncateUtf16Safe and adds a surrogate-boundary regression test.

PR surface: Source +1, Tests +44. Total +45 across 2 files.

Reproducibility: yes. Source inspection shows current main uses raw .slice at the affected UTF-16 code-unit caps, and the PR body provides terminal proof from the PR head showing the boundary path no longer returns a lone surrogate.

Review metrics: none identified.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge

No ClawSweeper repair is needed; the PR already contains the narrow fix and sufficient proof, so the remaining action is maintainer review and merge handling.

Security
Cleared: The diff only changes TypeScript prompt-summary truncation and a focused test; it adds no dependency, workflow, secret, permission, network, or code-execution surface.

Review details

Best possible solution:

Land the narrow helper-based fix after normal maintainer review and exact-head merge gates, keeping the current caps and prompt marker semantics unchanged.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main uses raw .slice at the affected UTF-16 code-unit caps, and the PR body provides terminal proof from the PR head showing the boundary path no longer returns a lone surrogate.

Is this the best way to solve the issue?

Yes. Reusing the existing truncateUtf16Safe helper at the three cap points is the narrowest maintainable fix; a custom local sanitizer or broader prompt refactor would add unnecessary surface.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a596ca1795b6.

Label changes

Label changes:

add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

P2: The PR fixes a narrow agent prompt-formatting bug with limited blast radius and no config, migration, auth, or security surface.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The refreshed PR body includes copied terminal output from a PR-head runtime probe showing hasLoneSurrogate:false plus focused Vitest output, which is sufficient for this narrow non-visual helper fix.
proof: sufficient: Contributor real behavior proof is sufficient. The refreshed PR body includes copied terminal output from a PR-head runtime probe showing hasLoneSurrogate:false plus focused Vitest output, which is sufficient for this narrow non-visual helper fix.

Evidence reviewed

PR surface:

Source +1, Tests +44. Total +45 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	4	3	+1
Tests	1	44	0	+44
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	48	3	+45

What I checked:

Repository policy read: Read the root and scoped agent-runner AGENTS.md files; their review guidance drove the broader code-path, helper-test, and no-local-test review shape. (AGENTS.md:1, a596ca1795b6)
Live PR state: Live GitHub metadata shows the PR is open, non-draft, mergeable but behind, with one commit at b35de52, two changed files, and labels including proof: sufficient and status: 👀 ready for maintainer look. (b35de5205580)
Current main behavior: Current main still slices structured media refs, structured JSON strings, and JSON fallback text with raw .slice(...) at the affected prompt-summary caps. (src/agents/embedded-agent-runner/run/attempt.prompt-helpers.ts:340, a596ca1795b6)
PR fix shape: The PR head imports truncateUtf16Safe and uses it at the three changed prompt-summary truncation sites without changing caps, suffixes, or reported source lengths. (src/agents/embedded-agent-runner/run/attempt.prompt-helpers.ts:341, b35de5205580)
Safe helper contract: truncateUtf16Safe delegates to sliceUtf16Safe, which adjusts slice boundaries away from surrogate halves before returning the slice. (packages/normalization-core/src/utf16-slice.ts:44, a596ca1795b6)
Runtime entry point: Embedded attempts call the message-merge strategy for trailing user leaf messages before provider submission, so structured orphan content flows through mergeOrphanedTrailingUserPrompt. (src/agents/embedded-agent-runner/run/attempt.ts:4231, a596ca1795b6)

Likely related people:

vincentkoc: Current-main blame for the structured prompt-summary helper block points to Vincent Koc, and live PR metadata shows vincentkoc authored/merged the adjacent UTF-16 bounded-output consolidation in fix(text): keep bounded outputs UTF-16 safe #101355. (role: recent area contributor and adjacent merger; confidence: high; commits: 084cf2a43ddf, 84e5327720b4; files: src/agents/embedded-agent-runner/run/attempt.prompt-helpers.ts, src/agents/tools/web-fetch-utils.ts, packages/speech-core/src/tts.ts)
Alix-007: Beyond authoring this PR, Alix-007 is listed as co-author on the merged adjacent UTF-16 bounded-output hardening commit, so they have current-main history in the same safe-truncation cleanup area. (role: recent adjacent contributor; confidence: medium; commits: 84e5327720b4, b35de5205580; files: src/agents/tools/web-fetch-utils.ts, packages/speech-core/src/tts.ts, src/agents/embedded-agent-runner/run/attempt.prompt-helpers.ts)
steipete: The earliest visible safe UTF-16 truncation history in this checkout includes commit 63f5fa4 adding invalid-UTF-16 avoidance through shared utility code. (role: original helper-pattern contributor; confidence: medium; commits: 63f5fa47deb6; files: src/utils.ts, src/agents/bash-tools.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (2 earlier review cycles)

reviewed 2026-07-07T03:57:36.272Z sha 4ecc7b0 :: needs real behavior proof before merge. :: none
reviewed 2026-07-07T04:46:42.404Z sha b35de52 :: needs maintainer review before merge. :: none

steipete · 2026-07-07T06:31:51Z

Land-ready proof

Reviewed the full structured orphan-message summary pipeline: media references, nested JSON strings, depth/key/array bounds, data-URI redaction, final serialization, and prompt merge ownership. No unresolved findings.
Best fix: reuse truncateUtf16Safe from packages/normalization-core/src/utf16-slice.ts at the three existing character caps. Limits, ellipses, length annotations, and raw-text behavior remain unchanged; no new truncation abstraction or compatibility path.
Sanitized AWS Crabbox cbx_5af78ef0a87e, run run_e8daafefe7cf: original exact-head focused suite passed 15/15.
Blacksmith Testbox tbx_01kwxhf00awefzyke5p0phz686: hardened focused suite passed 15/15, including explicit media, nested-string, and final-JSON surrogate-boundary coverage.
Fresh branch autoreview: no accepted/actionable findings (0.90 confidence).
Exact prepared SHA f111b51a419d8ce6ce92b31d6f1ef9ad8b6d3d1f; CI run 28845979195 passed. OPENCLAW_TESTBOX=1 scripts/pr prepare-run 101311 passed the hosted exact-head gate and verified the remote tree.

Release-note context: this prevents emoji at structured prompt-summary limits from becoming malformed prompt content. The native normal-PR gate keeps root changelog edits release-owned.

steipete · 2026-07-07T06:32:09Z

Merged via squash.

Prepared head SHA: f111b51a419d8ce6ce92b31d6f1ef9ad8b6d3d1f
Landed commit: 3929c52069fb0b4d1f8d13f229c26aa9ad2853f5

…01311) * fix(agents): keep structured prompt summaries UTF-16 safe * chore: align prompt summary branch with current main * fix(agents): keep structured summaries UTF-16 safe * chore: keep release notes in PR context * chore: keep release notes in PR context --------- Co-authored-by: Peter Steinberger <[email protected]> Co-authored-by: Peter Steinberger <[email protected]>

openclaw-barnacle Bot added agents Agent runtime and tooling size: S labels Jul 7, 2026

Alix-007 force-pushed the alix/utf16-attempt-prompt-helpers branch 2 times, most recently from 16a097f to 00ecba7 Compare July 7, 2026 04:23

fix(agents): keep structured prompt summaries UTF-16 safe

b35de52

Alix-007 force-pushed the alix/utf16-attempt-prompt-helpers branch from 00ecba7 to b35de52 Compare July 7, 2026 04:40

steipete self-assigned this Jul 7, 2026

steipete and others added 2 commits July 7, 2026 07:18

chore: align prompt summary branch with current main

51bf4d6

Merge branch 'main' into alix/utf16-attempt-prompt-helpers

28ad6f1

openclaw-barnacle Bot added size: XS and removed agents Agent runtime and tooling size: S labels Jul 7, 2026

fix(agents): keep structured summaries UTF-16 safe

9e56245

openclaw-barnacle Bot added agents Agent runtime and tooling size: S and removed size: XS labels Jul 7, 2026

steipete added 2 commits July 7, 2026 07:20

chore: keep release notes in PR context

af60bfc

chore: keep release notes in PR context

f111b51

steipete merged commit 3929c52 into openclaw:main Jul 7, 2026
101 checks passed

github-actions Bot mentioned this pull request Jul 7, 2026

📡 Upstream Digest — 2026-07-07 07:31 UTC curtismercier/openclaw-mods#1228

Open

clawsweeper Bot mentioned this pull request Jul 7, 2026

fix(session-cost-usage): keep emoji / surrogate pairs intact during content truncation #101517

Merged

wangmiao0668000666 mentioned this pull request Jul 9, 2026

fix(matrix): keep HTTP error and tool-progress truncations UTF-16 safe #102395

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(agents): keep structured prompt summaries UTF-16 safe#101311

fix(agents): keep structured prompt summaries UTF-16 safe#101311
steipete merged 6 commits into
openclaw:mainfrom
Alix-007:alix/utf16-attempt-prompt-helpers

Alix-007 commented Jul 7, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

Alix-007 commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Linked context

Real behavior proof (required for external PRs)

Tests and validation

Risk checklist

Current review state

Uh oh!

clawsweeper Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Alix-007 commented Jul 7, 2026 •

edited

Loading

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading