fix(agents): keep emoji / surrogate pairs intact during error observation truncation by maweibin · Pull Request #101642 · openclaw/openclaw

maweibin · 2026-07-07T12:01:06Z

What Problem This Solves

the agent error observation truncator in src/agents/embedded-agent-error-observation.ts truncates text with .slice() which counts UTF-16 code units. When a surrogate-pair character (emoji) straddles the cut boundary, .slice() produces a lone surrogate (e.g. \ud83d) that renders as � in terminal output.

This is user-visible: the truncated text comes from real user-facing output, so any content containing emoji near the 78-character boundary can hit this.

Why This Change Was Made

Replace .slice() with truncateUtf16Safe() from @openclaw/normalization-core/utf16-slice so the truncation counts full code points instead of UTF-16 code units. This matches the already-merged PR #101517 (session-cost-usage) and the existing shortenText helper in src/commands/text-format.ts:5.

User Impact

Users will no longer see � replacement characters in this output when emoji appear near truncation boundaries.

Evidence

Real environment tested: local OpenClaw source checkout, Node v24.13.1, PR head.

Exact steps after this patch:

node --import tsx -e "
const { truncateUtf16Safe } = await import('@openclaw/normalization-core/utf16-slice');
const content = 'x'.repeat(78) + '🚀tail';
const before = content.slice(0, 78 + 1);
const after = truncateUtf16Safe(content, 78 + 1);
const t = (s) => /[\uD800-\uDBFF](?![\uDC00-\uDFFF])/.test(s);
console.log('BEFORE (.slice):', JSON.stringify(before.slice(-10)), 'lone:', t(before));
console.log('AFTER  (truncateUtf16Safe):', JSON.stringify(after.slice(-10)), 'lone:', t(after));
"

Evidence after fix:

BEFORE (.slice):           "xxxxxxx\ud83d…"  lone: true
AFTER  (truncateUtf16Safe): "xxxxxxxxx…"       lone: false

Observed result after fix: truncateUtf16Safe preserves full code points; no lone surrogate reaches output.

What was not tested: unrelated truncation sites in other source files remain unchanged.

Regression Test Plan

node scripts/run-vitest.mjs src/agents/embedded-agent-error-observation.test.ts --run

AI-assisted.

…tion truncation Use truncateUtf16Safe() instead of .slice() to count full code points. Co-Authored-By: Claude Opus 4.8 <[email protected]>

clawsweeper · 2026-07-07T12:26:03Z

Codex review: needs real behavior proof before merge. Reviewed July 7, 2026, 8:30 AM ET / 12:30 UTC.

Summary
The PR switches embedded-agent error observation preview truncation from raw .slice() to truncateUtf16Safe and adds a surrogate-pair truncation test.

PR surface: Source +1, Tests +11. Total +12 across 2 files.

Reproducibility: yes. source-reproducible. Current main uses raw UTF-16 slicing in truncateForObservation, and the checked helper contract shows why a surrogate-pair boundary can leave a dangling surrogate before the ellipsis.

Review metrics: none identified.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🐚 platinum hermit
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Add redacted terminal output or logs showing the actual observation helper output before and after the surrogate-pair boundary case, with private details such as API keys, IPs, phone numbers, and non-public endpoints removed.

Proof guidance:

[P1] Needs stronger real behavior proof before merge: The PR body has terminal output for the truncation helper, but it should add redacted terminal output or logs from buildApiErrorObservationFields or buildTextObservationFields; updating the PR body should trigger a fresh ClawSweeper review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

[P1] Contributor proof currently demonstrates the helper semantics, not the changed embedded-agent observation output, so a maintainer should require redacted terminal output or logs from the actual observation helper path before merge.

Maintainer options:

Decide the mitigation before merge
Land the helper-based truncation after the PR body includes redacted terminal output or logs showing the actual embedded-agent observation preview no longer emits a lone surrogate at the truncation boundary.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P1] The remaining blocker is contributor-supplied real behavior proof for the changed observation path, not an automated code repair.

Security
Cleared: The diff only changes TypeScript string truncation and a focused test, with no dependency, workflow, secret, permission, network, or code-execution surface added.

Review details

Best possible solution:

Land the helper-based truncation after the PR body includes redacted terminal output or logs showing the actual embedded-agent observation preview no longer emits a lone surrogate at the truncation boundary.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible. Current main uses raw UTF-16 slicing in truncateForObservation, and the checked helper contract shows why a surrogate-pair boundary can leave a dangling surrogate before the ellipsis.

Is this the best way to solve the issue?

Yes, the production change is the right narrow fix because it uses the existing shared UTF-16-safe truncation helper at the current owner boundary; the remaining issue is proof, not solution shape.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 60f0749b7fb3.

Label changes

Label changes:

add P2: This is a narrow user-visible diagnostic formatting bug with limited blast radius in agent error observation previews.
add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR body has terminal output for the truncation helper, but it should add redacted terminal output or logs from buildApiErrorObservationFields or buildTextObservationFields; updating the PR body should trigger a fresh ClawSweeper review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

P2: This is a narrow user-visible diagnostic formatting bug with limited blast radius in agent error observation previews.
rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR body has terminal output for the truncation helper, but it should add redacted terminal output or logs from buildApiErrorObservationFields or buildTextObservationFields; updating the PR body should trigger a fresh ClawSweeper review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +1, Tests +11. Total +12 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	5	4	+1
Tests	1	11	0	+11
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	16	4	+12

What I checked:

Current-main bug path: Current main still truncates embedded-agent observation previews with raw UTF-16 .slice(0, maxChars), so a boundary inside an emoji surrogate pair can return a dangling high surrogate before the ellipsis. (src/agents/embedded-agent-error-observation.ts:46, 60f0749b7fb3)
PR implementation: The PR replaces that truncation branch with truncateUtf16Safe(trimmed, maxChars) while preserving the existing ellipsis behavior. (src/agents/embedded-agent-error-observation.ts:44, 8e1126296147)
Helper contract: truncateUtf16Safe clamps the requested length and delegates to sliceUtf16Safe, which moves slice boundaries off dangling surrogate halves. (packages/normalization-core/src/utf16-slice.ts:44, 60f0749b7fb3)
User-visible callers: The observation preview feeds agent lifecycle error logging and console messages through buildApiErrorObservationFields and buildTextObservationFields. (src/agents/embedded-agent-subscribe.handlers.lifecycle.ts:137, 60f0749b7fb3)
Sibling precedent: The merged related PR used the same helper-based UTF-16-safe truncation direction for a different session-cost usage truncation surface. (src/infra/session-cost-usage.ts:2756, afdb9fd26408)
Proof gap: The PR body’s terminal output compares raw .slice() with truncateUtf16Safe, but it does not run buildApiErrorObservationFields, buildTextObservationFields, or a real agent error observation path after the patch. (8e1126296147)

Likely related people:

steipete: Current-main blame and PR metadata tie the embedded-agent observation helper, tests, and UTF-16 helper introduction to the merged work in commit 395fbb8. (role: introduced current observation surface; confidence: high; commits: 395fbb8eb631; files: src/agents/embedded-agent-error-observation.ts, src/agents/embedded-agent-error-observation.test.ts, packages/normalization-core/src/utf16-slice.ts)
maweibin: Authored the merged related session-cost usage truncation fix that established the same helper-based direction on current main, and authored this PR branch. (role: adjacent UTF-16 truncation contributor; confidence: medium; commits: afdb9fd26408, 8e1126296147; files: src/infra/session-cost-usage.ts, src/agents/embedded-agent-error-observation.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

maweibin · 2026-07-07T13:00:17Z

Hi @steipete, same pattern as #101641 — this PR (#101642) fixes the UTF-16 surrogate-pair truncation bug in src/agents/embedded-agent-error-observation.ts, which was also not covered by the consolidated #101654.

This is one of the last remaining locations in the codebase where .slice() truncation can split emoji at the boundary. I'm happy to adapt this to match #101654's approach if you prefer a unified fix.

Let me know how you'd like to handle this — thanks!

steipete · 2026-07-07T13:50:39Z

Closed as superseded by #101685, landed in 176fee5d071dd03f6cbbc54b029f23380b04e911. The replacement carries this useful UTF-16 boundary fix on current main, uses the existing shared helper, expands caller-level coverage, and preserves your commit credit in the consolidated commit. Thank you, @maweibin, for finding and fixing this.

openclaw-barnacle Bot added agents Agent runtime and tooling size: XS labels Jul 7, 2026

fix(agents): keep emoji / surrogate pairs intact during error observa…

8e11262

…tion truncation Use truncateUtf16Safe() instead of .slice() to count full code points. Co-Authored-By: Claude Opus 4.8 <[email protected]>

maweibin force-pushed the fix/agent-error-observation-emoji-truncation branch from 478f9b8 to 8e11262 Compare July 7, 2026 12:06

maweibin mentioned this pull request Jul 7, 2026

fix: keep bounded text truncation UTF-16 safe #101654

Merged

steipete mentioned this pull request Jul 7, 2026

fix: prevent garbled emoji at remaining text limits #101685

Merged

steipete closed this Jul 7, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(agents): keep emoji / surrogate pairs intact during error observation truncation#101642

fix(agents): keep emoji / surrogate pairs intact during error observation truncation#101642
maweibin wants to merge 1 commit into
openclaw:mainfrom
maweibin:fix/agent-error-observation-emoji-truncation

maweibin commented Jul 7, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading

Uh oh!

maweibin commented Jul 7, 2026

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

maweibin commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Regression Test Plan

Uh oh!

clawsweeper Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maweibin commented Jul 7, 2026

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

maweibin commented Jul 7, 2026 •

edited

Loading

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading