fix(commitments): keep emoji / surrogate pairs intact during terminal output truncation by maweibin · Pull Request #101605 · openclaw/openclaw

maweibin · 2026-07-07T11:33:44Z

What Problem This Solves

The commitment list formatter in src/commands/commitments.ts truncates table cells with .slice() which counts UTF-16 code units. When a surrogate-pair character (emoji) straddles the cut boundary, .slice() produces a lone surrogate (e.g. \ud83d) that renders as � in openclaw commitments CLI output.

This is user-visible: commitment descriptions are user-provided text that can contain emoji near the column width boundary of 16 characters (id) or 90 characters (suggested text).

Why This Change Was Made

Replace .slice() with truncateUtf16Safe() from @openclaw/normalization-core/utf16-slice so truncation counts full code points instead of UTF-16 code units. This matches the already-merged PR #101517 (session-cost-usage).

User Impact

openclaw commitments table cells no longer show � when commitment descriptions contain emoji near column boundaries.

Evidence

Real environment tested: local OpenClaw source checkout, Node v24.13.1, PR head.

Exact steps after this patch:

node --import tsx -e "
const { truncateUtf16Safe } = await import('@openclaw/normalization-core/utf16-slice');
const content = 'y'.repeat(78) + '\u{1F680}xx';
const before = content.slice(0, 79) + '…';
const after = truncateUtf16Safe(content, 79) + '…';
const t = (s) => /[\uD800-\uDBFF](?![\uDC00-\uDFFF])/.test(s);
console.log('BEFORE (.slice 79):', JSON.stringify(before.slice(-12)), 'lone:', t(before));
console.log('AFTER  (truncateUtf16Safe 79):', JSON.stringify(after.slice(-12)), 'lone:', t(after));
"

Evidence after fix:

BEFORE (.slice 79):           "yyyyyyyyyy\ud83d…"  lone: true
AFTER  (truncateUtf16Safe 79): "yyyyyyyyyyy…"       lone: false

Observed result after fix: truncateUtf16Safe preserves full code points; no lone surrogate reaches terminal output. Cell width caps are unchanged — the broken surrogate is dropped cleanly rather than rendered as �.

What was not tested: full openclaw commitments end-to-end run on exact PR head (production CLI uses installed binary).

Regression Test Plan

node scripts/run-vitest.mjs src/commands/commitments.test.ts --run
  Test Files  1 passed (1)
       Tests  8 passed (8)

AI-assisted.

clawsweeper · 2026-07-07T11:47:45Z

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Close as superseded: the open maintainer-labeled UTF-16 sweep already carries this exact commitments helper change, adds caller-level regression coverage, and has broader Testbox proof, so this branch should not remain a separate landing path.

Root-cause cluster
Relationship: superseded
Canonical: #101654
Summary: The current PR is a narrow instance of the bounded UTF-16 truncation bug that is now owned by the consolidated sweep.

Members:

canonical: fix: keep bounded text truncation UTF-16 safe #101654 - Open maintainer-labeled PR that explicitly consolidates this PR and includes the same commitments code change plus regression coverage.
partial_overlap: fix(cli): keep task/flow/commitment/audit table text truncation UTF-16 safe #101622 - Open broader CLI truncation PR that overlaps the commitments fix but is itself superseded by the consolidated sweep.
same_root_cause: fix(session-cost-usage): keep emoji / surrogate pairs intact during content truncation #101517 - Merged related UTF-16 truncation fix for another output surface; it establishes the shared helper direction but does not itself fix commitments output.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Canonical path: Use #101654 as the canonical landing path for this UTF-16 truncation fix, and only revive this narrow branch if the consolidated sweep does not land.

So I’m closing this here and keeping the remaining discussion on #101654.

Review details

Best possible solution:

Use #101654 as the canonical landing path for this UTF-16 truncation fix, and only revive this narrow branch if the consolidated sweep does not land.

Do we have a high-confidence way to reproduce the issue?

Yes from source inspection: current main truncates commitment table cells with raw UTF-16 slicing before appending an ellipsis, so a surrogate pair at the cutoff can leave a dangling high surrogate. I did not run the full CLI path locally during this read-only review.

Is this the best way to solve the issue?

No for keeping this branch as the best landing path. The updated code is correct for the narrow commitments helper, but the consolidated PR is the better solution because it covers this exact change, sibling truncation surfaces, regression tests, and broader proof together.

Security review:

Security review cleared: The diff only changes local TypeScript string formatting and imports an existing internal helper; it does not touch dependencies, workflows, secrets, permissions, package metadata, or code-execution paths.

AGENTS.md: found and applied where relevant.

What I checked:

Current main still has the raw-slice bug: src/commands/commitments.ts on current main truncates table fields with value.slice(0, maxChars - 1) before appending an ellipsis, so a surrogate pair crossing that boundary can leave a dangling surrogate. (src/commands/commitments.ts:26, 60f0749b7fb3)
This PR uses the right narrow helper: The current PR head changes the commitments helper to call truncateUtf16Safe(value, maxChars - 1), fixing the previous helper-choice review finding. (src/commands/commitments.ts:25, 5d4f0ab91fee)
Helper contract preserves UTF-16 caps safely: truncateUtf16Safe delegates to sliceUtf16Safe and backs off split surrogate boundaries while preserving a UTF-16 code-unit limit. (packages/normalization-core/src/utf16-slice.ts:43, 60f0749b7fb3)
Canonical PR contains the same commitments code change: The consolidated sweep at fix: keep bounded text truncation UTF-16 safe #101654 includes the same src/commands/commitments.ts change against the current base. (src/commands/commitments.ts:25, 15a1df3f7e35)
Canonical PR adds caller-level regression coverage: The consolidated sweep adds a commitments command test that exercises an emoji-boundary truncated table cell and asserts no dangling high surrogate remains. (src/commands/commitments.test.ts:204, 15a1df3f7e35)
Canonical PR is a viable replacement path: Live PR metadata shows fix: keep bounded text truncation UTF-16 safe #101654 is open, mergeable, maintainer-labeled, credits this contributor, explicitly says it consolidates this PR, and reports Blacksmith Testbox plus pnpm check:changed proof in the PR body. (15a1df3f7e35)

Likely related people:

steipete: Current blame points the commitments command and shared UTF-16 helper to Peter Steinberger's commit, and he authored the open consolidated sweep that supersedes this branch. (role: feature introducer and canonical follow-up owner; confidence: high; commits: 395fbb8eb631, 080eb95c554c; files: src/commands/commitments.ts, src/commands/commitments.test.ts, packages/normalization-core/src/utf16-slice.ts)
vincentkoc: Vincent Koc is co-author on the commit that introduced the commitments command on current main and authored the current main tip in adjacent UI/command work. (role: adjacent current-main contributor; confidence: medium; commits: 395fbb8eb631, 60f0749b7fb3; files: src/commands/commitments.ts, src/commands/commitments.test.ts)
maweibin: maweibin authored the merged session-cost UTF-16 truncation fix and is credited as a co-author in the consolidated UTF-16 sweep, so they are relevant to this specific truncation behavior beyond opening this PR. (role: recent related UTF-16 truncation contributor; confidence: medium; commits: afdb9fd26408, 080eb95c554c; files: src/infra/session-cost-usage.ts, src/infra/session-cost-usage.test.ts, src/commands/commitments.ts)

Codex review notes: model internal, reasoning high; reviewed against 60f0749b7fb3.

maweibin · 2026-07-07T12:04:41Z

@clawsweeper re-review

clawsweeper · 2026-07-07T12:04:44Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

… output truncation .use Array.from() to count full code points instead of UTF-16 code units, preventing lone surrogates in commitment list output. Co-Authored-By: Claude Opus 4.8 <[email protected]>

steipete · 2026-07-07T12:46:20Z

Thanks @maweibin. The useful behavior from this PR was consolidated with the sibling UTF-16 boundary fixes into #101654 and landed on main in a9582a1bb62aa70bb0b0ceb72536d0d76f08eab8.

The landed fix uses the existing truncateUtf16Safe helper, preserves this caller's prior code-unit limit and output shape, and adds caller-level surrogate-boundary coverage. Closing this PR as superseded by the canonical landed change.

openclaw-barnacle Bot added commands Command implementations size: XS labels Jul 7, 2026

fix(commitments): keep emoji / surrogate pairs intact during terminal…

5d4f0ab

… output truncation .use Array.from() to count full code points instead of UTF-16 code units, preventing lone surrogates in commitment list output. Co-Authored-By: Claude Opus 4.8 <[email protected]>

maweibin force-pushed the fix/commitments-emoji-truncation branch from 1aedead to 5d4f0ab Compare July 7, 2026 12:16

maweibin closed this Jul 7, 2026

maweibin reopened this Jul 7, 2026

steipete mentioned this pull request Jul 7, 2026

fix: keep bounded text truncation UTF-16 safe #101654

Merged

clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jul 7, 2026

clawsweeper Bot mentioned this pull request Jul 7, 2026

fix(cli): keep task/flow/commitment/audit table text truncation UTF-16 safe #101622

Closed

steipete closed this Jul 7, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(commitments): keep emoji / surrogate pairs intact during terminal output truncation#101605

fix(commitments): keep emoji / surrogate pairs intact during terminal output truncation#101605
maweibin wants to merge 1 commit into
openclaw:mainfrom
maweibin:fix/commitments-emoji-truncation

maweibin commented Jul 7, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading

Uh oh!

maweibin commented Jul 7, 2026

Uh oh!

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

maweibin commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Regression Test Plan

Uh oh!

clawsweeper Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maweibin commented Jul 7, 2026

Uh oh!

clawsweeper Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

maweibin commented Jul 7, 2026 •

edited

Loading

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading

clawsweeper Bot commented Jul 7, 2026 •

edited

Loading