fix(crestodian): keep prompt history UTF-16 safe by wangyan2026 · Pull Request #102627 · openclaw/openclaw

wangyan2026 · 2026-07-09T09:36:08Z

What Problem This Solves

Crestodian bounded each prior conversation turn with raw UTF-16 code-unit slicing before placing it in the planner prompt. A boundary through an emoji or another supplementary character could leave a lone surrogate in model-facing history.

Why This Change Was Made

Oversized turns now use the shared truncateUtf16Safe primitive before the existing ellipsis. During review, the regression was consolidated into the established assistant suite, replacing a separate 73-line test module with one exact prompt assertion.

User Impact

Long Crestodian conversation turns remain valid Unicode in subsequent planner prompts. The 500-unit threshold, 12-turn history window, labels, and short-turn behavior are unchanged.

Evidence

Focused Crestodian assistant suite passed.
Exact regression assertion places an emoji across the 500-unit boundary and verifies the complete rendered history prefix.
Focused oxfmt and oxlint passed; git diff --check passed.
Fresh branch autoreview found no accepted or actionable findings (0.99 confidence).

clawsweeper · 2026-07-09T09:46:46Z

Codex review: needs real behavior proof before merge. Reviewed July 9, 2026, 9:45 AM ET / 13:45 UTC.

Summary
The PR replaces Crestodian history truncation with the shared UTF-16-safe truncator and adds a focused assistant prompt regression assertion.

PR surface: Source +1, Tests +13. Total +14 across 2 files.

Reproducibility: yes. from source inspection: current main slices long Crestodian history at 500 UTF-16 code units, so 499 BMP characters followed by an emoji leaves a lone high surrogate. I did not run a live Crestodian session in this read-only review.

Review metrics: none identified.

Root-cause cluster
Relationship: canonical
Canonical: #102627
Summary: This PR is the focused open Crestodian UTF-16 history fix; the similar cross-owner PR was closed unmerged and should not supersede it.

Members:

superseded: fix(crestodian): use truncateUtf16Safe for conversation text truncation #102770 - It overlapped the Crestodian truncation change but was closed unmerged after maintainer feedback about bundled unrelated sites and insufficient focused tests.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🦞 diamond lobster
Result: blocked until real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Add redacted Crestodian prompt or run output showing a long emoji-containing history turn truncates without a dangling surrogate.
Update the PR body after adding proof so ClawSweeper re-reviews automatically, or ask a maintainer to comment @clawsweeper re-review if it does not.

Proof guidance:

[P1] Needs real behavior proof before merge: Missing: the PR body and comments only show tests/lint/autoreview evidence; the contributor should add redacted Crestodian prompt/run output or logs, then update the PR body to trigger re-review or ask a maintainer for @clawsweeper re-review.

Risk before merge

[P1] The PR still lacks redacted real Crestodian prompt/run proof, so merge readiness depends on source inspection, tests, and CI rather than a user-like run of the changed prompt path.

Maintainer options:

Decide the mitigation before merge
Land the localized helper-based fix after redacted real Crestodian prompt or run proof confirms long emoji-containing history stays valid Unicode.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P1] No repair lane is appropriate because there are no actionable code findings; only contributor-supplied real behavior proof remains.

Security
Cleared: The diff imports an existing workspace helper and adds a regression test; it does not touch dependencies, lockfiles, scripts, CI, secrets, permissions, or package resolution.

Review details

Best possible solution:

Land the localized helper-based fix after redacted real Crestodian prompt or run proof confirms long emoji-containing history stays valid Unicode.

Do we have a high-confidence way to reproduce the issue?

Yes, from source inspection: current main slices long Crestodian history at 500 UTF-16 code units, so 499 BMP characters followed by an emoji leaves a lone high surrogate. I did not run a live Crestodian session in this read-only review.

Is this the best way to solve the issue?

Yes for the code path: using the existing shared UTF-16-safe truncator at the single history cap is the narrow maintainable fix. The remaining blocker is proof from a real Crestodian prompt/run, not a different implementation.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f6b990124353.

Label changes

Label justifications:

P2: This is a focused model-facing prompt text corruption fix with limited blast radius and no config, storage, protocol, auth, or delivery change.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦞 diamond lobster.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: Missing: the PR body and comments only show tests/lint/autoreview evidence; the contributor should add redacted Crestodian prompt/run output or logs, then update the PR body to trigger re-review or ask a maintainer for @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +1, Tests +13. Total +14 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	2	1	+1
Tests	1	13	0	+13
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	15	1	+14

What I checked:

Root policy read: The full root AGENTS.md was read; its OpenClaw PR review and real-behavior-proof guidance affected the keep-open verdict. (AGENTS.md:13, f6b990124353)
Current main bug path: Current main still uses raw turn.text.slice(0, HISTORY_TURN_MAX_CHARS) for long Crestodian history turns, so a 499-BMP-character prefix followed by an emoji can cut a surrogate pair at the 500-code-unit boundary. (src/crestodian/assistant-prompts.ts:119, f6b990124353)
Runtime caller path: Both configured-model and local-runtime planners pass prior history through buildCrestodianAssistantUserPrompt, making this model-facing prompt construction path active. (src/crestodian/assistant.ts:118, f6b990124353)
Shared helper contract: truncateUtf16Safe floors the limit and delegates to sliceUtf16Safe, whose edge checks avoid returning dangling surrogate halves. (packages/normalization-core/src/utf16-slice.ts:44, f6b990124353)
PR implementation: The latest PR head imports truncateUtf16Safe and uses it at the single Crestodian history truncation site while preserving the existing 500-unit limit and ellipsis behavior. (src/crestodian/assistant-prompts.ts:112, 065b708bdb6b)
Regression coverage: The latest PR head asserts the rendered prompt prefix for a 499-character-plus-emoji boundary and expects the emoji to be dropped cleanly before the ellipsis. (src/crestodian/assistant.test.ts:124, 065b708bdb6b)

Likely related people:

obviyus: Git blame and GitHub commit metadata tie the current Crestodian history formatter and shared UTF-16 helper to commit 34d2577. (role: introduced current behavior; confidence: high; commits: 34d257713ea3; files: src/crestodian/assistant-prompts.ts, packages/normalization-core/src/utf16-slice.ts, packages/normalization-core/src/utf16-slice.test.ts)
steipete: Recent current-main onboarding work changed the Crestodian prompt surface in PR feat(onboard): guided CLI onboarding with live AI verification and classic fallback #101880, and the live PR timeline shows title/test-shape follow-up on this branch. (role: recent area contributor and reviewer; confidence: high; commits: e2a112a5565e, f11f5db907e4, 065b708bdb6b; files: src/crestodian/assistant-prompts.ts, src/crestodian/assistant.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (3 earlier review cycles)

reviewed 2026-07-09T09:50:01.067Z sha d79eac9 :: needs real behavior proof before merge. :: [P3] Assert the actual dangling surrogate condition
reviewed 2026-07-09T12:08:39.799Z sha 9f6619c :: needs real behavior proof before merge. :: none
reviewed 2026-07-09T12:32:44.032Z sha 9f6619c :: needs real behavior proof before merge. :: none

wangyan2026 · 2026-07-09T12:07:36Z

@clawsweeper re-review

clawsweeper · 2026-07-09T12:07:39Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

…ence

steipete · 2026-07-09T14:01:39Z

Merged via squash.

Prepared head SHA: 27630110e4b91af5ea27e1b500249ec27d4401bb
Landed commit: 2e9e46bb37948660567ba75da523b794b0c9980b

* fix(crestodian): keep assistant prompt history truncation UTF-16 safe * test(crestodian): simplify history boundary proof * test(crestodian): add dangling-surrogate proof and raw-slice bug evidence * test(crestodian): remove duplicate UTF-16 cases --------- Co-authored-by: Peter Steinberger <[email protected]>

openclaw-barnacle Bot added the size: S label Jul 9, 2026

openclaw-barnacle Bot added size: XS and removed size: S labels Jul 9, 2026

steipete changed the title ~~fix(crestodian): keep assistant prompt history truncation UTF-16 safe~~ fix(crestodian): keep prompt history UTF-16 safe Jul 9, 2026

openclaw-barnacle Bot added size: S and removed size: XS labels Jul 9, 2026

clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jul 9, 2026

openclaw-barnacle Bot added size: XS and removed size: S labels Jul 9, 2026

wangyan2026 and others added 4 commits July 9, 2026 14:49

fix(crestodian): keep assistant prompt history truncation UTF-16 safe

b8fe3f4

test(crestodian): simplify history boundary proof

eaece37

test(crestodian): add dangling-surrogate proof and raw-slice bug evid…

1954aff

…ence

test(crestodian): remove duplicate UTF-16 cases

2763011

steipete force-pushed the fix/crestodian-history-utf16-safe-truncation branch from 065b708 to 2763011 Compare July 9, 2026 13:52

steipete merged commit 2e9e46b into openclaw:main Jul 9, 2026
97 checks passed

github-actions Bot mentioned this pull request Jul 9, 2026

📡 Upstream Digest — 2026-07-09 14:36 UTC curtismercier/openclaw-mods#1243

Open

This was referenced Jul 9, 2026

fix(qqbot): guard mentionPatterns against ReDoS via compileSafeRegex #102976

Merged

fix(workshop): keep approval target names UTF-16 safe #102963

Merged

steipete mentioned this pull request Jul 9, 2026

docs(changelog): credit recent UTF-16 and display fixes #103102

Merged

mushuiyu886 mentioned this pull request Jul 10, 2026

fix(diagnostics-otel): preserve JSON Unicode boundaries #103646

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(crestodian): keep prompt history UTF-16 safe#102627

fix(crestodian): keep prompt history UTF-16 safe#102627
steipete merged 4 commits into
openclaw:mainfrom
wangyan2026:fix/crestodian-history-utf16-safe-truncation

wangyan2026 commented Jul 9, 2026 •

edited by steipete

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

wangyan2026 commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

wangyan2026 commented Jul 9, 2026 • edited by steipete Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wangyan2026 commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wangyan2026 commented Jul 9, 2026 •

edited by steipete

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading