fix(gateway): keep chat history display text truncation surrogate-safe#102470
Conversation
truncateChatHistoryText uses slice(0, N) to truncate chat history text for the Control UI. When the truncation boundary falls inside a UTF-16 surrogate pair (emoji, CJK extended), the resulting string contains a dangling surrogate that browsers render as U+FFFD (�). Replace slice(0, maxChars) with truncateUtf16Safe(text, maxChars) so the truncation point always falls on a complete code-point boundary.
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Codex review: needs real behavior proof before merge. Reviewed July 9, 2026, 2:59 AM ET / 06:59 UTC. Summary PR surface: Source +1, Tests +21. Total +22 across 2 files. Reproducibility: yes. source-reproducible: current main and the latest release use raw Review metrics: none identified. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Proof guidance:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Land the helper substitution after exact-head gates once the PR includes redacted real behavior proof showing the gateway/UI projection no longer emits a replacement glyph at the boundary. Do we have a high-confidence way to reproduce the issue? Yes, source-reproducible: current main and the latest release use raw Is this the best way to solve the issue? Yes for the code shape: reusing the existing AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 245b91b83de0. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Source +1, Tests +21. Total +22 across 2 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (1 earlier review cycle)
|
|
Land-ready maintainer pass complete at exact head The shared gateway fix was correct but untested in the diff. I added a production-path regression through Proof:
The focused test ran locally because fork-contributor code is not eligible for credential-hydrated Testbox execution before landing review. Exact-head fork CI supplies the broad hosted proof. No remaining proof gap. |
|
Merged via squash.
|
openclaw#102470) * fix(gateway): keep chat history display text truncation surrogate-safe truncateChatHistoryText uses slice(0, N) to truncate chat history text for the Control UI. When the truncation boundary falls inside a UTF-16 surrogate pair (emoji, CJK extended), the resulting string contains a dangling surrogate that browsers render as U+FFFD (�). Replace slice(0, maxChars) with truncateUtf16Safe(text, maxChars) so the truncation point always falls on a complete code-point boundary. * test(gateway): cover chat display UTF-16 boundary --------- Co-authored-by: Peter Steinberger <[email protected]>
…cript truncation UTF-16 safe normalizeCliContextValue (cli-output.ts) and formatSkillExperienceReviewTranscript (experience-review-prompt.ts) both used raw .slice(0, N) / .slice(-N) to truncate user-facing strings. When a truncation boundary falls inside a surrogate pair (emoji), the resulting string contains a lone surrogate — encodeURIComponent throws URIError, JSON serialisers may reject it, and downstream display surfaces can render garbage. Replace .slice(0, 200) with truncateUtf16Safe in normalizeCliContextValue. Replace .slice(0, 6_000) with truncateUtf16Safe and full.slice(-tailBudget) with sliceUtf16Safe(full, tailStart) in formatSkillExperienceReviewTranscript. Follows the pattern established by openclaw#98644, openclaw#102470, openclaw#101728 and the broader UTF-16 safety sweep already applied across the codebase.
…cript truncation UTF-16 safe normalizeCliContextValue (cli-output.ts) and formatSkillExperienceReviewTranscript (experience-review-prompt.ts) both used raw .slice(0, N) / .slice(-N) to truncate user-facing strings. When a truncation boundary falls inside a surrogate pair (emoji), the resulting string contains a lone surrogate — encodeURIComponent throws URIError, JSON serialisers may reject it, and downstream display surfaces can render garbage. Replace .slice(0, 200) with truncateUtf16Safe in normalizeCliContextValue. Replace .slice(0, 6_000) with truncateUtf16Safe and full.slice(-tailBudget) with sliceUtf16Safe(full, tailStart) in formatSkillExperienceReviewTranscript. Follows the pattern established by openclaw#98644, openclaw#102470, openclaw#101728 and the broader UTF-16 safety sweep already applied across the codebase.
…cript truncation UTF-16 safe normalizeCliContextValue (cli-output.ts) and formatSkillExperienceReviewTranscript (experience-review-prompt.ts) both used raw .slice(0, N) / .slice(-N) to truncate user-facing strings. When a truncation boundary falls inside a surrogate pair (emoji), the resulting string contains a lone surrogate — encodeURIComponent throws URIError, JSON serialisers may reject it, and downstream display surfaces can render garbage. Replace .slice(0, 200) with truncateUtf16Safe in normalizeCliContextValue. Replace .slice(0, 6_000) with truncateUtf16Safe and full.slice(-tailBudget) with sliceUtf16Safe(full, tailStart) in formatSkillExperienceReviewTranscript. Follows the pattern established by openclaw#98644, openclaw#102470, openclaw#101728 and the broader UTF-16 safety sweep already applied across the codebase.
…cript truncation UTF-16 safe normalizeCliContextValue (cli-output.ts) and formatSkillExperienceReviewTranscript (experience-review-prompt.ts) both used raw .slice(0, N) / .slice(-N) to truncate user-facing strings. When a truncation boundary falls inside a surrogate pair (emoji), the resulting string contains a lone surrogate — encodeURIComponent throws URIError, JSON serialisers may reject it, and downstream display surfaces can render garbage. Replace .slice(0, 200) with truncateUtf16Safe in normalizeCliContextValue. Replace .slice(0, 6_000) with truncateUtf16Safe and full.slice(-tailBudget) with sliceUtf16Safe(full, tailStart) in formatSkillExperienceReviewTranscript. Follows the pattern established by openclaw#98644, openclaw#102470, openclaw#101728 and the broader UTF-16 safety sweep already applied across the codebase.
…cript truncation UTF-16 safe (#106370) * fix(agents,skills): keep session-identity and experience-review transcript truncation UTF-16 safe normalizeCliContextValue (cli-output.ts) and formatSkillExperienceReviewTranscript (experience-review-prompt.ts) both used raw .slice(0, N) / .slice(-N) to truncate user-facing strings. When a truncation boundary falls inside a surrogate pair (emoji), the resulting string contains a lone surrogate — encodeURIComponent throws URIError, JSON serialisers may reject it, and downstream display surfaces can render garbage. Replace .slice(0, 200) with truncateUtf16Safe in normalizeCliContextValue. Replace .slice(0, 6_000) with truncateUtf16Safe and full.slice(-tailBudget) with sliceUtf16Safe(full, tailStart) in formatSkillExperienceReviewTranscript. Follows the pattern established by #98644, #102470, #101728 and the broader UTF-16 safety sweep already applied across the codebase. * chore: re-trigger CI after PR body update * test(agents,skills): construct transcripts exceeding 60k chars, verify old slices dangle Both experience-review fixtures now exceed EXPERIENCE_REVIEW_MAX_TRANSCRIPT_CHARS so the truncation branch actually fires. Pre-condition assertions prove the old raw .slice(0,6000) / .slice(tailStart) produce isolated surrogates; the production functions (formatSkillExperienceReviewTranscript, formatCliOutputError) do not. cli-output test also adds pre-condition check showing normalizeCliContextValue with raw .slice(0,200) would split a surrogate pair. * test: tighten UTF-16 truncation coverage Co-authored-by: 毛宇豪0668001457 <[email protected]> --------- Co-authored-by: Peter Steinberger <[email protected]>
…cript truncation UTF-16 safe (openclaw#106370) * fix(agents,skills): keep session-identity and experience-review transcript truncation UTF-16 safe normalizeCliContextValue (cli-output.ts) and formatSkillExperienceReviewTranscript (experience-review-prompt.ts) both used raw .slice(0, N) / .slice(-N) to truncate user-facing strings. When a truncation boundary falls inside a surrogate pair (emoji), the resulting string contains a lone surrogate — encodeURIComponent throws URIError, JSON serialisers may reject it, and downstream display surfaces can render garbage. Replace .slice(0, 200) with truncateUtf16Safe in normalizeCliContextValue. Replace .slice(0, 6_000) with truncateUtf16Safe and full.slice(-tailBudget) with sliceUtf16Safe(full, tailStart) in formatSkillExperienceReviewTranscript. Follows the pattern established by openclaw#98644, openclaw#102470, openclaw#101728 and the broader UTF-16 safety sweep already applied across the codebase. * chore: re-trigger CI after PR body update * test(agents,skills): construct transcripts exceeding 60k chars, verify old slices dangle Both experience-review fixtures now exceed EXPERIENCE_REVIEW_MAX_TRANSCRIPT_CHARS so the truncation branch actually fires. Pre-condition assertions prove the old raw .slice(0,6000) / .slice(tailStart) produce isolated surrogates; the production functions (formatSkillExperienceReviewTranscript, formatCliOutputError) do not. cli-output test also adds pre-condition check showing normalizeCliContextValue with raw .slice(0,200) would split a surrogate pair. * test: tighten UTF-16 truncation coverage Co-authored-by: 毛宇豪0668001457 <[email protected]> --------- Co-authored-by: Peter Steinberger <[email protected]>
…cript truncation UTF-16 safe (openclaw#106370) * fix(agents,skills): keep session-identity and experience-review transcript truncation UTF-16 safe normalizeCliContextValue (cli-output.ts) and formatSkillExperienceReviewTranscript (experience-review-prompt.ts) both used raw .slice(0, N) / .slice(-N) to truncate user-facing strings. When a truncation boundary falls inside a surrogate pair (emoji), the resulting string contains a lone surrogate — encodeURIComponent throws URIError, JSON serialisers may reject it, and downstream display surfaces can render garbage. Replace .slice(0, 200) with truncateUtf16Safe in normalizeCliContextValue. Replace .slice(0, 6_000) with truncateUtf16Safe and full.slice(-tailBudget) with sliceUtf16Safe(full, tailStart) in formatSkillExperienceReviewTranscript. Follows the pattern established by openclaw#98644, openclaw#102470, openclaw#101728 and the broader UTF-16 safety sweep already applied across the codebase. * chore: re-trigger CI after PR body update * test(agents,skills): construct transcripts exceeding 60k chars, verify old slices dangle Both experience-review fixtures now exceed EXPERIENCE_REVIEW_MAX_TRANSCRIPT_CHARS so the truncation branch actually fires. Pre-condition assertions prove the old raw .slice(0,6000) / .slice(tailStart) produce isolated surrogates; the production functions (formatSkillExperienceReviewTranscript, formatCliOutputError) do not. cli-output test also adds pre-condition check showing normalizeCliContextValue with raw .slice(0,200) would split a surrogate pair. * test: tighten UTF-16 truncation coverage Co-authored-by: 毛宇豪0668001457 <[email protected]> --------- Co-authored-by: Peter Steinberger <[email protected]>
What Problem This Solves
Chat-history display projection truncated text with a raw UTF-16 slice. When the configured display boundary fell inside an emoji or another supplementary character, the Control UI/history payload could contain a lone surrogate that rendered as a replacement character.
Why This Change Was Made
The owning
truncateChatHistoryTexthelper now uses the shared normalization-coretruncateUtf16Safeprimitive. This covers assistant text, message previews, thinking text, and other projected display fields while preserving the existing code-unit cap and truncation marker.A production-path regression drives assistant content through
sanitizeChatHistoryMessagesat a split surrogate boundary and asserts the exact projected payload.User Impact
Control UI and chat-history previews no longer display a replacement glyph when emoji or other supplementary Unicode characters cross a display truncation boundary.
Evidence
b9e92b96b6015fbb5a464e76d4126cfb9b9f2e23node scripts/run-vitest.mjs src/gateway/server-methods/server-methods.test.ts— 1 file, 171 tests passed.git diff --checkpassed.0.99).Local narrow test fallback was used because fork-contributor code is not eligible for credential-hydrated Testbox execution before landing review.
AI-assisted; maintainer-reviewed.