Skip to content

fix(matrix): truncate reply context on code-point boundaries to avoid splitting surrogate pairs#96412

Closed
ly-wang19 wants to merge 1 commit into
openclaw:mainfrom
ly-wang19:fix/matrix-reply-context-surrogate-safe
Closed

fix(matrix): truncate reply context on code-point boundaries to avoid splitting surrogate pairs#96412
ly-wang19 wants to merge 1 commit into
openclaw:mainfrom
ly-wang19:fix/matrix-reply-context-surrogate-safe

Conversation

@ly-wang19

Copy link
Copy Markdown
Contributor

What Problem This Solves

truncateReplyBody in extensions/matrix/src/matrix/monitor/reply-context.ts (reached via the exported summarizeMatrixReplyEvent) shortens long reply bodies with a raw value.slice(0, MAX_REPLY_BODY_LENGTH - 3). UTF-16 .slice cuts at a code-unit index, not a code-point boundary. When the cut index lands between the two halves of a surrogate pair (any emoji or other astral character), the slice keeps a lone high surrogate, which is appended directly before the ... ellipsis. The reply context surfaced to the agent then contains a dangling \uD83D that renders as a broken replacement glyph.

Repro: a plain m.room.message whose content.body is 496 a characters, then the emoji 😀 (U+1F600, a surrogate pair at UTF-16 indices 496–497), then bcd (total length 501):

summarizeMatrixReplyEvent({
  type: "m.room.message",
  content: { msgtype: "m.text", body: "a".repeat(496) + "😀" + "bcd" },
});
  • Before (wrong): value.slice(0, 497) splits the emoji, returning "a".repeat(496) + "\uD83D" + "..." — a lone high surrogate (broken glyph) before the ellipsis.
  • After (fixed): truncation falls back to the previous code-point boundary: "a".repeat(496) + "...". The half emoji is dropped; no orphaned surrogate.

The fix swaps the raw slice for sliceUtf16Safe (already exported from openclaw/plugin-sdk/text-utility-runtime and used by sibling extensions such as discord and memory-lancedb), which never cuts inside a surrogate pair. Bodies without astral characters are unaffected — a 600-char ASCII body still truncates to "x".repeat(497) + "..." exactly as before.

Evidence

Standalone Node red/green proof. It copies sliceUtf16Safe verbatim from src/utils.ts and runs both the buggy (.slice) and fixed (sliceUtf16Safe) versions of truncateReplyBody against the repro input plus an unaffected normal body:

PASS: repro input length is 501
PASS: emoji high surrogate at index 496
PASS: emoji low surrogate at index 497
PASS: BEFORE leaves a lone high surrogate (the bug)
PASS: BEFORE differs from expected (proves bug)
PASS: BEFORE is 'a'*496 + \uD83D + '...'
PASS: AFTER equals expected (a*496 + ...)
PASS: AFTER has no orphaned high surrogate
PASS: AFTER length <= 500
PASS: AFTER ends with ...
PASS: normal long ASCII body: BEFORE === AFTER (no behavior change)
PASS: normal long ASCII body truncates to x*497 + ...
PASS: short body unchanged BEFORE
PASS: short body unchanged AFTER

ALL CHECKS PASSED — red/green proof confirmed.

A regression test is added to reply-context.test.ts (truncates on a code-point boundary without orphaning a surrogate half) that fails on the raw-slice code and passes with the fix.

truncateReplyBody used a raw value.slice(0, MAX_REPLY_BODY_LENGTH - 3)
to shorten long reply bodies. When the cut index fell between the two
UTF-16 code units of a surrogate pair (e.g. an emoji), the slice left a
lone high surrogate before the ellipsis, which renders as a broken glyph
in the reply context shown to the agent.

Replace the raw slice with sliceUtf16Safe from the plugin SDK so the
truncation never cuts inside a surrogate pair. A normal (non-astral)
body is unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@openclaw-barnacle openclaw-barnacle Bot added channel: matrix Channel integration: matrix size: XS r: too-many-prs Auto-close: author has more than twenty active PRs. labels Jun 24, 2026
@openclaw-barnacle

Copy link
Copy Markdown

Closing this PR because the author has more than 20 active PRs in this repo. Please reduce the active PR queue and reopen or resubmit once it is back under the limit. You can close your own PRs to get back under the limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: matrix Channel integration: matrix r: too-many-prs Auto-close: author has more than twenty active PRs. size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant