fix(channels): keep inbound log previews UTF-16 safe by miorbnli · Pull Request #102407 · openclaw/openclaw

miorbnli · 2026-07-09T04:08:47Z

What Problem This Solves

Matrix, Line, and Telegram verbose inbound previews used raw String.prototype.slice limits. A supplementary-plane character crossing one of those UTF-16 code-unit boundaries produced a lone surrogate in diagnostic output. Telegram's reply-context preview had the same defect at its 120-unit limit.

Why This Change Was Made

Reuse the existing truncateUtf16Safe helper at every affected channel-owned logging boundary.
Preserve the existing 200-unit inbound and 120-unit reply-context limits plus newline/whitespace normalization.
Replace the original helper-only Matrix test with production-path assertions for Matrix, Line, Telegram inbound, and Telegram reply-context logging.
Include Line and Telegram now because they shared the exact invariant and had no competing open fix.

User Impact

Verbose channel logs no longer contain malformed lone surrogates when emoji or other astral characters cross a preview boundary. ASCII output, routing, delivery, prompts, and configuration are unchanged.

Linked context

Sibling fix: #101630.

Evidence

Exact head: cd7e34fe4857102133b9df9ee6e6095441f6de33

Blacksmith Testbox run 28997946449:
- Matrix handler: 119 passed
- Line message context: 17 passed
- Telegram verbose log context: 2 passed
- Total: 138 passed across three Vitest shards
Blacksmith Testbox run 28998352418: full test TypeScript check passed after tightening the Line fixture types.
Targeted oxfmt and git diff --check: passed.
Fresh full-diff autoreview: clean, confidence 0.98.

The regression inputs place the high surrogate of an emoji exactly at each production preview limit and assert the complete emitted log line. The safe result backs up one code unit instead of emitting a malformed half-pair.

Real behavior proof

Field	Value
Behavior addressed	UTF-16 surrogate pairs split by Matrix, Line, and Telegram diagnostic preview limits
Environment	Blacksmith Testbox, Linux x64, exact PR head
Production paths	Matrix handler, Line message-context finalizer, Telegram inbound and reply-context builders
Result	All 138 focused tests passed; exact emitted previews contain no lone surrogate
Not tested	Credentialed live Matrix/Line/Telegram traffic; the change only affects local verbose-log formatting and the production handlers are exercised directly

Risk checklist

No config, environment, default, SDK, protocol, auth, or provider change.
No new dependency.
Existing limits and normalization remain unchanged.
No changelog entry required for diagnostic-only internal behavior.
Allow edits by maintainers.

The matrix inbound verbose preview used bodyText.slice(0, 200) before logging. When a supplementary-plane character (emoji, extended CJK) straddles the 200th code unit, the slice splits its surrogate pair and emits a lone surrogate into the verbose log line. Lone surrogates corrupt JSON log serialization, break terminal rendering, and crash UTF-8-validating log shippers. Use the shared truncateUtf16Safe helper (already used by the mattermost sibling monitor after openclaw#101630) to keep complete surrogate pairs intact. No behavior change for ASCII input; the preview is still capped at 200 code units. Co-Authored-By: Claude <[email protected]>

clawsweeper · 2026-07-09T04:13:22Z

Codex review: needs maintainer review before merge. Reviewed July 9, 2026, 1:12 AM ET / 05:12 UTC.

Summary
This PR replaces Matrix inbound verbose-preview bodyText.slice(0, 200) with truncateUtf16Safe(bodyText, 200) and adds a focused UTF-16 boundary regression test.

PR surface: Source +1, Tests +49. Total +50 across 2 files.

Reproducibility: yes. from source inspection: current main and v2026.6.11 slice bodyText at 200 UTF-16 code units before logging, so a 199-ASCII prefix followed by an emoji reaches the lone-surrogate cut point. I did not run the reproduction because this review is read-only.

Review metrics: none identified.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Refresh exact-head or merge-head validation after updating from current main, since GitHub reports the branch as behind.

Risk before merge

[P1] The PR is behind current main, so landing should use a refreshed merge-head or exact-head validation rather than relying only on the earlier base state.

Maintainer options:

Decide the mitigation before merge
Land the narrow Matrix verbose-log helper fix after normal maintainer review and refreshed exact-head or merge-head validation; leave broader preview cleanup to separate, path-specific work.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

No ClawSweeper repair lane is needed because there are no blocking review findings; the remaining action is normal maintainer review plus refreshed validation for the behind branch.

Security
Cleared: The diff imports an existing SDK helper and adds a focused test, with no dependency, lockfile, workflow, package, secret, permission, auth, or executable supply-chain change.

Review details

Best possible solution:

Land the narrow Matrix verbose-log helper fix after normal maintainer review and refreshed exact-head or merge-head validation; leave broader preview cleanup to separate, path-specific work.

Do we have a high-confidence way to reproduce the issue?

Yes, from source inspection: current main and v2026.6.11 slice bodyText at 200 UTF-16 code units before logging, so a 199-ASCII prefix followed by an emoji reaches the lone-surrogate cut point. I did not run the reproduction because this review is read-only.

Is this the best way to solve the issue?

Yes. Using the existing plugin SDK truncateUtf16Safe helper at the Matrix verbose-preview call site is the narrowest maintainable fix; a full handler-path regression would be stronger evidence, but the code change itself is a direct expression replacement in a diagnostic log path.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 624dfa6cf6dc.

Label changes

Label changes:

add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied live Node output comparing the old raw slice with truncateUtf16Safe on a boundary surrogate payload; no live Matrix homeserver was used, but the changed behavior is a diagnostic preview expression.
add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🦞 diamond lobster.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied live Node output comparing the old raw slice with truncateUtf16Safe on a boundary surrogate payload; no live Matrix homeserver was used, but the changed behavior is a diagnostic preview expression.

Label justifications:

P3: This is a low-blast-radius Matrix verbose diagnostic preview fix with no delivery, config, auth, storage, migration, or dependency change.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied live Node output comparing the old raw slice with truncateUtf16Safe on a boundary surrogate payload; no live Matrix homeserver was used, but the changed behavior is a diagnostic preview expression.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied live Node output comparing the old raw slice with truncateUtf16Safe on a boundary surrogate payload; no live Matrix homeserver was used, but the changed behavior is a diagnostic preview expression.

Evidence reviewed

PR surface:

Source +1, Tests +49. Total +50 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	2	1	+1
Tests	1	49	0	+49
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	51	1	+50

What I checked:

Repository policy read: Root AGENTS.md and scoped extensions/AGENTS.md were read; the extension SDK import boundary and review-depth guidance affected this review. (AGENTS.md:1, 624dfa6cf6dc)
Current main still has the bug shape: Current main builds the Matrix inbound verbose preview with raw bodyText.slice(0, 200), which can split a surrogate pair at the 200-code-unit boundary. (extensions/matrix/src/matrix/monitor/handler.ts:1659, 624dfa6cf6dc)
Latest release still has the raw slice: Release v2026.6.11 also contains bodyText.slice(0, 200) in the Matrix inbound preview path, so this change is not already shipped. (extensions/matrix/src/matrix/monitor/handler.ts:1699, e085fa1a3ffd)
PR head applies the established helper: The PR imports truncateUtf16Safe from the public plugin SDK text utility runtime and applies it before newline escaping in the Matrix verbose preview line. (extensions/matrix/src/matrix/monitor/handler.ts:1657, 47618a12fdd4)
Helper contract checked: truncateUtf16Safe clamps/floors the limit and delegates to sliceUtf16Safe, whose end-boundary check backs off before returning a dangling high surrogate. (packages/normalization-core/src/utf16-slice.ts:44, 624dfa6cf6dc)
SDK export path checked: openclaw/plugin-sdk/text-utility-runtime is an existing exported subpath, so the Matrix extension stays within the scoped extension boundary. (package.json:1427, 624dfa6cf6dc)

Likely related people:

steipete: GitHub file history shows repeated Matrix handler and channel-inbound refactors, including Matrix voice/thread work and central inbound supplemental context that carries this handler path. (role: feature/refactor owner; confidence: high; commits: 25149801189f, 1507a9701b83, fa2b2ffab46f; files: extensions/matrix/src/matrix/monitor/handler.ts, extensions/matrix/src/matrix/monitor/handler.test-helpers.ts)
gumadeiras: Matrix monitor history includes Matrix progress-preview, sync outage, E2EE QA, and reviewed Matrix PRs associated with this channel area. (role: adjacent Matrix channel contributor/reviewer; confidence: medium; commits: 24068f19c6ae, 0c00c3c2306f, 9b19c0b87f32; files: extensions/matrix/src/matrix/monitor/handler.ts, extensions/matrix/src/matrix/monitor/index.ts)
vincentkoc: Recent Matrix history includes local internal-type work plus several Matrix inbound/final-delivery fixes near the same plugin boundary. (role: recent area contributor; confidence: medium; commits: 4115e832d833, d242774ec6aa, 0d565833e15e; files: extensions/matrix/src/matrix/monitor/index.ts, extensions/matrix/src/matrix/monitor/handler.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (2 earlier review cycles)

reviewed 2026-07-09T04:17:35.531Z sha 9d2d398 :: needs real behavior proof before merge. :: [P2] Avoid ES2024 isWellFormed in the test helper
reviewed 2026-07-09T05:00:44.376Z sha 9d2d398 :: needs real behavior proof before merge. :: [P2] Avoid ES2024 isWellFormed in the test helper

- Avoid String.prototype.isWellFormed (ES2024 lib): use the encodeURIComponent well-formedness probe instead, which works on the project's TS lib target. - Replace literal string concatenation with template strings to satisfy the no-useless-concat lint rule. No behavior change in the assertions. Co-Authored-By: Claude <[email protected]>

steipete · 2026-07-09T06:32:17Z

Land-ready verification at exact head cd7e34fe4857102133b9df9ee6e6095441f6de33:

Completed the invariant across Matrix, Line, Telegram inbound, and Telegram reply-context diagnostic previews; existing 200/120-unit limits and normalization remain unchanged.
Replaced helper-only coverage with exact production-log assertions.
Blacksmith Testbox 28997946449: 138 focused tests passed (Matrix 119, Line 17, Telegram 2).
The first exact-head CI exposed two missing LINE SDK fixture assertions in check-test-types; those were fixed. Blacksmith Testbox 28998352418 then passed the full test TypeScript check.
Replacement exact-head CI 28998504675: passed.
Targeted formatting and diff checks passed; fresh whole-branch autoreview clean at 0.98 confidence.
Native OPENCLAW_TESTBOX=1 scripts/pr prepare-run 102407: passed with validated review artifacts.

No changelog entry is required because this only corrects internal verbose-log formatting.

steipete · 2026-07-09T06:32:59Z

Merged via squash.

Prepared head SHA: cd7e34fe4857102133b9df9ee6e6095441f6de33
Landed commit: 18e80760a70da0be87e976e32dd29a9c0ab7e098

* fix(matrix): truncate inbound preview on UTF-16 code-point boundary The matrix inbound verbose preview used bodyText.slice(0, 200) before logging. When a supplementary-plane character (emoji, extended CJK) straddles the 200th code unit, the slice splits its surrogate pair and emits a lone surrogate into the verbose log line. Lone surrogates corrupt JSON log serialization, break terminal rendering, and crash UTF-8-validating log shippers. Use the shared truncateUtf16Safe helper (already used by the mattermost sibling monitor after openclaw#101630) to keep complete surrogate pairs intact. No behavior change for ASCII input; the preview is still capped at 200 code units. Co-Authored-By: Claude <[email protected]> * test(matrix): fix inbound-preview test for lint and lib target - Avoid String.prototype.isWellFormed (ES2024 lib): use the encodeURIComponent well-formedness probe instead, which works on the project's TS lib target. - Replace literal string concatenation with template strings to satisfy the no-useless-concat lint rule. No behavior change in the assertions. Co-Authored-By: Claude <[email protected]> * fix(channels): keep inbound log previews UTF-16 safe * test(line): type verbose preview fixtures --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Peter Steinberger <[email protected]>

openclaw-barnacle Bot added channel: matrix Channel integration: matrix size: S triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jul 9, 2026

miorbnli mentioned this pull request Jul 9, 2026

fix(proxy-capture): truncate inline preview on UTF-8 character boundary #102409

Merged

openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jul 9, 2026

steipete added 2 commits July 9, 2026 07:00

Merge remote-tracking branch 'origin/main' into codex/pr-102407-fixups

523bc00

fix(channels): keep inbound log previews UTF-16 safe

69b691b

openclaw-barnacle Bot added channel: line Channel integration: line channel: telegram Channel integration: telegram labels Jul 9, 2026

steipete changed the title ~~fix(matrix): truncate inbound preview on UTF-16 code-point boundary~~ fix(channels): keep inbound log previews UTF-16 safe Jul 9, 2026

test(line): type verbose preview fixtures

cd7e34f

steipete merged commit 18e8076 into openclaw:main Jul 9, 2026
97 of 98 checks passed

This was referenced Jul 9, 2026

fix(text): keep five src-side text truncations UTF-16 safe #102630

Closed

fix(chat,cli): keep session snippet and CLI output digest truncation UTF-16 safe #102747

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(channels): keep inbound log previews UTF-16 safe#102407

fix(channels): keep inbound log previews UTF-16 safe#102407
steipete merged 5 commits into
openclaw:mainfrom
miorbnli:fix/matrix-inbound-preview-surrogate-truncation

miorbnli commented Jul 9, 2026 •

edited by steipete

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

miorbnli commented Jul 9, 2026 • edited by steipete Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Linked context

Evidence

Real behavior proof

Risk checklist

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

miorbnli commented Jul 9, 2026 •

edited by steipete

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading