fix(mattermost): truncate inbound preview on code-point boundary#101630
Conversation
|
Codex review: needs real behavior proof before merge. Reviewed July 8, 2026, 12:47 AM ET / 04:47 UTC. Summary PR surface: Source +1, Tests +48. Total +49 across 2 files. Reproducibility: yes. Current main and Review metrics: none identified. Stored data model Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Proof guidance:
Risk before merge
Maintainer options:
Next step before merge
Maintainer decision needed
Security Review detailsBest possible solution: Land the helper-based Mattermost logging fix after a maintainer proof override or redacted live Mattermost verbose-log proof confirms the safe preview on the latest head. Do we have a high-confidence way to reproduce the issue? Yes. Current main and Is this the best way to solve the issue? Yes for the code shape. Using the existing SDK UTF-16-safe truncation helper at the Mattermost verbose-preview site is the narrowest maintainable fix; the remaining question is proof sufficiency, not patch correctness. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against bdc1ce8dede0. Label changesLabel justifications:
Evidence reviewedPR surface: Source +1, Tests +48. Total +49 across 2 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (9 earlier review cycles; latest 8 shown)
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
a9d040b to
dee2062
Compare
dee2062 to
72d2857
Compare
|
@clawsweeper re-review Squashed to one commit, removed |
|
🦞🧹 I asked ClawSweeper to review this item again. |
e7889fe to
42edba3
Compare
|
Maintainer fixup pushed at The websocket monitor regression already exercises the real production logging path, so I removed the later standalone proof script, which only duplicated the truncation expression and helper import. The runtime fix and production-path regression are unchanged. I am running the native exact-head landing sequence now. Please hold further head updates until it completes so GitHub does not cancel the reviewed CI run. |
14e7883 to
b1f504f
Compare
|
🦞🧹 I asked ClawSweeper to review this item again. |
b1f504f to
569e61e
Compare
|
🦞🧹 I asked ClawSweeper to review this item again. |
Co-authored-by: Cursor <[email protected]>
569e61e to
1fb6e28
Compare
Hi @steipete — branch is now rebased on latest main (head: 1fb6e28, one commit). I won't push again until you decide. The proof gate is still showing needs proof. You already confirmed the posted-event regression exercises the real production logging path. Could you either apply the proof: override label or complete the landing sequence from this head? I'll hold here. |
|
Land-ready at exact head
Known proof gap: no live Mattermost server capture; the production monitor/websocket path is covered in the focused regression test. No ingress, auth, routing, config, or message-body behavior changes. |
|
Merged via squash.
|
* fix(matrix): truncate inbound preview on UTF-16 code-point boundary The matrix inbound verbose preview used bodyText.slice(0, 200) before logging. When a supplementary-plane character (emoji, extended CJK) straddles the 200th code unit, the slice splits its surrogate pair and emits a lone surrogate into the verbose log line. Lone surrogates corrupt JSON log serialization, break terminal rendering, and crash UTF-8-validating log shippers. Use the shared truncateUtf16Safe helper (already used by the mattermost sibling monitor after #101630) to keep complete surrogate pairs intact. No behavior change for ASCII input; the preview is still capped at 200 code units. Co-Authored-By: Claude <[email protected]> * test(matrix): fix inbound-preview test for lint and lib target - Avoid String.prototype.isWellFormed (ES2024 lib): use the encodeURIComponent well-formedness probe instead, which works on the project's TS lib target. - Replace literal string concatenation with template strings to satisfy the no-useless-concat lint rule. No behavior change in the assertions. Co-Authored-By: Claude <[email protected]> * fix(channels): keep inbound log previews UTF-16 safe * test(line): type verbose preview fixtures --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Peter Steinberger <[email protected]>
…nclaw#101630) Co-authored-by: NIO <[email protected]> Co-authored-by: Cursor <[email protected]>
* fix(matrix): truncate inbound preview on UTF-16 code-point boundary The matrix inbound verbose preview used bodyText.slice(0, 200) before logging. When a supplementary-plane character (emoji, extended CJK) straddles the 200th code unit, the slice splits its surrogate pair and emits a lone surrogate into the verbose log line. Lone surrogates corrupt JSON log serialization, break terminal rendering, and crash UTF-8-validating log shippers. Use the shared truncateUtf16Safe helper (already used by the mattermost sibling monitor after openclaw#101630) to keep complete surrogate pairs intact. No behavior change for ASCII input; the preview is still capped at 200 code units. Co-Authored-By: Claude <[email protected]> * test(matrix): fix inbound-preview test for lint and lib target - Avoid String.prototype.isWellFormed (ES2024 lib): use the encodeURIComponent well-formedness probe instead, which works on the project's TS lib target. - Replace literal string concatenation with template strings to satisfy the no-useless-concat lint rule. No behavior change in the assertions. Co-Authored-By: Claude <[email protected]> * fix(channels): keep inbound log previews UTF-16 safe * test(line): type verbose preview fixtures --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Peter Steinberger <[email protected]>
…hreadDirectMessages The 7.2 cherry-pick resolved extensions/mattermost/src/mattermost/monitor.ts by taking our side wholesale, silently reverting six upstream commits landed between 7.1 and 7.2. Adopt them instead of clobbering: f77c2aa truncate the inbound preview on a code-point boundary (openclaw#101630) b7ab62c preserve text-block boundaries in the draft preview (openclaw#87449) da6a24d prevent hung pre-delivery hooks from blocking lanes (openclaw#104256) 218dcd8 noUncheckedIndexedAccess in the extensions lane (openclaw#105132) 9cdf166 trim the monitor gating facade (openclaw#106021) 70a7e6e (opt-in DM threading by chat type) was already in the impl; three 7.1-era assertions in monitor.test.ts that upstream had superseded move to upstream's semantics (the implementations are byte-identical to upstream). b7ab62c's 173-line monitor hunk (block-preview boundary machinery) was hand-ported onto our activity/warm-fallback interleaving; the block-preview code is now byte-identical to pristine 7.2. threadDirectMessages kept as a thin compat feeding upstream's replyToModeByChatType, because Dockerfile.base's shared config probe still writes it and the probe spans matrix bases older than 2026.7.2. Retire both together later. Mattermost suite 630/632. The 2 remaining timeouts are localized to deliverMattermostReplyWithDraftPreview: with a mocked-rejecting updateMattermostPost, our patched buildFinalEdit attempts the in-place edit (pristine skips it), the edit rejects, and the deliverNormally fallback awaits the unmocked deliverMattermostReplyPayload which never resolves under the test's bare sendMessageMattermost stub. Behavioral divergence in our activity/warm-fallback finalize path, not the upstream port. TODO.
What Problem This Solves
Fixes garbled verbose inbound previews in Mattermost when message bodies contain emoji or other astral characters near the 200-character preview limit. The verbose debug path used raw
bodyText.slice(0, 200), which can split a UTF-16 surrogate pair and leave a lone high surrogate in gateway logs.Concrete example: A 205-code-unit message
${"a".repeat(199)}😀tailhits the preview cap inside the emoji.slice(0, 200)keeps the emoji's high surrogate (0xD83D) but drops its low surrogate, producing mojibake inmattermost inbound: … preview="…"debug lines.Why This Change Was Made
Replaces
bodyText.slice(0, 200)withtruncateUtf16Safe(bodyText, 200)fromopenclaw/plugin-sdk/text-utility-runtimein the verbose inbound preview path (extensions/mattermost/src/mattermost/monitor.ts:1681).User Impact
Before: Verbose Mattermost inbound logging can show broken previews for emoji-heavy messages.
After: Previews truncate on complete UTF-16 boundaries; emoji at the cap boundary are excluded rather than split.
Evidence
extensions/mattermost/src/mattermost/monitor.ts:27,1681–1683—truncateUtf16Safeimport and verbose preview wiringpackages/normalization-core/src/utf16-slice.ts— sharedtruncateUtf16Safeimplementationmonitor.tsandmonitor.inbound-system-event.test.tsBehavior proof
1) Node runtime (
truncateUtf16Safevs rawslice, production helper)Uses the exact test payload from
monitor.inbound-system-event.test.ts:2) Call-site regression through production monitor path
Regression flow (
monitorMattermostProviderwebsocketpostedevent → verbose debug):${"a".repeat(199)}😀tail(len 205 code units).${"a".repeat(199)}— emoji excluded, not split.verboseDebugcalled withpreview="${"a".repeat(199)}"(no lone surrogate).Full touched file still green:
Proof gap: No redacted live Mattermost websocket capture with verbose logging enabled in this environment.