fix(matrix): truncate thread starter body on code-point boundaries by Bartok9 · Pull Request #96961 · openclaw/openclaw

Bartok9 · 2026-06-26T06:08:27Z

What Problem This Solves

Matrix thread-starter previews truncate long bodies by UTF-16 code-unit offset. If the boundary lands inside an astral character such as an emoji, the preview contains a lone surrogate and renders mojibake in the agent's thread context.

Why This Change Was Made

The truncation site is the canonical producer of the bounded thread-starter body. This change reuses the existing public sliceUtf16Safe helper so the cut backs up to a valid surrogate boundary while preserving the existing 500-code-unit limit and ... suffix.

User Impact

Long Matrix thread starters no longer split emoji or other astral characters at the truncation boundary. Short bodies and BMP-only text retain their existing output.

Evidence

Current main reproduces the defect with 496 a characters followed by 😀: the raw slice leaves a lone high surrogate before the ellipsis.
The added regression test fails against the pre-fix implementation and passes with the fix.
Focused Matrix suite on this branch: 6/6 tests passed.
git diff --check passed.
Fresh maintainer autoreview reported no accepted/actionable findings.

Provenance and Credit

This PR salvages the original fix from #96407 after that PR was auto-closed by the active-PR queue cap. The implementation preserves @ly-wang19's original authorship and is rebased by @Bartok9 onto current main.

truncateThreadStarterBody (reached via summarizeMatrixThreadStarterEvent) truncated the thread-root body with a raw `value.slice(0, 497)`. When the 500-char boundary falls in the middle of an astral character (emoji etc.), the raw slice cuts the surrogate pair in half and leaves a lone high surrogate before the ellipsis, rendering as a broken/replacement glyph. Use the SDK's surrogate-safe `sliceUtf16Safe` so truncation stops on a code-point boundary, dropping the half character entirely. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

clawsweeper · 2026-06-26T06:19:02Z

Codex review: needs real behavior proof before merge. Reviewed June 26, 2026, 7:10 PM ET / 23:10 UTC.

Summary
The current PR head applies Matrix surrogate-safe thread-starter truncation but also includes a large unrelated commit touching CI, changelog, ClawHub trust install flows, memory-core, Telegram ingress, providers, UI i18n, docs, scripts, and gateway protocol.

Reproducibility: yes. Current main still calls raw slice(0, 497) in truncateThreadStarterBody, and the reported 496 * "a" + emoji + "bcd" input is enough from source inspection to cut the emoji's high surrogate before the ellipsis.

Review metrics: 3 noteworthy metrics.

Post-proof scope drift: 1 new head commit expands the live PR from the described 2 Matrix files to 243 changed files. The contributor proof and prior review covered the Matrix fix, but the current head now includes many unrelated owner-sensitive surfaces.
CI concurrency changes: 6 max-parallel changes: five 8->12 and one 16->24. Runner-capacity changes are automation-sensitive and need their own CI-capacity review instead of landing under a Matrix bug-fix title.
Security/trust surface added: 1 new gateway-protocol trust-error module plus CLI/UI acknowledgement plumbing. ClawHub install trust behavior affects package/plugin security decisions and needs proof independent of the Matrix truncation fix.

Stored data model
Persistent data-model change detected: migration/backfill/repair: docs/cli/update.md, persistent cache schema: extensions/memory-core/src/memory/qmd-manager.ts, persistent cache schema: extensions/memory-core/src/memory/qmd-runtime-cache.test.ts, persistent cache schema: extensions/memory-core/src/memory/qmd-runtime-cache.ts, persistent cache schema: ui/src/i18n/.i18n/uk.tm.jsonl, persistent cache schema: ui/src/i18n/.i18n/vi.tm.jsonl, and 28 more. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: canonical
Canonical: #96961
Summary: This PR is the active replacement for the same Matrix surrogate-boundary fix in the closed predecessor PR, while the related agents PR addresses a separate truncation site with the same UTF-16 class of bug.

Members:

superseded: fix(matrix): truncate thread starter body on code-point boundaries #96407 - Closed unmerged by the active-PR queue cap but contains the same Matrix thread-starter code-point-boundary fix and regression test intent.
adjacent_distinct: fix(agents): truncate tool-display detail on code-point boundaries #96958 - Addresses UTF-16-safe truncation in agent tool-display detail, not Matrix thread-starter summaries.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦪 silver shellfish
Patch quality: 🧂 unranked krab
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Drop b3270d052cfa1650c959deeca8f874fbdc0bcf38 or replace the PR with the Matrix-only diff.
Rerun focused Matrix thread-context validation after narrowing.
[P1] If any non-Matrix changes remain, update the PR body and add real behavior proof for those surfaces.

Proof guidance:

[P1] Needs stronger real behavior proof before merge: The terminal before/after proof is sufficient for the Matrix two-file fix, but it does not prove the unrelated broad changes now present on the current head; after narrowing, the existing proof would be enough. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

Merging the current head would ship unrelated CI capacity changes, ClawHub trust/security install behavior, Telegram ingress/message delivery changes, memory-core runtime cache changes, gateway protocol additions, generated UI locales, provider changes, docs, scripts, and release-note edits under a Matrix truncation PR.
[P1] The Matrix proof is stale for the current head: the posted terminal proof covers the two-file Matrix fix, not the later broad changes now present in b3270d052cfa1650c959deeca8f874fbdc0bcf38.
[P1] The broad commit crosses owner-sensitive automation, security-boundary, message-delivery, plugin SDK/protocol, and generated-output surfaces without the focused owner review expected for those areas.

Maintainer options:

Reset to the Matrix-only fix (recommended)
Drop b3270d052cfa1650c959deeca8f874fbdc0bcf38 so this PR contains only the Matrix thread-context source and regression test changes, preserving the original fix authorship.
Split the broad work into owned PRs
If the later changes are intentional, move them into separate PRs for CI capacity, ClawHub trust/security, Telegram delivery, memory-core, UI i18n, and docs so each can receive matching proof and owner review.
Pause or replace this branch
If the current branch cannot be safely narrowed, keep this PR paused or open a new narrow replacement that carries only the Matrix bug fix.

Copy recommended automerge instruction

@clawsweeper automerge

Special instructions:
Drop commit b3270d052cfa1650c959deeca8f874fbdc0bcf38 from the branch, preserve f446ce5fe3ca63aa3fe68f5d8e7646e322c65bfa and its authorship, and leave only the Matrix thread-context source/test changes; do not carry CI, changelog, ClawHub trust, Telegram, memory-core, UI i18n, provider, docs, or script changes.

Next step before merge

[P1] A repair worker can mechanically remove the broad second commit or prepare a narrow replacement while preserving the valid Matrix source/test fix.

Security
Needs attention: The current head mixes unrelated ClawHub trust/security and CI workflow changes into a Matrix bug-fix PR, so the security and supply-chain surface is not cleared.

Review findings

[P1] Narrow this branch back to the Matrix fix — .github/workflows/ci.yml:861

Review details

Best possible solution:

Land a Matrix-only PR that imports sliceUtf16Safe and keeps the regression test; drop or split every broad b3270d052cfa1650c959deeca8f874fbdc0bcf38 change into separate owner-reviewed PRs with their own proof.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main still calls raw slice(0, 497) in truncateThreadStarterBody, and the reported 496 * "a" + emoji + "bcd" input is enough from source inspection to cut the emoji's high surrogate before the ellipsis.

Is this the best way to solve the issue?

No for the current head. The two-file Matrix fix is the right layer, but the current branch must be narrowed because the broad second commit is unrelated and crosses automation, security, message-delivery, protocol, docs, and generated-output boundaries.

Full review comments:

[P1] Narrow this branch back to the Matrix fix — .github/workflows/ci.yml:861
The current head changes CI concurrency here and now includes 240+ unrelated files, including changelog, ClawHub trust protocol/UI/CLI, memory-core QMD caches, Telegram ingress, generated i18n, providers, docs, and scripts, while the PR body and proof cover only Matrix thread-starter truncation. Please drop the broad second commit or split it into separate reviewed PRs before this Matrix fix can land.
Confidence: 0.96

Overall correctness: patch is incorrect
Overall confidence: 0.94

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a82902c725b2.

Label changes

Label changes:

add merge-risk: 🚨 message-delivery: The current head includes Telegram ingress and polling-session delivery changes that could affect message processing if merged with this Matrix fix.
add merge-risk: 🚨 security-boundary: The diff adds ClawHub trust error details and acknowledgement flows that affect plugin/skill install security decisions but are unrelated to the Matrix proof.
add merge-risk: 🚨 automation: The diff changes CI matrix concurrency and related CI-limit guidance, which can affect repository automation capacity after merge.
add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The terminal before/after proof is sufficient for the Matrix two-file fix, but it does not prove the unrelated broad changes now present on the current head; after narrowing, the existing proof would be enough. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
remove proof: sufficient: Current real behavior proof status is insufficient, not sufficient.
remove rating: 🐚 platinum hermit: Current PR rating is rating: 🧂 unranked krab, so this older rating label is no longer current.
remove status: 👀 ready for maintainer look: Current PR status label is status: 📣 needs proof.

Label justifications:

P2: The central Matrix mojibake bug is a normal bounded channel bug, but the current PR head needs cleanup before it can merge safely.
merge-risk: 🚨 automation: The diff changes CI matrix concurrency and related CI-limit guidance, which can affect repository automation capacity after merge.
merge-risk: 🚨 security-boundary: The diff adds ClawHub trust error details and acknowledgement flows that affect plugin/skill install security decisions but are unrelated to the Matrix proof.
merge-risk: 🚨 message-delivery: The current head includes Telegram ingress and polling-session delivery changes that could affect message processing if merged with this Matrix fix.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The terminal before/after proof is sufficient for the Matrix two-file fix, but it does not prove the unrelated broad changes now present on the current head; after narrowing, the existing proof would be enough. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

Security concerns:

[medium] Split ClawHub trust changes out of this Matrix fix — packages/gateway-protocol/src/clawhub-trust-error-details.ts:1
packages/gateway-protocol/src/clawhub-trust-error-details.ts adds structured trust error handling for ClawHub install flows, which affects plugin/skill install security decisions but is not described or proven by the Matrix truncation PR.
Confidence: 0.9
[medium] Review CI concurrency changes separately — .github/workflows/ci.yml:861
The workflow raises several matrix max-parallel caps, which is an automation capacity and supply-chain execution surface that needs its own CI-limit proof rather than landing under this Matrix fix.
Confidence: 0.88

Acceptance criteria:

[P1] node scripts/run-vitest.mjs extensions/matrix/src/matrix/monitor/thread-context.test.ts.
[P1] git diff --check.

What I checked:

Current main still has the Matrix bug: On current main, truncateThreadStarterBody still truncates with raw value.slice(0, MAX_THREAD_STARTER_BODY_LENGTH - 3), which can split a surrogate pair at the 497-code-unit boundary. (extensions/matrix/src/matrix/monitor/thread-context.ts:20, a82902c725b2)
Existing helper is the right low-level primitive: sliceUtf16Safe already backs off at dangling surrogate boundaries, and the plugin SDK text utility runtime re-exports it for bundled plugins. (src/utils.ts:80, a82902c725b2)
PR contains the intended Matrix fix: The PR patch imports sliceUtf16Safe and adds a regression test asserting "a".repeat(496) + "..." with no orphaned surrogate for the emoji boundary case. (extensions/matrix/src/matrix/monitor/thread-context.ts:18, b3270d052cfa)
Live PR head is no longer narrow: Live GitHub metadata shows head b3270d052cfa1650c959deeca8f874fbdc0bcf38 with 243 changed files and +19253/-917, even though the body and proof describe the Matrix two-file fix. (b3270d052cfa)
Unrelated automation changes are in the PR diff: The current head changes CI matrix max-parallel values, including 8 -> 12 and 16 -> 24, which is unrelated to Matrix thread-starter truncation and affects CI capacity behavior. (.github/workflows/ci.yml:861, b3270d052cfa)
Current raw truncation provenance: git blame ties the current Matrix thread-context implementation and raw slice to merged commit db255b1154c11fe8c3c19c62e27e0ff726a07edb, the merge commit for Fix Telegram spooled claim refresh #96962. (extensions/matrix/src/matrix/monitor/thread-context.ts:20, db255b1154c1)

Likely related people:

joshavant: git blame shows the current truncateThreadStarterBody raw slice was introduced on main by merge commit db255b1154c11fe8c3c19c62e27e0ff726a07edb for Fix Telegram spooled claim refresh #96962. (role: introduced current behavior; confidence: high; commits: db255b1154c1; files: extensions/matrix/src/matrix/monitor/thread-context.ts, extensions/matrix/src/matrix/monitor/thread-context.test.ts)
ly-wang19: Authored the closed predecessor Matrix fix in fix(matrix): truncate thread starter body on code-point boundaries #96407 and the first commit carried into this PR. (role: original fix author; confidence: high; commits: 741fe0caa196, f446ce5fe3ca; files: extensions/matrix/src/matrix/monitor/thread-context.ts, extensions/matrix/src/matrix/monitor/thread-context.test.ts)
vincentkoc: Authored the current head commit b3270d052cfa1650c959deeca8f874fbdc0bcf38, which expands this PR from the narrow Matrix fix into the broad 243-file diff, and is assigned on the live PR. (role: recent PR branch committer; confidence: medium; commits: b3270d052cfa; files: .github/workflows/ci.yml, CHANGELOG.md, extensions/matrix/src/matrix/monitor/thread-context.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Bartok9 · 2026-06-26T12:25:05Z

Real behavior proof — before/after

Host: macOS (Darwin 25.4.0, arm64), node v25.5.0, repo source via its own tsx. Driving the patched summarizeMatrixThreadStarterEvent end-to-end with a real m.room.message event. Body: "a"×496 + 😀 (U+1F600, surrogate pair at units 496-497) + "bcd". MAX_THREAD_STARTER_BODY_LENGTH=500, so the cut at 500-3=497 lands mid-pair.

Before (`upstream/main`: `value.slice(0, 497)`)

BEFORE naive slice(0,497): lone surrogate? true  (head ends with charCode d83d)

Old code keeps the emoji's lone high surrogate (U+D83D) before the ellipsis.

After (this branch — `sliceUtf16Safe`)

AFTER summary: "aaaa…(496 a)…..."
  === "a"*496 + "..."  : true     (half-emoji dropped at the cut)
  lone surrogate?      : false
RESULT: PASS

Matches the added thread-context.test.ts assertion (summary === "a"*496 + "...", no [\uD800-\uDFFF] survives). tsc --noEmit clean on changed files; build-artifacts + QA Smoke already green on this PR.

@clawsweeper re-review

clawsweeper · 2026-06-26T12:25:09Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Bartok9 · 2026-06-26T22:53:44Z

Branch narrowed as recommended. Force-pushed back to the single f446ce5 two-file Matrix commit — the stray 235-file c16af5c (ClawHub trust / CLI / UI / docs / etc.) has been dropped entirely. The PR now shows exactly 2 changed files (+22/-1): extensions/matrix/src/matrix/monitor/thread-context.ts and its regression test. Real-behavior before/after proof above still applies unchanged.

@clawsweeper re-review

openclaw-barnacle · 2026-06-26T23:11:41Z

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle · 2026-06-26T23:12:07Z

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

vincentkoc · 2026-06-26T23:28:43Z

Superseded by the clean replacement PR #97121, now merged to main as 289865b39226c685af382a81da55c38ce3df9049.

The landed diff preserves the original Matrix fix and contributor credit, touching only extensions/matrix/src/matrix/monitor/thread-context.ts and its regression test. Exact-main proof passed: node scripts/run-vitest.mjs extensions/matrix/src/matrix/monitor/thread-context.test.ts (6/6), and git diff --check was clean.

openclaw-barnacle Bot added channel: matrix Channel integration: matrix size: XS triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jun 26, 2026

Bartok9 mentioned this pull request Jun 26, 2026

fix(agents): truncate tool-display detail on code-point boundaries #96958

Merged

vincentkoc self-assigned this Jun 26, 2026

openclaw-barnacle Bot added size: XS and removed size: XL labels Jun 26, 2026

fix(matrix): truncate thread starter body on code-point boundaries

b3270d0

Bartok9 mentioned this pull request Jun 26, 2026

fix(matrix): truncate thread starter body on code-point boundaries #97121

Merged

clawsweeper Bot mentioned this pull request Jun 29, 2026

fix(matrix): truncate thread starter body on UTF-16 boundary #97596

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(matrix): truncate thread starter body on code-point boundaries#96961

fix(matrix): truncate thread starter body on code-point boundaries#96961
Bartok9 wants to merge 2 commits into
openclaw:mainfrom
Bartok9:salvage/96407-matrix-threadstarter-codepoint

Bartok9 commented Jun 26, 2026 •

edited by vincentkoc

Loading

Uh oh!

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading

Uh oh!

Bartok9 commented Jun 26, 2026

Uh oh!

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading

Uh oh!

Bartok9 commented Jun 26, 2026

Uh oh!

openclaw-barnacle Bot commented Jun 26, 2026

Uh oh!

openclaw-barnacle Bot commented Jun 26, 2026

Uh oh!

vincentkoc commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

Bartok9 commented Jun 26, 2026 • edited by vincentkoc Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Provenance and Credit

Uh oh!

clawsweeper Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Bartok9 commented Jun 26, 2026

Real behavior proof — before/after

Before (upstream/main: value.slice(0, 497))

After (this branch — sliceUtf16Safe)

Uh oh!

clawsweeper Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Bartok9 commented Jun 26, 2026

Uh oh!

openclaw-barnacle Bot commented Jun 26, 2026

Uh oh!

openclaw-barnacle Bot commented Jun 26, 2026

Uh oh!

vincentkoc commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Bartok9 commented Jun 26, 2026 •

edited by vincentkoc

Loading

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading

Before (`upstream/main`: `value.slice(0, 497)`)

After (this branch — `sliceUtf16Safe`)

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading