fix(telegram): clip progress text on code-point boundaries to avoid lone surrogates by he-yufeng · Pull Request #96456 · openclaw/openclaw

he-yufeng · 2026-06-24T13:49:16Z

What Problem This Solves

Fixes an issue where users sending a Telegram progress update containing an emoji (or any astral Unicode character) that straddles the 300-code-unit progress-text limit would cause the bot to emit an invalid character in the Telegram Bot API payload.

clipProgressMarkdownText clipped with text.slice(0, 299). JavaScript .slice operates on UTF-16 code units, so a surrogate pair exactly at that boundary is split in two — the high surrogate \uD83D is kept but the matching low surrogate is cut off. A lone high surrogate is not valid UTF-8/UTF-16 and Telegram's API rejects messages containing one.

This affects every progress line rendered via formatProgressAsMarkdownCode, renderTelegramProgressStringLine, and the detail field of renderTelegramProgressLine.

Why This Change Was Made

This is the same class of bug fixed in the Slack plugin by #96382. That PR introduced sliceUtf16Safe in openclaw/plugin-sdk/text-utility-runtime, which steps back one code unit when the cut falls on a high surrogate, ensuring the pair is dropped whole rather than split.

The Telegram progress clipper was not updated alongside the Slack fix.

User Impact

Users whose Telegram bots display progress text (e.g. tool-call output, reasoning steps) could see rejected Bot API calls or garbled ? characters when an emoji appears near the 300-char cutoff.

Evidence

Node.js reproduction of the broken behavior (before fix):

const base = 'a'.repeat(298);            // 298 'a's
const input = base + '😀tail';           // total: 302 code units
const broken = input.slice(0, 299);      // → base + '\uD83D' (lone high surrogate)
/[\uD800-\uDBFF](?![\uDC00-\uDFFF])/.test(broken + '…')
// → true  ← invalid character in Telegram payload

With sliceUtf16Safe:

sliceUtf16Safe(input, 0, 299)            // steps back at position 298 (high surrogate)
// → base  (emoji dropped whole)
// → base + '…'  — valid, no lone surrogate

Five vitest cases in extensions/telegram/src/truncate.test.ts cover: the straddle (lone-surrogate) case, an emoji that fits entirely before the cut, an unchanged short string, trailing-whitespace trimming, and plain ASCII at the exact limit.

CI is the authoritative test runner; the test logic was verified with Node.js manually.

clawsweeper · 2026-06-24T14:10:43Z

Codex review: needs maintainer review before merge. Reviewed June 24, 2026, 10:14 AM ET / 14:14 UTC.

Summary
The PR moves Telegram progress text clipping into a new helper using sliceUtf16Safe, updates the three progress-rendering call sites, and adds UTF-16 boundary tests.

PR surface: Source +13, Tests +48. Total +61 across 3 files.

Reproducibility: yes. A focused Node probe against current-main clipping logic reproduces a dangling high surrogate when an emoji straddles the 299/300 UTF-16 code-unit cut.

Review metrics: none identified.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Mantis proof suggestion
A native Telegram Desktop recording would add useful maintainer proof for the visible progress update path that unit tests cannot show. A maintainer can ask Mantis to capture proof by posting this exact PR comment:

@openclaw-mantis telegram desktop proof: verify a progress draft with an emoji at the 300-code-unit boundary sends without a replacement character or Bot API rejection.

Risk before merge

[P1] The PR body proves the boundary behavior with Node output, but it does not attach a live Telegram Desktop or Bot API transcript for the visible progress draft path.

Maintainer options:

Decide the mitigation before merge
Land the focused helper reuse after required CI and maintainer review, optionally collecting Telegram Desktop proof for the visible progress draft path.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

No automated repair is needed; this PR is already a narrow implementation candidate with no blocking code finding.

Security
Cleared: The diff only changes Telegram string clipping and adds tests; it does not add dependencies, workflows, lockfiles, secret handling, or executable supply-chain surface.

Review details

Best possible solution:

Land the focused helper reuse after required CI and maintainer review, optionally collecting Telegram Desktop proof for the visible progress draft path.

Do we have a high-confidence way to reproduce the issue?

Yes. A focused Node probe against current-main clipping logic reproduces a dangling high surrogate when an emoji straddles the 299/300 UTF-16 code-unit cut.

Is this the best way to solve the issue?

Yes. Reusing the existing sliceUtf16Safe SDK helper is narrower and lower-drift than adding Telegram-local surrogate-boundary logic or changing the broader progress compositor.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against acc2a0ee7297.

Label changes

Label changes:

add P2: This is a focused Telegram payload correctness bug with limited blast radius.
add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied Node before/after output for the exact UTF-16 boundary behavior, which is enough for this pure clipping helper even though live Telegram proof would be stronger.
add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied Node before/after output for the exact UTF-16 boundary behavior, which is enough for this pure clipping helper even though live Telegram proof would be stronger.
add mantis: telegram-visible-proof: Mantis should capture Telegram visible proof. This touches visible Telegram progress draft rendering, and a short Telegram Desktop proof could show the boundary emoji no longer corrupts or rejects the update.

Label justifications:

P2: This is a focused Telegram payload correctness bug with limited blast radius.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied Node before/after output for the exact UTF-16 boundary behavior, which is enough for this pure clipping helper even though live Telegram proof would be stronger.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied Node before/after output for the exact UTF-16 boundary behavior, which is enough for this pure clipping helper even though live Telegram proof would be stronger.
mantis: telegram-visible-proof: Mantis should capture Telegram visible proof. This touches visible Telegram progress draft rendering, and a short Telegram Desktop proof could show the boundary emoji no longer corrupts or rejects the update.

Evidence reviewed

PR surface:

Source +13, Tests +48. Total +61 across 3 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	2	24	11	+13
Tests	1	48	0	+48
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	3	72	11	+61

What I checked:

Current-main bug path: Current main's Telegram progress clipper uses text.slice(0, MAX_PROGRESS_MARKDOWN_TEXT_CHARS - 1) before appending an ellipsis, so a surrogate pair straddling that cut can leave a dangling high surrogate. (extensions/telegram/src/bot-message-dispatch.ts:370, acc2a0ee7297)
Focused reproduction: A read-only Node probe of the current-main clipping logic produced brokenHasDangling: true; the same probe using sliceUtf16Safe produced fixedHasDangling: false. (extensions/telegram/src/bot-message-dispatch.ts:374, acc2a0ee7297)
Patch scope: The PR replaces the three Telegram progress clipping call sites with clipTelegramProgressText and adds tests for the straddling emoji, fitting emoji, short text, trailing whitespace, and exact-limit paths. (extensions/telegram/src/truncate.ts:1, acb9de52736f)
SDK helper contract: sliceUtf16Safe adjusts both slice edges to avoid dangling surrogate halves, and openclaw/plugin-sdk/text-utility-runtime re-exports it through a package export allowed for extension production code. (src/utils.ts:80, acc2a0ee7297)
Progress rendering context: Telegram progress drafts pass structured lines into the channel compositor and render them through the progress line formatter before draft updates, so the changed helper sits on the actual visible progress path. (src/channels/progress-draft-compositor.ts:67, acc2a0ee7297)
Sibling fix context: The merged Slack PR fixed the same UTF-16 boundary class in a sibling channel by using sliceUtf16Safe, but it did not touch the Telegram progress clipper. (extensions/slack/src/truncate.ts:4, 1069c60e1e25)

Likely related people:

vincentkoc: Git blame and live PR metadata tie the current Telegram progress clipping helper and render path to the merged commit for docs(skills): add OpenClaw CI limits runbook #96302. (role: introduced current behavior and recent area contributor; confidence: medium; commits: f9cf00c3512a; files: extensions/telegram/src/bot-message-dispatch.ts, src/utils.ts, src/plugin-sdk/text-utility-runtime.ts)
ly-wang19: Authored the merged Slack PR that fixed the same UTF-16 truncation class by adopting sliceUtf16Safe in a sibling channel helper. (role: adjacent sibling fix contributor; confidence: medium; commits: 1069c60e1e25; files: extensions/slack/src/truncate.ts, extensions/slack/src/truncate.test.ts)
张贵萍0668001030: Recent history shows adjacent Telegram dispatch test maintenance, though not this exact clipping helper. (role: recent adjacent test contributor; confidence: low; commits: 8150b76b6fe7; files: extensions/telegram/src/bot-message-dispatch.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

…one surrogates clipProgressMarkdownText used text.slice(0, MAX - 1), which splits UTF-16 surrogate pairs. When an astral character (e.g. an emoji) straddles the 299/300-code-unit boundary, the slice keeps only the high surrogate half (\uD83D), producing an invalid character that the Telegram Bot API rejects. Extract the helper to extensions/telegram/src/truncate.ts as clipTelegramProgressText and switch to sliceUtf16Safe (from openclaw/plugin-sdk/text-utility-runtime), mirroring the fix applied to the equivalent Slack helper in openclaw#96382. Adds truncate.test.ts with five vitest cases covering the straddle, fits-before-cut, unchanged-short, trimEnd, and ASCII-exact-limit paths.

vincentkoc · 2026-06-24T15:28:06Z

Merged via squash.

Prepared head SHA: 765d6c08acb39e7c5cb8921a96b840efc662d14b
Merge commit: fa2379dbc883b49104c864d8b41caa6804ab83ec

Thanks @he-yufeng!

@vincentkoc

…one surrogates (openclaw#96456) Merged via squash. Prepared head SHA: 765d6c0 Co-authored-by: he-yufeng <[email protected]> Co-authored-by: vincentkoc <[email protected]> Reviewed-by: @vincentkoc

@vincentkoc

…one surrogates (openclaw#96456) Merged via squash. Prepared head SHA: 765d6c0 Co-authored-by: he-yufeng <[email protected]> Co-authored-by: vincentkoc <[email protected]> Reviewed-by: @vincentkoc

…void lone surrogates The Mattermost inbound handler in extensions/mattermost/src/mattermost/monitor.ts truncates the message body preview with bodyText.slice(0, 200), which can split a UTF-16 surrogate pair when an emoji (e.g. 🎉, 🦞) or other astral character lands at the 200-char boundary. The resulting lone surrogate half leaks into the verbose log preview and could propagate to downstream consumers that reject ill-formed strings. Replace the raw .slice(0, 200) with the shared truncateUtf16Safe helper (from openclaw/plugin-sdk/text-utility-runtime), which backs up one code unit when the cut would land inside a surrogate pair. This matches the same pattern already applied to the Slack, Discord, Telegram, IRC, Line, MS Teams, and Feishu channel previews (openclaw#96456, openclaw#96569, openclaw#96572, openclaw#96577, openclaw#96578, openclaw#97462, openclaw#97472, openclaw#98994). The preview is only used for the verbose inbound log line so the behavioral change is cosmetic: when the 200th code unit would split an emoji, the preview now includes one fewer code unit rather than emitting a broken surrogate.

openclaw-barnacle Bot added channel: telegram Channel integration: telegram size: S labels Jun 24, 2026

clawsweeper Bot had a problem deploying to qa-live-shared June 24, 2026 14:17 Failure

vincentkoc force-pushed the fix/telegram-progress-clip-surrogate-safe branch from acb9de5 to 765d6c0 Compare June 24, 2026 15:17

vincentkoc merged commit fa2379d into openclaw:main Jun 24, 2026
86 checks passed

mushuiyu886 mentioned this pull request Jun 24, 2026

fix(telegram): keep draft preview chunks surrogate-safe #96504

Merged

github-actions Bot mentioned this pull request Jun 24, 2026

📡 Upstream Digest — 2026-06-24 17:48 UTC curtismercier/openclaw-mods#1145

Open

clawsweeper Bot mentioned this pull request Jun 24, 2026

refactor: migrate bundled session metadata reads #96527

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(telegram): clip progress text on code-point boundaries to avoid lone surrogates#96456

fix(telegram): clip progress text on code-point boundaries to avoid lone surrogates#96456
vincentkoc merged 1 commit into
openclaw:mainfrom
he-yufeng:fix/telegram-progress-clip-surrogate-safe

he-yufeng commented Jun 24, 2026

Uh oh!

clawsweeper Bot commented Jun 24, 2026 •

edited

Loading

Uh oh!

Uh oh!

vincentkoc commented Jun 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

he-yufeng commented Jun 24, 2026

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

vincentkoc commented Jun 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jun 24, 2026 •

edited

Loading