Skip to content

fix(line): truncate outbound altText, location, menu, and code fields on code point boundaries#98994

Merged
vincentkoc merged 3 commits into
openclaw:mainfrom
LEXES7:fix/line-field-surrogate-truncation
Jul 4, 2026
Merged

fix(line): truncate outbound altText, location, menu, and code fields on code point boundaries#98994
vincentkoc merged 3 commits into
openclaw:mainfrom
LEXES7:fix/line-field-surrogate-truncation

Conversation

@LEXES7

@LEXES7 LEXES7 commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes an issue where several LINE outbound fields can be sent to the LINE API with a broken character (a lone UTF-16 surrogate) when an emoji straddles the field's hard length cap. Sixteen call sites across six files still capped user-facing strings with a raw .slice(0, N):

  • flex message altText (400-char cap) — outbound.ts, send.ts, auto-reply-delivery.ts, card-command.ts (×4)
  • location title/address (100) — outbound.ts, send.ts
  • rich-menu name (300) and chatBarText (14) — rich-menu.ts
  • code-block display text (2000) — markdown-to-line.ts

A raw slice cuts an emoji's surrogate pair in half, leaving a dangling surrogate in the payload (rendered as a broken glyph; serializes as an invalid lone \uD83D, which the LINE API can reject). The 14-char chatBarText cap is especially easy to hit with emoji menu labels.

Why This Change Was Made

Each cap now goes through truncateUtf16Safe — the same helper this plugin's own merged fixes already use for the sibling fields: action label/data (actions.ts, "fix(line): truncate action fields on code-point boundaries") and template title/text/altText (#97428). This PR brings the remaining outbound fields in line with those precedents; it is a mechanical 1:1 replacement of .slice(0, N) with truncateUtf16Safe(x, N) using the same limits — no behavior change for any string that fits or ends on a complete character.

User Impact

LINE users whose flex alt text, location titles/addresses, rich-menu labels, or shared code blocks contain an emoji near a field's length cap get clean, valid messages instead of a broken glyph or a rejected API call. No configuration changes.

Evidence

Real terminal proof (standalone, non-test)

Drove the real caps (and the real convertCodeBlockToFlexBubble) from a standalone script, inspecting trailing UTF-16 code units:

[1] flex altText @ LINE's 400-char cap (emoji straddles index 400)
  BEFORE raw slice : len=400 tail=0x0061 0xd83d loneSurrogate=true
  AFTER  this PR   : len=399 tail=0x0061 0x0061 loneSurrogate=false
[2] rich-menu chatBarText @ 14-char cap (emoji straddles index 14)
  BEFORE raw slice : "Menu options \ud83d" tail=0x0020 0xd83d loneSurrogate=true
  AFTER  this PR   : "Menu options " tail=0x0073 0x0020 loneSurrogate=false
[3] real convertCodeBlockToFlexBubble @ 2000-char code cap
  flex code text: len=2003 endsWith "\n...": true loneSurrogate=false

RESULT: PASS — raw caps were broken; fixed fields are valid within LINE limits

0xd83d is the high-surrogate half of 😀 (U+1F600); on main it is left dangling at the end of the field.

Regression tests (fail on unpatched code, pass here)

Added two focused tests driving the real production paths:

  • markdown-to-line.test.tsdoes not split a surrogate pair at the truncation boundary: code whose emoji straddles index 2000 through the real convertCodeBlockToFlexBubble.
  • auto-reply-delivery.test.tstruncates flex altText on a surrogate boundary: altText whose emoji straddles index 400 through the real deliverLineAutoReply, asserting the altText handed to the LINE message builder is ≤400 with no lone surrogate.

On unpatched main both fail (expected true to be false — a lone surrogate is present); with this PR both pass.

Touched-module suites all green: markdown-to-line (25), auto-reply-delivery (6), rich-menu + channel.sendPayload + reply-payload-transform (43), message-cards + outbound-media (51). oxlint and oxfmt --check clean on all eight files.

AI-assisted (Claude Code); change reviewed and understood by the author.

@openclaw-barnacle openclaw-barnacle Bot added channel: line Channel integration: line size: S labels Jul 2, 2026
@clawsweeper

clawsweeper Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 4, 2026, 2:45 AM ET / 06:45 UTC.

Summary
The PR replaces remaining LINE outbound raw string caps for flex altText, location, rich-menu text, card altText, and code-block display text with surrogate-safe or grapheme-aware truncation and adds regression tests.

PR surface: Source +26, Tests +107. Total +133 across 10 files.

Reproducibility: yes. source-level: current main still hard-caps outbound LINE strings with raw .slice(0, N), and an emoji straddling the cap can leave a lone UTF-16 surrogate; the PR body includes terminal before/after output for real caps.

Review metrics: 1 noteworthy metric.

  • Targeted raw hard caps: 0 retained on PR head. The earlier review blocker was a same-category raw cap, so confirming zero targeted hard-cap string slices shows the stated LINE outbound field class is now covered.

Root-cause cluster
Relationship: canonical
Canonical: #98994
Summary: This PR is the current canonical fix for the remaining LINE outbound surrogate-boundary truncation surfaces; merged PR #97428 handled an adjacent template-message surface.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge

  • No automated repair is needed; the prior receipt-card miss is fixed and remaining validation is ordinary maintainer review.

Security
Cleared: The diff only changes LINE plugin string truncation and tests, with no dependency, workflow, secret, install, or code-execution surface change.

Review details

Best possible solution:

Land this focused LINE outbound truncation fix after normal maintainer and CI gates.

Do we have a high-confidence way to reproduce the issue?

Yes, source-level: current main still hard-caps outbound LINE strings with raw .slice(0, N), and an emoji straddling the cap can leave a lone UTF-16 surrogate; the PR body includes terminal before/after output for real caps.

Is this the best way to solve the issue?

Yes, reusing truncateUtf16Safe for UTF-16 field caps and grapheme segmentation for rich-menu count semantics is a narrow fix that avoids new config or API surface.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against cafbd745c1b1.

Label changes

Label changes:

  • add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster. Replaced prior rating: 🦐 gold shrimp.
  • remove rating: 🦐 gold shrimp: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

  • P2: This is a focused LINE channel payload correctness bug with limited blast radius and no evidence of security, data-loss, or core availability impact.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster. Replaced prior rating: 🦐 gold shrimp.
  • status: 🔁 re-review loop: A fresh ClawSweeper review was explicitly requested after the latest review. Sufficient (terminal): The PR body includes terminal before/after output exercising real LINE caps and convertCodeBlockToFlexBubble, and the current head adds focused receipt-card coverage after the prior review.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal before/after output exercising real LINE caps and convertCodeBlockToFlexBubble, and the current head adds focused receipt-card coverage after the prior review.
Evidence reviewed

PR surface:

Source +26, Tests +107. Total +133 across 10 files.

View PR surface stats
Area Files Added Removed Net
Source 6 44 18 +26
Tests 4 109 2 +107
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 10 153 20 +133

What I checked:

  • Repository policy read: Root AGENTS.md was supplied in full and the scoped extensions policy was read; the review applied bundled-plugin boundary and changed-surface guidance. (AGENTS.md:1, cafbd745c1b1)
  • Current main still has raw hard-cap string slices: Current main has 16 targeted .slice(0, N) hard caps across LINE outbound altText, location, rich-menu, card, and code-block surfaces. (cafbd745c1b1)
  • PR head removes the targeted raw caps: The same targeted search returns no matches on the PR head, including the receipt-card path flagged by the previous review. (8b8b72c16514)
  • Receipt-card blocker fixed: The /card receipt flex altText now uses truncateUtf16Safe(..., 400), addressing the prior same-category raw slice finding. (extensions/line/src/card-command.ts:264, 8b8b72c16514)
  • Outbound send paths use the shared helper: Location title/address and push flex altText now route through truncateUtf16Safe at the existing LINE length caps. (extensions/line/src/send.ts:166, 8b8b72c16514)
  • Auto-reply flex paths use the shared helper: Direct LINE flexMessage altText and processed markdown flex altText are both truncated with truncateUtf16Safe(..., 400). (extensions/line/src/auto-reply-delivery.ts:107, 8b8b72c16514)

Likely related people:

  • plum-dawg: Introduced the original LINE plugin surface that includes the affected send, rich-menu, markdown, and card modules. (role: original feature contributor; confidence: medium; commits: c96ffa7186a4; files: extensions/line/src/send.ts, extensions/line/src/rich-menu.ts, extensions/line/src/markdown-to-line.ts)
  • vincentkoc: Moved channel/plugin ownership into extensions/line, merged the adjacent template truncation fix, and authored the PR-head rich-menu grapheme commit. (role: recent area contributor and merger; confidence: high; commits: 213198123042, 4b36cf451c74, 8b8b72c16514; files: extensions/line/src/rich-menu.ts, extensions/line/src/template-messages.ts, extensions/line/src/send.ts)
  • ly-wang19: Authored the merged LINE template and action truncation fixes that established the same surrogate-safe invariant on neighboring LINE surfaces. (role: adjacent invariant contributor; confidence: high; commits: 4b36cf451c74, eff68d2c7714; files: extensions/line/src/template-messages.ts, extensions/line/src/actions.ts)
  • Bartok: Authored the commit that introduced the dependency-free UTF-16 safe slicing helper reused by this PR. (role: shared helper introducer; confidence: medium; commits: e09b9dfc1ba9; files: src/shared/utf16-slice.ts, src/plugin-sdk/text-utility-runtime.ts)
  • Peter Steinberger: Recent LINE/plugin SDK history shows earlier refactors around the same helper and outbound modules. (role: prior LINE refactor contributor; confidence: medium; commits: 493d05b1c8c9, 2766c27b2aa6; files: extensions/line/src/card-command.ts, extensions/line/src/markdown-to-line.ts, extensions/line/src/send.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. labels Jul 2, 2026
@LEXES7

LEXES7 commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Addressed the retained raw cap in 8fd9e7d: the /card receipt flex altText now goes through truncateUtf16Safe(…, 400) like the other card types, closing the last raw .slice string cap in the plugin (verified with a multi-line-aware scan across extensions/line/src — zero remaining).

Added the requested regression in message-cards.test.ts (/card receipt altText truncates on a surrogate boundary): drives the real /card receipt handler with an item whose emoji straddles index 400 and asserts the flex altText is ≤400 with no lone surrogate. It fails on the previous head (lone high surrogate present) and passes now. message-cards suite: 38/38; oxlint and oxfmt --check clean.

@clawsweeper

clawsweeper Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 🔁 re-review loop A fresh ClawSweeper review was explicitly requested after the latest review. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 2, 2026
@vincentkoc vincentkoc self-assigned this Jul 4, 2026
@vincentkoc
vincentkoc force-pushed the fix/line-field-surrogate-truncation branch from 8fd9e7d to 8b8b72c Compare July 4, 2026 06:34
@vincentkoc

Copy link
Copy Markdown
Member

Land-ready review complete on exact head 8b8b72c165140e843c46096fe6012c81b4eb4245.

Maintainer repair:

  • kept ordinary LINE payload fields on the shared surrogate-safe UTF-16 truncation path
  • changed rich-menu name and chatBarText to LINE's documented grapheme-cluster counting, with a request-level regression covering emoji and ZWJ sequences

Proof:

Known proof gap: no live LINE credential call. The touched boundary is pure payload serialization and is covered by mocked SDK request assertions, the pinned SDK forwarding implementation, and the current LINE character-counting contract.

@vincentkoc
vincentkoc merged commit 02b529a into openclaw:main Jul 4, 2026
89 checks passed
@vincentkoc

Copy link
Copy Markdown
Member

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 4, 2026
… on code point boundaries (openclaw#98994)

* fix(line): truncate outbound altText, location, menu, and code fields on code-point boundaries

* fix(line): use safe truncation for receipt card altText

* fix(line): count rich menu limits by grapheme

---------

Co-authored-by: Vincent Koc <[email protected]>
hugenshen added a commit to hugenshen/openclaw that referenced this pull request Jul 7, 2026
…void lone surrogates

The Mattermost inbound handler in extensions/mattermost/src/mattermost/monitor.ts
truncates the message body preview with bodyText.slice(0, 200), which can split a
UTF-16 surrogate pair when an emoji (e.g. 🎉, 🦞) or other astral character lands at
the 200-char boundary. The resulting lone surrogate half leaks into the verbose log
preview and could propagate to downstream consumers that reject ill-formed strings.

Replace the raw .slice(0, 200) with the shared truncateUtf16Safe helper (from
openclaw/plugin-sdk/text-utility-runtime), which backs up one code unit when the cut
would land inside a surrogate pair. This matches the same pattern already applied to
the Slack, Discord, Telegram, IRC, Line, MS Teams, and Feishu channel previews
(openclaw#96456, openclaw#96569, openclaw#96572, openclaw#96577, openclaw#96578, openclaw#97462, openclaw#97472, openclaw#98994).

The preview is only used for the verbose inbound log line so the behavioral change
is cosmetic: when the 200th code unit would split an emoji, the preview now includes
one fewer code unit rather than emitting a broken surrogate.
hugenshen added a commit to hugenshen/openclaw that referenced this pull request Jul 7, 2026
…void lone surrogates

The Mattermost inbound handler in extensions/mattermost/src/mattermost/monitor.ts
truncates the message body preview with bodyText.slice(0, 200), which can split a
UTF-16 surrogate pair when an emoji (e.g. 🎉, 🦞) or other astral character lands at
the 200-char boundary. The resulting lone surrogate half leaks into the verbose log
preview and could propagate to downstream consumers that reject ill-formed strings.

Replace the raw .slice(0, 200) with the shared truncateUtf16Safe helper (from
openclaw/plugin-sdk/text-utility-runtime), which backs up one code unit when the cut
would land inside a surrogate pair. This matches the same pattern already applied to
the Slack, Discord, Telegram, IRC, Line, MS Teams, and Feishu channel previews
(openclaw#96456, openclaw#96569, openclaw#96572, openclaw#96577, openclaw#96578, openclaw#97462, openclaw#97472, openclaw#98994).

The preview is only used for the verbose inbound log line so the behavioral change
is cosmetic: when the 200th code unit would split an emoji, the preview now includes
one fewer code unit rather than emitting a broken surrogate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: line Channel integration: line P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. size: S status: 🔁 re-review loop A fresh ClawSweeper review was explicitly requested after the latest review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants