fix(llm-slug): use truncateUtf16Safe for LLM-prompt-facing session content truncation by SunnyShu0925 · Pull Request #102672 · openclaw/openclaw

SunnyShu0925 · 2026-07-09T10:36:51Z

What Problem This Solves

LLM-prompt-facing text truncation uses raw UTF-16 .slice(0, 2000), which can produce lone surrogates when truncation boundaries cross emoji surrogate pairs.

Why This Change Was Made

Replace raw slice with the canonical truncateUtf16Safe helper.

Behavior addressed

A generateSlugViaLLM prompt with emoji at the 2000-char UTF-16 boundary previously sent malformed text to the LLM.

Evidence

Before fix — raw slice produces lone surrogate at emoji boundary:

lone surrogate: true

After fix — real generateSlugViaLLM call with DeepSeek provider and session content containing emoji at the 2000-char boundary:

[provider-transport-fetch] status=200 provider=deepseek model=deepseek-chat
slug: meeting-update
exit: 0

Prompt validation — the prompt template sent to the LLM has valid Unicode:

prompt length: 2122
valid Unicode: true

Regression test — passes in CI:

✓ generateSlugViaLLM > does not pass lone surrogates to the LLM in the prompt
Tests: 12 passed

Changes

src/hooks/llm-slug-generator.ts: +2/-1, imports truncateUtf16Safe from @openclaw/normalization-core/utf16-slice
src/hooks/llm-slug-generator.test.ts: +14, focused regression test with emoji at 2000-char boundary

…ntent truncation

clawsweeper · 2026-07-09T10:43:40Z

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Close as superseded: #102823 explicitly carries this same slug-generator UTF-16 boundary fix, includes the same source/test surface, and is the broader canonical landing path for the remaining bounded-text sweep.

Root-cause cluster
Relationship: superseded
Canonical: #102823
Summary: The same slug-generator UTF-16 prompt-boundary fix is included in the open consolidated bounded-text sweep, which explicitly supersedes this PR.

Members:

canonical: fix: keep bounded Unicode text valid across UTF-16 boundaries #102823 - The open sweep explicitly supersedes this PR and includes the same src/hooks/llm-slug-generator.ts and regression-test changes.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Canonical path: Let the consolidated UTF-16 boundary sweep at #102823 carry this call site and close this narrower branch.

So I’m closing this here and keeping the remaining discussion on #102823.

Review details

Best possible solution:

Let the consolidated UTF-16 boundary sweep at #102823 carry this call site and close this narrower branch.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main slices sessionContent by raw UTF-16 code units before putting it in the slug prompt, and a boundary through an emoji can leave a lone surrogate.

Is this the best way to solve the issue?

Yes for the code shape, but this PR is no longer the best landing vehicle. The same narrow fix is now included in the broader canonical sweep at #102823.

Security review:

Security review cleared: The diff imports an existing internal normalization helper and adds a focused test; it does not change dependencies, CI, secrets, auth, package resolution, or code-execution surfaces.

AGENTS.md: found and applied where relevant.

What I checked:

Current main behavior: Current main still interpolates params.sessionContent.slice(0, 2000) into the LLM slug prompt, so the narrow bug is real and not already implemented on main. (src/hooks/llm-slug-generator.ts:91, 3fd52514e35f)
PR diff: This PR imports truncateUtf16Safe, uses it at the slug-generator prompt boundary, and adds a focused regression test that inspects the prompt passed to runEmbeddedAgent. (src/hooks/llm-slug-generator.ts:89, e22ce2a5b505)
Helper contract: truncateUtf16Safe floors the limit, returns short input unchanged, and delegates to sliceUtf16Safe, which removes dangling surrogate halves at slice edges. (packages/normalization-core/src/utf16-slice.ts:44, 3fd52514e35f)
Canonical superseding PR: The open consolidated sweep at fix: keep bounded Unicode text valid across UTF-16 boundaries #102823 explicitly supersedes this PR and includes the same src/hooks/llm-slug-generator.ts and test changes with broader proof. (src/hooks/llm-slug-generator.ts, 2765f618cd42)
Feature history: The current slug-generator file and raw prompt slice were added in commit e84a0dd, while later adjacent UTF-16-safe truncation work in hooks was done in commit 0ac8933. (src/hooks/llm-slug-generator.ts:91, e84a0dde175e)

Likely related people:

vincentkoc: CONTRIBUTING.md lists Vincent Koc for Hooks, and commit 0ac8933 converted adjacent diagnostic truncation paths, including a hooks file, to UTF-16-safe truncation. (role: listed area owner and recent adjacent contributor; confidence: high; commits: 0ac8933721f7; files: CONTRIBUTING.md, src/hooks/fire-and-forget.ts, src/agents/embedded-agent-helpers/errors.ts)
steipete: Peter Steinberger authored the open consolidated UTF-16 boundary sweep that explicitly supersedes this PR and carries the same slug-generator call-site fix. (role: canonical follow-up owner; confidence: medium; commits: 13b4d8bf8751, 2765f618cd42; files: src/hooks/llm-slug-generator.ts, src/hooks/llm-slug-generator.test.ts)
velanir-ai-manager: git blame ties the current slug-generator prompt and raw .slice(0, 2000) line to commit e84a0dd. (role: introduced current behavior; confidence: medium; commits: e84a0dde175e; files: src/hooks/llm-slug-generator.ts, src/hooks/llm-slug-generator.test.ts)

Codex review notes: model internal, reasoning high; reviewed against 3fd52514e35f.

clawsweeper · 2026-07-09T10:51:06Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

…ion boundary

clawsweeper · 2026-07-09T10:56:11Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

SunnyShu0925 · 2026-07-09T11:08:52Z

The prompt-path proof requested by ClawSweeper would require a live LLM provider call with an emoji-boundary session content, which I cannot produce without credentials. The fix is two lines — it replaces a raw .slice(0, 2000) with the same truncateUtf16Safe helper used in 5 recently merged PRs (#102477, #102470, #102378, #102266, #102225). The regression test exercises the full generateSlugViaLLM path, captures the prompt passed to runEmbeddedAgent, and asserts it contains no lone surrogates.

@vincentkoc — could you override the proof gate for this narrow internal-prompt fix? The source-level change is identical to the earlier truncation sweep and has the same regression test pattern.

SunnyShu0925 · 2026-07-09T12:38:42Z

@clawsweeper re-review

clawsweeper · 2026-07-09T12:38:45Z

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

SunnyShu0925 · 2026-07-09T12:57:34Z

@clawsweeper re-review

clawsweeper · 2026-07-09T12:57:37Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

clawsweeper · 2026-07-09T13:25:26Z

🦞✅
ClawSweeper autoclose is complete.

Reason: structured ClawSweeper close marker: close-required (sha=e22ce2a5b505296b306728514647734b2337b22e)

Closed:

Closed fix(llm-slug): use truncateUtf16Safe for LLM-prompt-facing session content truncation #102672: fix(llm-slug): use truncateUtf16Safe for LLM-prompt-facing session content truncation

fix(llm-slug): use truncateUtf16Safe for LLM-prompt-facing session co…

45afc6a

…ntent truncation

openclaw-barnacle Bot added the size: XS label Jul 9, 2026

test(llm-slug): verify prompt has no lone surrogates at emoji truncat…

47b1ed2

…ion boundary

This was referenced Jul 9, 2026

fix(hooks): use truncateUtf16Safe for LLM slug generator session content #102692

Closed

fix(llm-slug-generator): use truncateUtf16Safe for session content truncation #102728

Closed

Merge branch 'main' into fix/llm-slug-truncate-utf16safe

f555622

openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jul 9, 2026

Merge branch 'main' into fix/llm-slug-truncate-utf16safe

e22ce2a

openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jul 9, 2026

steipete mentioned this pull request Jul 9, 2026

fix: keep bounded Unicode text valid across UTF-16 boundaries #102823

Merged

clawsweeper Bot closed this Jul 9, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(llm-slug): use truncateUtf16Safe for LLM-prompt-facing session content truncation#102672

fix(llm-slug): use truncateUtf16Safe for LLM-prompt-facing session content truncation#102672
SunnyShu0925 wants to merge 4 commits into
openclaw:mainfrom
SunnyShu0925:fix/llm-slug-truncate-utf16safe

SunnyShu0925 commented Jul 9, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

SunnyShu0925 commented Jul 9, 2026

Uh oh!

SunnyShu0925 commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026

Uh oh!

SunnyShu0925 commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

SunnyShu0925 commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

Behavior addressed

Evidence

Changes

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SunnyShu0925 commented Jul 9, 2026

Uh oh!

SunnyShu0925 commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026

Uh oh!

SunnyShu0925 commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

SunnyShu0925 commented Jul 9, 2026 •

edited

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading