fix(embedding-provider): use truncateUtf16Safe for text truncation#102774
fix(embedding-provider): use truncateUtf16Safe for text truncation#102774coder-master-0915 wants to merge 2 commits into
Conversation
openclaw-barnacle
Bot
added
size: XS
triage: needs-pr-context
|
ClawSweeper status: review started. I am starting a fresh review of this pull request: fix(embedding-provider): use truncateUtf16Safe for text truncation This is item 1/1 in the current shard. Shard 0/1. This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking. Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted. |
|
Closing as a later subset of #102770. The Crestodian and embedding substitutions are untested UTF-16 micro-fragments, and the embedding half also overlaps #102744. #102770 is also being closed so the genuinely arbitrary-text sites can be handled once in the wider canonical cleanup with owner-specific active-path tests. |
3 participants
What Problem This Solves
One
.slice(0, N)truncation site may cut UTF-16 surrogate pairs in half when the text contains multi-byte characters such as emoji.Why This Change Was Made
Replacing
.slice(0, N)withtruncateUtf16Safeensures truncated output is always valid Unicode.User Impact
Truncated text remains valid Unicode at existing size limits. No behavioral changes for ASCII-only content.
Generated with Claude Code