fix(plugins,media): keep embedding error and input file truncation surrogate-safe#102744
fix(plugins,media): keep embedding error and input file truncation surrogate-safe#102744zw-xysk wants to merge 1 commit into
Conversation
…rrogate-safe Two locations use .slice(0, N) for text truncation: - openai-compatible-embedding-provider.ts: embedding API error body - input-files.ts: input file text clamp Replace .slice(0, N) with truncateUtf16Safe(text, N).
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
ClawSweeper status: review started. I am starting a fresh review of this pull request: fix(plugins,media): keep embedding error and input file truncation surrogate-safe This is item 1/1 in the current shard. Shard 0/1. This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking. Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted. |
|
Thanks for checking these truncation sites. I'm closing this version because it bundles two one-site substitutions without regression coverage, and it is not the best review unit for the bug class. The A future canonical change would need one coherent owner surface and focused tests that fail on the real call path, especially for the embedding error snippet. |
2 participants
What Problem This Solves
Two locations use
.slice(0, N)for text truncation without UTF-16 surrogate pair safety:When the truncation boundary falls inside a UTF-16 surrogate pair (emoji, CJK extended), the resulting text contains a dangling surrogate.
Changes
src/plugins/openai-compatible-embedding-provider.ts(+2, −1):text.slice(0, N)→truncateUtf16Safe(text, N)src/media/input-files.ts(+2, −1):text.slice(0, maxChars)→truncateUtf16Safe(text, maxChars)Evidence
Unit tests (26/26 passed):
Real behavior proof (platinum):
Review