Skip to content

fix(memory-lancedb): use truncateUtf16Safe for sanitize text truncation#102767

Closed
coder-master-0915 wants to merge 1 commit into
openclaw:mainfrom
coder-master-0915:fix/truncateUtf16Safe-memory-lancedb
Closed

fix(memory-lancedb): use truncateUtf16Safe for sanitize text truncation#102767
coder-master-0915 wants to merge 1 commit into
openclaw:mainfrom
coder-master-0915:fix/truncateUtf16Safe-memory-lancedb

Conversation

@coder-master-0915

Copy link
Copy Markdown

What Problem This Solves

One missed .slice(0, N) truncation site in the memory-lancedb extension may cut UTF-16 surrogate pairs in half when the text content contains multi-byte characters such as emoji. The same file already uses truncateUtf16Safe in other places.

Why This Change Was Made

extensions/memory-lancedb/index.ts was partially converted but text.slice(0, MAX_SANITIZE_CHARS) on the sanitize path was missed.

User Impact

Truncated text in memory-lancedb sanitization remains valid Unicode at existing size limits.

Evidence

  • Mechanical replacement with function already imported in the same file
  • No runtime behavior change for ASCII or valid Unicode input

Generated with Claude Code

@openclaw-barnacle openclaw-barnacle Bot added extensions: memory-lancedb Extension: memory-lancedb size: XS labels Jul 9, 2026
@clawsweeper

clawsweeper Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 9, 2026, 8:09 AM ET / 12:09 UTC.

Summary
The PR replaces the memory-lancedb sanitizer's pre-regex raw .slice(0, 10000) truncation with the existing truncateUtf16Safe helper.

PR surface: Source 0. Total 0 across 1 file.

Reproducibility: yes. from source: on current main, sanitizeForMemoryCapture pre-truncates long input with raw slice, so an emoji crossing the 10,000-code-unit boundary can leave a dangling surrogate. I did not execute a runtime repro because this review is read-only.

Review metrics: none identified.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🧂 unranked krab
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add redacted terminal output, logs, or a reproducible run showing sanitizeForMemoryCapture keeps valid Unicode at the 10,000-character boundary.
  • [P1] Add a focused sanitizer regression test for an emoji/surrogate boundary if the contributor wants stronger merge confidence.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body has only mechanical rationale and no after-fix live output/logs; please add redacted terminal output or a reproducible run showing the sanitizer result, then update the PR body to trigger re-review.

Risk before merge

  • [P1] The contributor has not provided after-fix real behavior proof; source inspection supports the fix, but the external-PR proof gate still needs redacted live output, logs, a terminal transcript, or maintainer override.
  • [P1] The adjacent sanitizer test only covers truncation length today; it would not catch the specific surrogate-boundary regression if this site regressed again.

Maintainer options:

  1. Decide the mitigation before merge
    Land the one-line sanitizer truncation fix after contributor proof or maintainer override, ideally with a focused surrogate-boundary sanitizer regression test.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] No code repair is currently indicated, but this external PR needs contributor-provided real behavior proof before normal merge consideration.

Security
Cleared: The diff is a one-line text truncation helper swap in plugin runtime code and does not touch dependencies, workflows, secrets, permissions, or package resolution.

Review details

Best possible solution:

Land the one-line sanitizer truncation fix after contributor proof or maintainer override, ideally with a focused surrogate-boundary sanitizer regression test.

Do we have a high-confidence way to reproduce the issue?

Yes from source: on current main, sanitizeForMemoryCapture pre-truncates long input with raw slice, so an emoji crossing the 10,000-code-unit boundary can leave a dangling surrogate. I did not execute a runtime repro because this review is read-only.

Is this the best way to solve the issue?

Yes: using the already imported truncateUtf16Safe at the missed sanitizer pre-truncation site is the narrowest maintainable fix. A broader refactor is unnecessary because the shared helper already owns the surrogate-safe contract.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f786efddcd88.

Label changes

Label changes:

  • add P2: This is a narrow correctness bug in a bundled memory plugin sanitizer with limited blast radius and no evidence of a live urgent regression.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body has only mechanical rationale and no after-fix live output/logs; please add redacted terminal output or a reproducible run showing the sanitizer result, then update the PR body to trigger re-review.

Label justifications:

  • P2: This is a narrow correctness bug in a bundled memory plugin sanitizer with limited blast radius and no evidence of a live urgent regression.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body has only mechanical rationale and no after-fix live output/logs; please add redacted terminal output or a reproducible run showing the sanitizer result, then update the PR body to trigger re-review.
Evidence reviewed

PR surface:

Source 0. Total 0 across 1 file.

View PR surface stats
Area Files Added Removed Net
Source 1 1 1 0
Tests 0 0 0 0
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 1 1 1 0

What I checked:

Likely related people:

  • vincentkoc: Current main blame attributes the sanitize function, raw pre-truncation line, and adjacent tests to commit 0ac8933, and GitHub commit metadata maps the author/committer to vincentkoc. (role: recent area contributor; confidence: high; commits: 0ac8933721f7; files: extensions/memory-lancedb/index.ts, extensions/memory-lancedb/index.test.ts)
  • amittell: A prior closed PR on the same memory-lancedb sanitizer surface describes adding sanitizeForMemoryCapture and extensive memory-lancedb sanitizer tests, making amittell relevant for domain context even though current main blame is on a later commit. (role: adjacent feature contributor; confidence: medium; commits: e47fcf4336a9; files: extensions/memory-lancedb/index.ts, extensions/memory-lancedb/index.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jul 9, 2026
@vincentkoc vincentkoc self-assigned this Jul 9, 2026
@vincentkoc

Copy link
Copy Markdown
Member

Closing this standalone fragment in favor of the wider UTF-16 cleanup. The sanitizer boundary is a valid cleanup site, but this is a one-line substitution with no regression test or user-visible reproduction. It should land with a focused capture-sanitizer boundary test as part of the consolidated work, not consume a separate review and CI cycle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: memory-lancedb Extension: memory-lancedb P2 Normal backlog priority with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants