Skip to content

fix(security-scanner): use truncateUtf16Safe for evidence truncation#102720

Closed
zhangqueping wants to merge 1 commit into
openclaw:mainfrom
zhangqueping:fix/utf16-safe-security-scanner
Closed

fix(security-scanner): use truncateUtf16Safe for evidence truncation#102720
zhangqueping wants to merge 1 commit into
openclaw:mainfrom
zhangqueping:fix/utf16-safe-security-scanner

Conversation

@zhangqueping

@zhangqueping zhangqueping commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

The skill security scanner uses a raw UTF-16 code-unit slice to truncate finding evidence at 120 characters. An emoji crossing the boundary could produce an unpaired surrogate in scan reports.

Why This Change Was Made

Replace .slice(0, maxLen) with truncateUtf16Safe in truncateEvidence. The existing default limit and ellipsis suffix remain unchanged.

Files Changed

File Change
src/skills/security/scanner.ts Replace raw .slice(0,N) with truncateUtf16Safe in truncateEvidence.

🤖 Generated with Claude Code

@zhangqueping
fix(security-scanner): use truncateUtf16Safe for evidence truncation
ddad734 
Replace raw UTF-16 slice in truncateEvidence with truncateUtf16Safe
to prevent surrogate pair splitting in skill security scan findings.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
@openclaw-barnacle openclaw-barnacle Bot added size: XS triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jul 9, 2026
@clawsweeper

clawsweeper Bot commented Jul 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 9, 2026, 7:32 AM ET / 11:32 UTC.

Summary
The branch imports truncateUtf16Safe and uses it in src/skills/security/scanner.ts so truncated skill security finding evidence does not split UTF-16 surrogate pairs.

PR surface: Source +1. Total +1 across 1 file.

Reproducibility: yes. Current main's truncateEvidence uses raw UTF-16 slicing, and a boundary string with an emoji leaves a dangling high surrogate; I confirmed the slice behavior with a standalone Node check but did not run the scanner entry point.

Review metrics: none identified.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add redacted terminal output, copied live output, logs, or a short validation transcript from a real scanner run showing the fixed evidence path, then update the PR body to trigger re-review.
  • Consider adding a focused scanner regression that asserts truncated finding evidence remains UTF-16 well-formed at the emoji boundary.

Proof guidance:

  • [P1] Needs real behavior proof before merge: Missing: the PR body has no redacted terminal output, live scanner transcript, logs, screenshot, recording, or linked artifact showing after-fix scanner evidence; contributor should redact private paths, endpoints, keys, phone numbers, and other sensitive details before posting proof. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] Contributor real behavior proof is missing, so reviewers have not seen a redacted real scanner run, terminal transcript, or log proving the after-fix evidence path.

Maintainer options:

  1. Decide the mitigation before merge
    Land the narrow scanner truncation change after contributor proof is added; a focused scanner regression for well-formed evidence would further strengthen the fix.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] The remaining blocker is contributor real behavior proof plus ordinary maintainer review; there is no narrow ClawSweeper repair to queue from this review.

Security
Cleared: The diff only swaps local scanner string truncation to an existing internal helper and does not change dependencies, workflows, secrets, package metadata, or code execution behavior.

Review details

Best possible solution:

Land the narrow scanner truncation change after contributor proof is added; a focused scanner regression for well-formed evidence would further strengthen the fix.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main's truncateEvidence uses raw UTF-16 slicing, and a boundary string with an emoji leaves a dangling high surrogate; I confirmed the slice behavior with a standalone Node check but did not run the scanner entry point.

Is this the best way to solve the issue?

Yes. Reusing the existing UTF-16-safe helper at the local scanner evidence truncation point is the narrow maintainable fix; proof and preferably a focused scanner regression are the remaining confidence gaps.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 34d257713ea3.

Label changes

Label changes:

  • add P3: This is a small scanner report-correctness fix with limited blast radius and no evidence of a broken user workflow or compatibility surface.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: Missing: the PR body has no redacted terminal output, live scanner transcript, logs, screenshot, recording, or linked artifact showing after-fix scanner evidence; contributor should redact private paths, endpoints, keys, phone numbers, and other sensitive details before posting proof. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P3: This is a small scanner report-correctness fix with limited blast radius and no evidence of a broken user workflow or compatibility surface.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: Missing: the PR body has no redacted terminal output, live scanner transcript, logs, screenshot, recording, or linked artifact showing after-fix scanner evidence; contributor should redact private paths, endpoints, keys, phone numbers, and other sensitive details before posting proof. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +1. Total +1 across 1 file.

View PR surface stats
Area Files Added Removed Net
Source 1 2 1 +1
Tests 0 0 0 0
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 1 2 1 +1

What I checked:

  • Repository policy read: Root AGENTS.md was read fully; no scoped AGENTS.md owns src/skills/security, and the review applied the deep PR review and proof-gate guidance. (AGENTS.md:1, 34d257713ea3)
  • Current main behavior: truncateEvidence currently returns ${evidence.slice(0, maxLen)}…, so scanner evidence can retain a dangling high surrogate when an emoji crosses the 120-code-unit boundary. (src/skills/security/scanner.ts:274, 34d257713ea3)
  • Scanner entry points: Line and source-rule findings call truncateEvidence before storing SkillScanFinding.evidence, so the changed helper is on the affected report path rather than an unused utility. (src/skills/security/scanner.ts:430, 34d257713ea3)
  • Helper contract: truncateUtf16Safe clamps the limit and delegates to sliceUtf16Safe, which avoids returning dangling surrogate halves at slice edges. (packages/normalization-core/src/utf16-slice.ts:43, 34d257713ea3)
  • Import surface exists: The @openclaw/normalization-core/utf16-slice subpath is exported by the package and mapped in tsconfig, matching many existing core imports. (packages/normalization-core/package.json:47, 34d257713ea3)
  • Source reproduction: A standalone Node check with 119 ASCII code units followed by an emoji showed raw .slice(0, 120) leaves boundary code unit 0xd83d, a dangling high surrogate.

Likely related people:

  • lsr911: git blame and git log --follow show the current scanner, truncateEvidence, and UTF-16 helper were added in landed commit e39e628a8416ef5426b5bf62e42539bd058e86bb, authored by lsr911. (role: introduced behavior; confidence: high; commits: e39e628a8416; files: src/skills/security/scanner.ts, src/skills/security/scanner.test.ts, packages/normalization-core/src/utf16-slice.ts)
  • steipete: The live discussion on fix(active-memory): use truncateUtf16Safe for log value truncation #102551 shows steipete provided land-ready verification and merged the landed commit that introduced the scanner and helper surfaces. (role: recent reviewer and merger; confidence: medium; commits: 94fb898bd4d1, e39e628a8416; files: src/skills/security/scanner.ts, packages/normalization-core/src/utf16-slice.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jul 9, 2026
@steipete steipete mentioned this pull request Jul 9, 2026
Merged
@steipete

steipete commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Thanks @zhangqueping. #102823 landed the same UTF-16-safe security-scanner evidence cap in 5ef269c, with regression coverage and the sibling source-rule fallback fixed too. This branch is fully superseded, so I’m closing it.

@steipete steipete closed this Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@zhangqueping @steipete