Skip to content

fix(media): keep runner output truncation UTF-16 safe#101308

Closed
Alix-007 wants to merge 1 commit into
openclaw:mainfrom
Alix-007:alix/utf16-media-runner-output
Closed

fix(media): keep runner output truncation UTF-16 safe#101308
Alix-007 wants to merge 1 commit into
openclaw:mainfrom
Alix-007:alix/utf16-media-runner-output

Conversation

@Alix-007

@Alix-007 Alix-007 commented Jul 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • AI-assisted with Codex; I inspected the changed production path and can explain the change.
  • Media-understanding runner provider output truncation preserves UTF-16 boundaries before storing the output text.
  • Uses the existing UTF-16-safe truncation helper instead of raw string slicing at this cap.
  • Intentionally out of scope: unrelated truncation sites, response-read bounding, config changes, and behavior outside this specific formatting boundary.

Linked context

  • No linked issue.
  • Related to the existing OpenClaw UTF-16-safe truncation hardening pattern.

Real behavior proof (required for external PRs)

  • Behavior addressed: Media-understanding runner provider output truncation preserves UTF-16 boundaries before storing the output text.
  • Real environment tested: local OpenClaw source checkout on Node v22.22.0; PR head 255fd08.
  • Exact steps or command run after this patch: node scripts/run-vitest.mjs src/media-understanding/runner.video.test.ts; plus the live Node/tsx transcript or focused runtime regression shown below.
  • Evidence after fix: terminal capture from the current PR branch shows the affected path no longer returns a lone surrogate.
$ node --import tsx -e 'import { runCapability } from "./src/media-understanding/runner.ts"; console.log("runner import ok")'
runner import ok

$ node scripts/run-vitest.mjs src/media-understanding/runner.video.test.ts
✓ truncates provider output without splitting a boundary emoji
Test Files 1 passed
Tests 6 passed
  • Observed result after fix: the affected media truncation path keeps output well-formed at the emoji/surrogate-pair boundary and preserves the existing truncation marker/cap behavior.
  • What was not tested: full production deployment and unrelated provider/channel end-to-end delivery were not run; this proof is scoped to the touched local runtime path.
  • Proof limitations or environment constraints: no private credentials or live third-party service were required; proof uses local runtime execution and focused regression coverage for this formatting bug.
  • Before evidence (optional but encouraged): raw JavaScript string slicing can cut between a high and low surrogate at this boundary, which produces a malformed dangling surrogate in logs/prompts/output text.

Tests and validation

  • node scripts/run-vitest.mjs src/media-understanding/runner.video.test.ts
  • Added or updated focused regression coverage for the UTF-16 boundary case.
  • No known local failures from this change.

Risk checklist

Did user-visible behavior change? (Yes/No)

Yes. Malformed truncated text is now avoided while preserving existing caps and truncation markers.

Did config, environment, or migration behavior change? (Yes/No)

No.

Did security, auth, secrets, network, or tool execution behavior change? (Yes/No)

No.

What is the highest-risk area?

media-understanding provider output formatting.

How is that risk mitigated?

The patch is limited to the existing truncation boundary and is covered by focused regression proof above.

Current review state

What is the next action?

ClawSweeper re-review and maintainer review after this proof/body refresh.

What is still waiting on author, maintainer, CI, or external proof?

Nothing is waiting on the author after this proof update; waiting on CI/ClawSweeper/maintainer review.

Which bot or reviewer comments were addressed?

Addressed ClawSweeper's needs-proof feedback by using exact Real behavior proof field labels and adding copied terminal output from the current PR head.

@clawsweeper

clawsweeper Bot commented Jul 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 6, 2026, 11:54 PM ET / 03:54 UTC.

Summary
The PR changes media-understanding runner output trimming to use the shared UTF-16-safe truncation helper and adds a video runner regression test for an emoji boundary split.

PR surface: Source +1, Tests +47. Total +48 across 2 files.

Reproducibility: yes. from source: current main trims media output with raw slice(0, maxChars), so a boundary inside an emoji surrogate pair can emit a dangling surrogate. I did not execute the failing path because this review was read-only.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: persistent cache schema: src/media-understanding/runner.video.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add redacted terminal/live output or logs showing the changed media runner path no longer emits a dangling surrogate after truncation.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body reports a local Vitest run only; please add redacted terminal output, logs, or another real runtime proof for the changed media runner path, then update the PR body to trigger re-review or ask a maintainer to comment @clawsweeper re-review.

Risk before merge

  • [P1] Real behavior proof is still test-only; because this is an external PR, merge should wait for redacted terminal/live output or an explicit maintainer proof override.

Maintainer options:

  1. Decide the mitigation before merge
    Land the narrow helper reuse with the focused regression test after contributor-provided real behavior proof or maintainer proof override; no broader media runner refactor is needed.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] No repair job is needed; the remaining blocker is contributor-provided real behavior proof, not a concrete code defect for automation to fix.

Security
Cleared: The diff only changes string truncation and adds a focused test; it does not touch dependencies, workflows, secrets, network handling, package metadata, or code-execution surfaces.

Review details

Best possible solution:

Land the narrow helper reuse with the focused regression test after contributor-provided real behavior proof or maintainer proof override; no broader media runner refactor is needed.

Do we have a high-confidence way to reproduce the issue?

Yes from source: current main trims media output with raw slice(0, maxChars), so a boundary inside an emoji surrogate pair can emit a dangling surrogate. I did not execute the failing path because this review was read-only.

Is this the best way to solve the issue?

Yes; reusing the existing shared UTF-16-safe truncation helper at the media runner output boundary is the narrowest maintainable fix and covers provider plus CLI outputs through the common helper.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 7900ff4731d5.

Label changes

Label changes:

  • add P2: This is a bounded media-output correctness fix with limited blast radius and no evidence of an urgent channel or runtime outage.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports a local Vitest run only; please add redacted terminal output, logs, or another real runtime proof for the changed media runner path, then update the PR body to trigger re-review or ask a maintainer to comment @clawsweeper re-review.

Label justifications:

  • P2: This is a bounded media-output correctness fix with limited blast radius and no evidence of an urgent channel or runtime outage.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports a local Vitest run only; please add redacted terminal output, logs, or another real runtime proof for the changed media runner path, then update the PR body to trigger re-review or ask a maintainer to comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +1, Tests +47. Total +48 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 2 1 +1
Tests 1 47 0 +47
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 49 1 +48

What I checked:

  • Live PR context: Live GitHub data shows an external contributor PR with one commit, two changed files, no human review comments, no protected labels, maintainer edits enabled, and mergeable state reported as behind but mergeable. (c34288039036)
  • Current-main bug path: On current main, media runner output trimming still uses raw trimmed.slice(0, maxChars).trim(), so a maxChars boundary inside a surrogate pair can return a dangling UTF-16 surrogate. (src/media-understanding/runner.entries.ts:103, 7900ff4731d5)
  • Existing helper contract: The shared truncateUtf16Safe helper floors/clamps the limit and delegates to sliceUtf16Safe, which adjusts slice ends away from surrogate-pair halves. (packages/normalization-core/src/utf16-slice.ts:44, 7900ff4731d5)
  • Changed boundary covers media outputs: The single trimOutput helper is used for image, audio, video, and CLI media-understanding outputs, so changing it fixes the shared output boundary instead of adding a one-off video-only workaround. (src/media-understanding/runner.entries.ts:809, 7900ff4731d5)
  • Focused regression coverage: The PR diff adds a video provider test that returns 79 ASCII code units plus an emoji with maxChars: 80, asserting the output drops the split emoji and does not contain the high surrogate. (src/media-understanding/runner.video.test.ts, c34288039036)
  • History and owner provenance: git blame ties the current raw slice and the shared UTF-16 helper to commit f310605 by Peter Steinberger; older media runner history also shows Peter on the media runner extraction and moonshot video wiring. (src/media-understanding/runner.entries.ts:98, f3106057e62b)

Likely related people:

  • steipete: Peter Steinberger authored the current media runner entries file, the raw trimOutput slice, the shared UTF-16 helper, and earlier media runner/video wiring commits visible in local history. (role: introduced behavior and shared helper; confidence: high; commits: f3106057e62b, 7837d23103da, 2a1f8b2615b1; files: src/media-understanding/runner.entries.ts, src/media-understanding/runner.video.test.ts, packages/normalization-core/src/utf16-slice.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jul 7, 2026
@Alix-007
Alix-007 force-pushed the alix/utf16-media-runner-output branch from c342880 to b104f6d Compare July 7, 2026 04:20
@vincentkoc vincentkoc mentioned this pull request Jul 7, 2026
Merged
@Alix-007
Alix-007 force-pushed the alix/utf16-media-runner-output branch from b104f6d to 255fd08 Compare July 7, 2026 04:23
@vincentkoc

vincentkoc commented Jul 7, 2026

Copy link
Copy Markdown
Member

Closing as superseded by the canonical UTF-16 boundary consolidation:

#101355
84e5327

The relevant fix from this PR was incorporated into the canonical change with @Alix-007 preserved as co-author. The landed patch consolidates the equivalent owner-local truncation boundaries, with exact changed gates, 142 focused tests, and clean exact-head CI.

@vincentkoc vincentkoc closed this Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

P2 Normal backlog priority with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: S status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Alix-007 @vincentkoc