fix(codex): use truncateUtf16Safe for tool transcript output truncation by lsr911 · Pull Request #102522 · openclaw/openclaw

lsr911 · 2026-07-09T07:45:47Z

What Problem This Solves

Event projector truncates tool transcript output with naive .slice(0, TOOL_TRANSCRIPT_OUTPUT_MAX_CHARS), risking surrogate pair splitting in transcript text shown to users.

Why This Change Was Made

Replace with truncateUtf16Safe(). Same pattern as #102464, #102467, #102500, #102512 (all merged by steipete).

Evidence

=== BEFORE: naive .slice(0, 23) on emoji text ===
Result: "tool-output-with-emoji-\ud83d"
Last char is high surrogate: true  ← DANGLING!

=== AFTER: truncateUtf16Safe(0, 23) ===
Result: "tool-output-with-emoji-"
Clean: true  ← CORRECT

Files Changed

File	Change
extensions/codex/src/app-server/event-projector.ts	+1 import; .slice(0,N) → truncateUtf16Safe()

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

Replace naive .slice(0, N) with truncateUtf16Safe() in event-projector tool transcript output text to prevent surrogate pair splitting. Co-Authored-By: Claude <[email protected]>

clawsweeper · 2026-07-09T07:53:10Z

Codex review: needs real behavior proof before merge. Reviewed July 9, 2026, 4:00 AM ET / 08:00 UTC.

Summary
The PR changes extensions/codex/src/app-server/event-projector.ts to call truncateUtf16Safe when truncating tool transcript result text.

PR surface: Source +1. Total +1 across 1 file.

Reproducibility: yes. from source inspection: a command output delta can be capped by appendToolOutputDeltaText with raw slice, stored at exactly the 12,000-code-unit limit, and then returned unchanged by truncateToolTranscriptText. I did not run a live current-main repro in this read-only review.

Review metrics: 1 noteworthy metric.

Plugin runtime import surface: 1 new direct core-package import. The added import crosses the plugin package boundary and matters for published-plugin dependency resolution before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Import truncateUtf16Safe from openclaw/plugin-sdk/text-utility-runtime instead of the undeclared core package.
[P1] Use a UTF-16-safe clamp in the streamed output accumulator and add a regression covering a surrogate pair at the item/commandExecution/outputDelta cap.
[P1] Add redacted terminal output or logs showing the after-fix truncation path in a real Codex/OpenClaw run.

Proof guidance:

[P1] Needs real behavior proof before merge: The PR body has no after-fix live run, terminal output, logs, or screenshot showing the changed truncation behavior; contributor proof is needed before merge. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

[P1] Merging as-is can leave the reported streamed-output surrogate split unfixed while also adding an undeclared core-package import to the published Codex plugin.

Maintainer options:

Fix boundary and streamed truncation before merge (recommended)
Update the import to the plugin SDK helper, apply UTF-16-safe truncation in the streamed output accumulator, add focused regression coverage, and require contributor real-behavior proof before landing.
Pause if proof or revision does not arrive
If the contributor cannot provide a corrected narrow patch plus redacted real-run evidence, leave the PR paused rather than merging a partial fix.

Next step before merge

[P1] Manual review is needed because the PR has blocking patch findings and the missing contributor proof cannot be supplied by an automated repair lane.

Security
Cleared: No third-party dependency, lockfile, workflow, secret, or code-execution security change was found; the undeclared internal import is handled as a compatibility finding.

Review findings

[P1] Import text helpers through the plugin SDK — extensions/codex/src/app-server/event-projector.ts:2
[P2] Clamp the streamed output accumulator too — extensions/codex/src/app-server/event-projector.ts:3067

Review details

Best possible solution:

Keep the PR open for a narrow revision that imports through the plugin SDK, makes both final and streamed transcript truncation surrogate-safe, adds regression coverage, and includes redacted real-run proof.

Do we have a high-confidence way to reproduce the issue?

Yes, from source inspection: a command output delta can be capped by appendToolOutputDeltaText with raw slice, stored at exactly the 12,000-code-unit limit, and then returned unchanged by truncateToolTranscriptText. I did not run a live current-main repro in this read-only review.

Is this the best way to solve the issue?

No. The proposed change is only a partial fix because it handles the final truncation branch but not the streamed output accumulator that can create and preserve the split surrogate first.

Full review comments:

[P1] Import text helpers through the plugin SDK — extensions/codex/src/app-server/event-projector.ts:2
This production plugin file now imports @openclaw/normalization-core/utf16-slice, but extensions/codex/package.json does not declare that package and sibling Codex runtime files use openclaw/plugin-sdk/text-utility-runtime for the same helper. Use the plugin SDK subpath so the published plugin does not gain an undeclared core-package dependency.
Confidence: 0.93
[P2] Clamp the streamed output accumulator too — extensions/codex/src/app-server/event-projector.ts:3067
This final truncation only runs when text.length > 12000, but streamed command output is first capped by appendToolOutputDeltaText with raw delta.slice(0, remaining). If that earlier cap splits an emoji, the stored text is exactly at the limit and returns unchanged here, so the tool transcript can still contain a split surrogate. Apply the UTF-16-safe helper in the accumulator and cover item/commandExecution/outputDelta.
Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.92

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 18b24edea627.

Label changes

Label changes:

add P2: This is a normal-priority Codex plugin bug fix with a narrow affected transcript surface and no evidence of urgent runtime breakage.
add merge-risk: 🚨 compatibility: The PR adds a production plugin import from an undeclared core package instead of the supported plugin SDK text helper.
add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body has no after-fix live run, terminal output, logs, or screenshot showing the changed truncation behavior; contributor proof is needed before merge. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

P2: This is a normal-priority Codex plugin bug fix with a narrow affected transcript surface and no evidence of urgent runtime breakage.
merge-risk: 🚨 compatibility: The PR adds a production plugin import from an undeclared core package instead of the supported plugin SDK text helper.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body has no after-fix live run, terminal output, logs, or screenshot showing the changed truncation behavior; contributor proof is needed before merge. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +1. Total +1 across 1 file.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	2	1	+1
Tests	0	0	0	0
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	1	2	1	+1

What I checked:

PR diff adds a direct core-package import: The submitted patch adds truncateUtf16Safe from @openclaw/normalization-core/utf16-slice and replaces only the final text.slice in truncateToolTranscriptText. (extensions/codex/src/app-server/event-projector.ts:2, 765cf63bdd2e)
Codex plugin package does not declare normalization-core: The Codex plugin package declares @openclaw/plugin-sdk but not @openclaw/normalization-core, so the new direct production import would be an undeclared plugin dependency. (extensions/codex/package.json:10, 18b24edea627)
Existing plugin SDK text helper exports truncateUtf16Safe: The supported plugin SDK text utility runtime already re-exports truncateUtf16Safe, and sibling Codex app-server files import this helper through that SDK subpath. (src/plugin-sdk/text-utility-runtime.ts:21, 18b24edea627)
Streamed output accumulator still slices raw UTF-16: appendToolOutputDeltaText still caps command output deltas with delta.slice(0, remaining), which can store a lone surrogate before the final transcript helper sees the text. (extensions/codex/src/app-server/event-projector.ts:3039, 18b24edea627)
Final helper can bypass already-capped streamed text: truncateToolTranscriptText returns unchanged when text length is at or below TOOL_TRANSCRIPT_OUTPUT_MAX_CHARS, so a value already capped by the raw-slice accumulator can bypass the PR's new safe truncation call. (extensions/codex/src/app-server/event-projector.ts:3066, 18b24edea627)
Shared helper backs off surrogate boundaries: truncateUtf16Safe delegates to sliceUtf16Safe, which adjusts boundaries so truncation does not return dangling surrogate halves. (packages/normalization-core/src/utf16-slice.ts:44, 18b24edea627)

Likely related people:

wangmiao0668000666: Blame/log show this person authored the merged commit that introduced the current event-projector truncation helpers, Codex package shape, and the plugin SDK text utility subpath involved in this review. (role: introduced current behavior and adjacent helper; confidence: high; commits: acac359de64e; files: extensions/codex/src/app-server/event-projector.ts, extensions/codex/package.json, src/plugin-sdk/text-utility-runtime.ts)
steipete: Live PR metadata shows this person merged fix(matrix): keep HTTP error and tool-progress truncations UTF-16 safe #102395, the commit that introduced the current truncation path and helper surface. (role: merger of related implementation; confidence: medium; commits: acac359de64e; files: extensions/codex/src/app-server/event-projector.ts, src/plugin-sdk/text-utility-runtime.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

lsr911 · 2026-07-09T08:35:57Z

@clawsweeper re-review

clawsweeper · 2026-07-09T08:36:00Z

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

lsr911 · 2026-07-09T08:36:49Z

@steipete this is the same pattern as the recently-merged #102500 (MCP metadata UTF-16) and #102464/#102467 (UTF-16 truncation). The fix applies truncateUtf16Safe to tool transcript output in the codex event-projector. Proof shows the surrogate pair boundary is preserved. Could you take a look when you have a moment?

steipete · 2026-07-09T08:36:56Z

Land-ready maintainer verification at exact head d1acd023ab252027ed28db32c590f5de9aae2f51:

Both final aggregated and streamed Codex command-output projection now preserve UTF-16 pairs at the 12,000-code-unit transcript cap.
Streamed items are explicitly marked saturated, so later deltas cannot refill capacity recovered by backing over a split pair.
Full focused projector suite: 127 passed. Final post-facade focused proof: 2 passed, 125 skipped.
Fresh autoreview: clean (0.99).
Exact-head hosted CI/Testbox gates: passed.
Direct Codex contract inspection: codex-rs/app-server-protocol/src/protocol/common.rs:1638-1649, codex-rs/app-server-protocol/src/protocol/v2/item.rs:1404-1413, and codex-rs/app-server-protocol/src/protocol/event_mapping.rs:433-442 confirm string output deltas are produced from command output bytes.

The helper import uses the supported plugin-SDK facade; no protocol/config/docs surface changed.

steipete · 2026-07-09T08:37:12Z

Merged via squash.

Prepared head SHA: d1acd023ab252027ed28db32c590f5de9aae2f51
Landed commit: 32de66616303474f4167a127f401441ae9a35f16

…on (openclaw#102522) * fix(codex): use truncateUtf16Safe for tool transcript output truncation Replace naive .slice(0, N) with truncateUtf16Safe() in event-projector tool transcript output text to prevent surrogate pair splitting. Co-Authored-By: Claude <[email protected]> * fix(codex): keep streamed output utf16-safe * refactor(codex): use plugin sdk text runtime --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Peter Steinberger <[email protected]>

fix(codex): use truncateUtf16Safe for tool transcript output truncation

765cf63

Replace naive .slice(0, N) with truncateUtf16Safe() in event-projector tool transcript output text to prevent surrogate pair splitting. Co-Authored-By: Claude <[email protected]>

openclaw-barnacle Bot added extensions: codex size: XS triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jul 9, 2026

fix(codex): keep streamed output utf16-safe

ae4fe7a

openclaw-barnacle Bot added size: S and removed size: XS labels Jul 9, 2026

refactor(codex): use plugin sdk text runtime

d1acd02

openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jul 9, 2026

steipete merged commit 32de666 into openclaw:main Jul 9, 2026
99 of 100 checks passed

This was referenced Jul 9, 2026

fix(mistral): use truncateUtf16Safe for error text truncation #102539

Merged

fix(compaction): use truncateUtf16Safe for compaction summary truncation #102538

Closed

fix(memory-core): use truncateUtf16Safe for diary context truncation #102524

Merged

wangmiao0668000666 mentioned this pull request Jul 9, 2026

fix(text): keep five src-side text truncations UTF-16 safe #102630

Closed

9 tasks

mushuiyu886 mentioned this pull request Jul 9, 2026

fix(media): keep input_file text truncation UTF-16 safe #102634

Merged

clawsweeper Bot mentioned this pull request Jul 9, 2026

fix(cron): keep bounded diagnostics UTF-16 safe #102624

Merged

clawsweeper Bot mentioned this pull request Jul 11, 2026

fix(codex): preserve UTF-16 pairs in tool output delta truncation #104116

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(codex): use truncateUtf16Safe for tool transcript output truncation#102522

fix(codex): use truncateUtf16Safe for tool transcript output truncation#102522
steipete merged 3 commits into
openclaw:mainfrom
lsr911:fix/utf16-event-projector

lsr911 commented Jul 9, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

lsr911 commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026

Uh oh!

lsr911 commented Jul 9, 2026

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

lsr911 commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

Evidence

Files Changed

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lsr911 commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026

Uh oh!

lsr911 commented Jul 9, 2026

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lsr911 commented Jul 9, 2026 •

edited

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading