Skip to content

fix(codex,tui,cli): keep retained text UTF-16 safe#102599

Merged
steipete merged 1 commit into
openclaw:mainfrom
wings1029:fix/utf16-safe-codex-telegram-tui-session-history
Jul 9, 2026
Merged

fix(codex,tui,cli): keep retained text UTF-16 safe#102599
steipete merged 1 commit into
openclaw:mainfrom
wings1029:fix/utf16-safe-codex-telegram-tui-session-history

Conversation

@wings1029

@wings1029 wings1029 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Three retained-text boundaries could split a UTF-16 surrogate pair: Codex context text-part truncation, CLI session-history head/tail retention, and TUI local-shell output tails. A split emoji could therefore leave malformed text in a Codex prompt, reseed prompt, or operator-visible shell result.

Why This Change Was Made

Rebuilt the PR on current main, removed the Telegram hunk already superseded by the centralized raw-update formatter, and used the shared normalization primitives for the three remaining owners. The Codex projection also deletes its private tail-boundary implementation in favor of the same shared helper.

User Impact

Long Codex context, CLI reseed history, and TUI shell output remain valid Unicode at their existing limits. Retention direction, size budgets, marker text, and ASCII behavior are unchanged.

Evidence

  • Codex projection suite: 21 passed, including an exact 6,000-unit text-part boundary assertion.
  • CLI session-history suite: 26 passed, including exact retained-head and retained-tail boundaries.
  • TUI local-shell suite: 6 passed, including the combined stdout/stderr tail boundary.
  • Focused oxfmt, oxlint, and git diff --check passed.
  • Fresh autoreview clean (0.96).
  • Direct Codex source inspection confirmed turn/start sums Unicode scalar counts against MAX_USER_INPUT_TEXT_CHARS; this patch preserves OpenClaw's conservative existing budget and only prevents invalid UTF-16 projection output.

Co-authored-by: 陈志强0668000989 [email protected]

@openclaw-barnacle openclaw-barnacle Bot added channel: telegram Channel integration: telegram agents Agent runtime and tooling extensions: codex size: S labels Jul 9, 2026
@clawsweeper

clawsweeper Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 9, 2026, 5:36 AM ET / 09:36 UTC.

Summary
The PR replaces raw string slicing at Codex context projection, Telegram raw update logging, TUI local shell output, and CLI session-history truncation sites with UTF-16-safe helper calls and adds focused Codex/session-history tests.

PR surface: Source +10, Tests +31. Total +41 across 6 files.

Reproducibility: yes. source-reproducible: current main uses raw slice at the touched truncation boundaries, so placing an emoji surrogate pair across those caps can produce a dangling surrogate. I did not run a live OpenClaw/Codex/Telegram/TUI scenario in this read-only review.

Review metrics: 1 noteworthy metric.

  • Plugin production imports: 2 added. Both changed plugin files import a private workspace package where the repository boundary expects the plugin SDK facade or declared package-local runtime dependencies.

Stored data model
Persistent data-model change detected: serialized state: src/agents/cli-runner/session-history.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦪 silver shellfish
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Replace plugin production imports with openclaw/plugin-sdk/text-utility-runtime while keeping core imports on the internal helper where appropriate.
  • [P1] Add redacted real behavior proof, such as terminal output or logs showing one changed runtime path after the fix.

Proof guidance:

  • [P1] Needs real behavior proof before merge: Only test/CI-style evidence is visible; the contributor should add redacted terminal output, copied live output, logs, or a recording showing an after-fix changed runtime path, then update the PR body for re-review.

Risk before merge

  • [P1] Merging as-is can make the published @openclaw/codex plugin import a private workspace package that it does not declare or bundle, which is an upgrade/package compatibility risk.
  • [P1] The external PR has no real after-fix runtime proof showing a changed Codex, Telegram log, TUI shell, or CLI reseed path with emoji at the truncation boundary.

Maintainer options:

  1. Repair plugin imports before merge (recommended)
    Use the plugin SDK text helper facade in Codex and Telegram production code, then refresh checks and contributor proof before landing.
  2. Accept private-package coupling
    Maintainers could intentionally allow plugin code to import the private workspace package, but that would need explicit package/distribution ownership approval.
  3. Split the overlapping Telegram change
    If maintainers prefer the narrower Telegram-only branch, let fix(telegram): use truncateUtf16Safe for raw update log truncation #102567 own that site and keep this PR scoped to the remaining unique surfaces.

Next step before merge

  • [P1] The remaining blockers are contributor/human follow-up for a package-boundary code fix and real behavior proof; ClawSweeper should not repair this automatically while the external proof gate remains unmet.

Security
Cleared: No concrete security or supply-chain concern found; the diff changes string truncation calls/tests and does not alter workflows, lockfiles, secrets, permissions, or executable dependency sources.

Review findings

  • [P1] Use the plugin SDK helper in Codex plugin code — extensions/codex/src/app-server/context-engine-projection.ts:1
  • [P2] Keep Telegram on the plugin SDK helper surface — extensions/telegram/src/bot-core.ts:1
Review details

Best possible solution:

Keep the helper substitution, but route plugin code through openclaw/plugin-sdk/text-utility-runtime, preserve package dependency contracts, and add redacted real behavior proof for at least one changed runtime path before merge.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: current main uses raw slice at the touched truncation boundaries, so placing an emoji surrogate pair across those caps can produce a dangling surrogate. I did not run a live OpenClaw/Codex/Telegram/TUI scenario in this read-only review.

Is this the best way to solve the issue?

No as submitted: using the UTF-16-safe helper is the right implementation shape, but plugin production code should use the existing plugin SDK facade instead of importing the private normalization package directly.

Full review comments:

  • [P1] Use the plugin SDK helper in Codex plugin code — extensions/codex/src/app-server/context-engine-projection.ts:1
    This file is part of the published @openclaw/codex plugin, whose package metadata does not declare or bundle @openclaw/normalization-core, and that package is private. Importing it directly can make the installed plugin fail to resolve the helper; use openclaw/plugin-sdk/text-utility-runtime, which already exports truncateUtf16Safe, and apply the same facade pattern to the Telegram plugin import.
    Confidence: 0.91
  • [P2] Keep Telegram on the plugin SDK helper surface — extensions/telegram/src/bot-core.ts:1
    Telegram plugin production code is under the same extension boundary, and nearby raw-update-log.ts already imports truncateUtf16Safe from openclaw/plugin-sdk/text-utility-runtime. This new private workspace-package import creates a second dependency path for the same helper and should use the facade instead.
    Confidence: 0.86

Overall correctness: patch is incorrect
Overall confidence: 0.84

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 6433b8272350.

Label changes

Label changes:

  • add P2: This is a bounded text-corruption bug fix with limited blast radius, but the submitted PR needs package-boundary repair and proof before merge.
  • add merge-risk: 🚨 compatibility: The PR can break installed or upgraded plugin packages by adding private workspace-package imports that the plugin package metadata does not satisfy.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: Only test/CI-style evidence is visible; the contributor should add redacted terminal output, copied live output, logs, or a recording showing an after-fix changed runtime path, then update the PR body for re-review.

Label justifications:

  • P2: This is a bounded text-corruption bug fix with limited blast radius, but the submitted PR needs package-boundary repair and proof before merge.
  • merge-risk: 🚨 compatibility: The PR can break installed or upgraded plugin packages by adding private workspace-package imports that the plugin package metadata does not satisfy.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: Only test/CI-style evidence is visible; the contributor should add redacted terminal output, copied live output, logs, or a recording showing an after-fix changed runtime path, then update the PR body for re-review.
Evidence reviewed

PR surface:

Source +10, Tests +31. Total +41 across 6 files.

View PR surface stats
Area Files Added Removed Net
Source 4 23 13 +10
Tests 2 31 0 +31
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 6 54 13 +41

What I checked:

Likely related people:

  • steipete: Recent maintainer verification and merge comments on adjacent UTF-16 truncation fixes established the helper/facade pattern and proof expectations for Codex/Gateway display paths. (role: recent reviewer and merger; confidence: high; commits: c2cc50c9a07a, 8c608ae67485; files: extensions/codex/src/app-server/approval-bridge.ts, packages/normalization-core/src/utf16-slice.ts, src/plugin-sdk/text-utility-runtime.ts)
  • lsr911: Current-main blame and related PR history show recent work across the same UTF-16-safe truncation helper rollout, including the overlapping Telegram raw-update-log PR. (role: recent adjacent UTF-16 truncation contributor; confidence: medium; commits: 8c608ae67485, 96a77fd057f1; files: extensions/telegram/src/bot-core.ts, packages/normalization-core/src/utf16-slice.ts)
  • xialonglee: Merged Codex app-server UTF-16 truncation work in the same plugin area used the plugin SDK text helper facade and is relevant to the Codex-side import pattern. (role: adjacent Codex plugin contributor; confidence: medium; commits: c2cc50c9a07a, 96c885e4aabd; files: extensions/codex/src/app-server/approval-bridge.ts, extensions/codex/src/app-server/elicitation-bridge.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jul 9, 2026
@steipete
steipete force-pushed the fix/utf16-safe-codex-telegram-tui-session-history branch from d7104bd to 13c4e1a Compare July 9, 2026 11:16
@steipete steipete changed the title fix(codex,telegram,tui,cli): keep text truncation UTF-16 safe at four remaining call sites fix(codex,tui,cli): keep retained text UTF-16 safe Jul 9, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the channel: telegram Channel integration: telegram label Jul 9, 2026
@steipete
steipete merged commit 0bdd646 into openclaw:main Jul 9, 2026
102 of 105 checks passed
@steipete

steipete commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling extensions: codex merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: S status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants