fix: keep bounded Unicode text valid across UTF-16 boundaries by steipete · Pull Request #102823 · openclaw/openclaw

steipete · 2026-07-09T13:02:22Z

Supersedes #102669, #102671, #102672, #102674, #102685, #102695, #102696, #102697, #102715, #102719, #102720, #102721, #102722, #102723, #102734, #102737, and #102741.

Also preserves the useful unique changes from closed #102667, #102767, #102774, and #102775.

What Problem This Solves

Fixes an issue where bounded text could end or begin with half of a UTF-16 surrogate pair when an emoji or supplementary-plane character crossed a size limit. Depending on the consumer, that could surface as a replacement character in user-visible errors, labels, diagnostics, stored artifacts, model prompts, QA evidence, or Control UI previews.

Why This Change Was Made

This consolidates the remaining real call sites onto the existing UTF-16-safe slicing primitives while preserving each surface's established UTF-16-unit budget and undefined versus empty-string semantics. It also makes head-plus-tail previews safe at both boundaries, keeps the QA Lab browser build source-resolvable, and carries the exact compacted-prefix boundary through tool-call stream suppression so safe slicing cannot weaken terminal snapshot scrubbing.

User Impact

Emoji and supplementary-plane text now remains valid when these bounded previews, labels, filenames, errors, and prompts are truncated. No configuration or migration is required.

Evidence

Standalone clean-source QA Lab Vite build passed on Blacksmith Testbox.
Post-refactor QA Lab fallback proof (while Testbox lease startup was unavailable): Vite built 18 modules and the focused UI suite passed 9/9.
Focused Testbox run passed 6 Vitest shards, including 125 unit-fast assertions plus hooks, auto-reply, UI, WhatsApp, and QA Lab suites: https://github.com/openclaw/openclaw/actions/runs/29019077492
Exact follow-up stream/label regression run passed 67 tool-call stream tests and 9 conversation-label tests: https://github.com/openclaw/openclaw/actions/runs/29019254011
Exact follow-up media/QA run passed 48 media-store tests and 9 QA Lab UI tests: https://github.com/openclaw/openclaw/actions/runs/29019373020
Full production build, including plugin SDK exports and Control UI, passed on Blacksmith Testbox: https://github.com/openclaw/openclaw/actions/runs/29019579334
Dependency report: one intentional internal workspace link plus its importer lockfile entry; 0 added, removed, or version-changed resolved packages. QA Lab stays behind the public plugin SDK facade.
Exact-head hosted CI passed for e206900c5ab277857af14ff88f511a9b29473e3f: https://github.com/openclaw/openclaw/actions/runs/29024343939.
Fresh autoreview: no accepted/actionable findings after repairing the compacted-buffer state and QA Lab build-resolution findings.
Independent source review: no remaining findings across the consolidated diff.

github-actions · 2026-07-09T13:03:33Z

Dependency Guard

This PR changes dependency-related files. Maintainers should confirm these changes are intentional.

Changed files:

packages/tool-call-repair/package.json
pnpm-lock.yaml

Maintainer follow-up:

Review whether the dependency changes are intentional.
Inspect resolved package deltas when lockfile, shrinkwrap, or workspace dependency policy changes are present.
Treat package-lock.json and npm-shrinkwrap.json diffs as security-review surfaces.
Run pnpm deps:changes:report -- --base-ref origin/main --markdown /tmp/dependency-changes.md --json /tmp/dependency-changes.json locally for detailed release-style evidence.

github-actions · 2026-07-09T13:03:34Z

Dependency graph changes noted

This PR includes dependency graph changes. The dependency guard is informational because the PR author is a repository admin or a member of @openclaw/openclaw-secops.

Current SHA: e206900c5ab277857af14ff88f511a9b29473e3f
Trusted actor: @steipete
Trusted role: pull request author; openclaw-secops

Security review is still recommended before merge when the dependency graph change is intentional.

clawsweeper · 2026-07-09T13:23:28Z

Codex review: stale review; fresh review needed.

Summary
The latest durable ClawSweeper review was for head 5463b996daa36a52c4fd90afc962f0d465c705d2, but the PR head is now e206900c5ab277857af14ff88f511a9b29473e3f. Its old verdict and PR readiness labels are no longer current.

Next step
Run or wait for a fresh ClawSweeper review on the current PR head.

Review history (1 earlier review cycle)

reviewed 2026-07-09T13:30:03.614Z sha 2765f61 :: needs maintainer review before merge. :: none

Co-authored-by: 黄剑雄0668001315 <[email protected]> Co-authored-by: 0668000539 <[email protected]> Co-authored-by: 张鹊平0668001085 <[email protected]> Co-authored-by: 赵旺0668001248 <[email protected]> Co-authored-by: 陈宪彪0668000387 <[email protected]> Co-authored-by: lizeyu-xydt <[email protected]>

steipete · 2026-07-09T14:21:04Z

Land-ready verification completed for exact head e206900c5ab277857af14ff88f511a9b29473e3f.

Signed real ancestry: all three PR commits have good signatures; the live PR remains the intended 34-file patch (+364/−64).
Patch identity: 09f355786ae6634c2abd8d412f3cf060730540e2 remained stable across the final rebase.
Exact-head CI: 44 jobs passed, 11 expected skips, 0 failures: https://github.com/openclaw/openclaw/actions/runs/29024343939
Companion CodeQL, Critical Quality, Real behavior proof, Workflow Sanity, OpenGrep, Plugin Init, dependency/security guards, and iOS scope checks all passed.
Focused regressions: https://github.com/openclaw/openclaw/actions/runs/29019077492, https://github.com/openclaw/openclaw/actions/runs/29019254011, and https://github.com/openclaw/openclaw/actions/runs/29019373020
Full production, plugin SDK, and Control UI build: https://github.com/openclaw/openclaw/actions/runs/29019579334
Fresh full-diff autoreview: clean, no accepted/actionable findings (0.97 confidence).
Native prepare: OPENCLAW_TESTBOX=1 scripts/pr prepare-run 102823 passed and verified the remote tree at the exact head.

Known proof gaps: none.

steipete · 2026-07-09T14:21:28Z

Merged via squash.

Prepared head SHA: e206900c5ab277857af14ff88f511a9b29473e3f
Landed commit: 5ef269c2bbfc020e86db25691afbdeeb1df5b268

…aw#102823) * fix(text): keep bounded output UTF-16 safe Co-authored-by: 黄剑雄0668001315 <[email protected]> Co-authored-by: 0668000539 <[email protected]> Co-authored-by: 张鹊平0668001085 <[email protected]> Co-authored-by: 赵旺0668001248 <[email protected]> Co-authored-by: 陈宪彪0668000387 <[email protected]> Co-authored-by: lizeyu-xydt <[email protected]> * chore(changelog): defer release note to automation * refactor(qa-lab): keep text helper behind SDK --------- Co-authored-by: 黄剑雄0668001315 <[email protected]> Co-authored-by: 0668000539 <[email protected]> Co-authored-by: 张鹊平0668001085 <[email protected]> Co-authored-by: 赵旺0668001248 <[email protected]> Co-authored-by: 陈宪彪0668000387 <[email protected]> Co-authored-by: lizeyu-xydt <[email protected]>

steipete requested a review from a team as a code owner July 9, 2026 13:02

openclaw-barnacle Bot added channel: whatsapp-web Channel integration: whatsapp-web app: web-ui App: web-ui gateway Gateway runtime extensions: memory-lancedb Extension: memory-lancedb extensions: qa-lab size: M maintainer Maintainer-authored PR labels Jul 9, 2026

github-actions Bot added the dependencies-changed PR changes dependency-related files label Jul 9, 2026

clawsweeper Bot mentioned this pull request Jul 9, 2026

fix(llm-slug): use truncateUtf16Safe for LLM-prompt-facing session content truncation #102672

Closed

clawsweeper Bot mentioned this pull request Jul 9, 2026

fix(auto-reply): keep conversation label truncation surrogate-safe #102697

Closed

openclaw-barnacle Bot added channel: msteams Channel integration: msteams app: android App: android commands Command implementations agents Agent runtime and tooling channel: qqbot extensions: codex size: XL and removed size: M labels Jul 9, 2026

clawsweeper Bot mentioned this pull request Jul 9, 2026

fix(agent-runner-memory): use truncateUtf16Safe for memory flush error truncation #102685

Merged

steipete force-pushed the codex/utf16-boundary-sweep branch from 42d6193 to c0557ca Compare July 9, 2026 13:52

openclaw-barnacle Bot removed channel: msteams Channel integration: msteams app: android App: android labels Jul 9, 2026

clawsweeper Bot removed the status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. label Jul 9, 2026

steipete and others added 3 commits July 9, 2026 15:10

chore(changelog): defer release note to automation

dd854a2

refactor(qa-lab): keep text helper behind SDK

e206900

steipete force-pushed the codex/utf16-boundary-sweep branch from 5463b99 to e206900 Compare July 9, 2026 14:11

steipete merged commit 5ef269c into main Jul 9, 2026
104 checks passed

steipete deleted the codex/utf16-boundary-sweep branch July 9, 2026 14:21

github-actions Bot mentioned this pull request Jul 9, 2026

📡 Upstream Digest — 2026-07-09 14:36 UTC curtismercier/openclaw-mods#1243

Open

This was referenced Jul 9, 2026

docs(changelog): credit recent UTF-16 and display fixes #103102

Merged

docs(changelog): credit preserved UTF-16 contributions #103105

Merged

lsr911 mentioned this pull request Jul 10, 2026

fix(qa-lab): use truncateUtf16Safe for tool search gateway and mock OpenAI text truncation #103219

Merged

clawsweeper Bot mentioned this pull request Jul 11, 2026

fix(gateway): keep hook console value truncation UTF-16 safe #104243

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix: keep bounded Unicode text valid across UTF-16 boundaries#102823

fix: keep bounded Unicode text valid across UTF-16 boundaries#102823
steipete merged 3 commits into
mainfrom
codex/utf16-boundary-sweep

steipete commented Jul 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

steipete commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

github-actions Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency Guard

Uh oh!

github-actions Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency graph changes noted

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

steipete commented Jul 9, 2026 •

edited

Loading

github-actions Bot commented Jul 9, 2026 •

edited

Loading

github-actions Bot commented Jul 9, 2026 •

edited

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading