Skip to content

fix(auto-reply,infra): keep startup context and heartbeat event text UTF-16 safe#102483

Merged
steipete merged 2 commits into
openclaw:mainfrom
wings1029:fix/utf16-safe-startup-heartbeat-truncation
Jul 9, 2026
Merged

fix(auto-reply,infra): keep startup context and heartbeat event text UTF-16 safe#102483
steipete merged 2 commits into
openclaw:mainfrom
wings1029:fix/utf16-safe-startup-heartbeat-truncation

Conversation

@wings1029

Copy link
Copy Markdown
Contributor

What Problem This Solves

trimStartupMemoryContent() in startup-context.ts and buildExecEventPrompt() in heartbeat-events-filter.ts use raw String.prototype.slice() to truncate text. This can split emoji surrogate pairs, producing broken in startup memory context (MEMORY.md content) and async command output relayed via heartbeat events.

Why This Change Was Made

These two functions truncate user-authored content that may contain emoji — startup memory files and async exec command output. They now use truncateUtf16Safe from @openclaw/normalization-core/utf16-slice.

User Impact

  • MEMORY.md startup context truncation no longer risks broken emoji
  • Heartbeat exec event prompts keep emoji intact when truncated at 8,000 chars
  • No behavior change for ASCII text

Evidence

  • 3 new test cases in startup-context.test.ts — verifying truncation with emoji content
  • 2 new test cases in heartbeat-events-filter.test.ts — verifying long event text with emoji stays intact
  • Existing test suites continue to pass

🤖 Generated with Claude Code

@clawsweeper

clawsweeper Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 9, 2026, 3:08 AM ET / 07:08 UTC.

Summary
The PR replaces two prompt-context truncation sites with truncateUtf16Safe and adds focused tests for startup memory and heartbeat exec-event prompt truncation.

PR surface: Source +2, Tests +38. Total +40 across 4 files.

Reproducibility: yes. source inspection gives a high-confidence current-main reproduction path: call the affected truncation functions with content whose cutoff lands between a surrogate pair, and raw slice() can return a dangling surrogate. I did not run tests locally because this review was read-only.

Review metrics: none identified.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Add after-fix real behavior proof, such as terminal output or redacted logs from a startup memory or heartbeat exec-output path with emoji at the truncation boundary.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body and CI provide unit-test/check evidence only; before merge, the contributor should add a redacted live run, terminal output, logs, or equivalent runtime proof showing the changed truncation behavior after the fix. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The contributor has not posted after-fix real behavior proof; tests and CI support the patch, but an external PR still needs a live run, terminal output, redacted logs, or equivalent runtime evidence before merge.

Maintainer options:

  1. Decide the mitigation before merge
    Land the narrow helper-based fix after the contributor adds real behavior proof showing a long startup memory or heartbeat exec-output truncation with emoji stays well-formed, or after a maintainer explicitly overrides the proof gate.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] No automated repair is needed; the next action is contributor real-behavior proof or maintainer proof override before ordinary merge review.

Security
Cleared: No concrete security or supply-chain concern found; the diff only swaps two internal string truncation calls to an existing workspace helper and adds tests.

Review details

Best possible solution:

Land the narrow helper-based fix after the contributor adds real behavior proof showing a long startup memory or heartbeat exec-output truncation with emoji stays well-formed, or after a maintainer explicitly overrides the proof gate.

Do we have a high-confidence way to reproduce the issue?

Yes, source inspection gives a high-confidence current-main reproduction path: call the affected truncation functions with content whose cutoff lands between a surrogate pair, and raw slice() can return a dangling surrogate. I did not run tests locally because this review was read-only.

Is this the best way to solve the issue?

Yes, this is the narrowest maintainable fix shape: use the existing truncateUtf16Safe helper at the two raw prefix-truncation sites while preserving current limits and markers. The PR tests are useful, though real behavior proof is still missing.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 4fa3118049c1.

Label changes

Label changes:

  • add P2: This is a focused bug fix for malformed model/user-visible text at truncation boundaries with limited blast radius and no evidence of an urgent live outage.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body and CI provide unit-test/check evidence only; before merge, the contributor should add a redacted live run, terminal output, logs, or equivalent runtime proof showing the changed truncation behavior after the fix. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This is a focused bug fix for malformed model/user-visible text at truncation boundaries with limited blast radius and no evidence of an urgent live outage.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body and CI provide unit-test/check evidence only; before merge, the contributor should add a redacted live run, terminal output, logs, or equivalent runtime proof showing the changed truncation behavior after the fix. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +2, Tests +38. Total +40 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 5 3 +2
Tests 2 39 1 +38
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 44 4 +40

What I checked:

  • Current startup-context behavior: trimStartupMemoryContent trims startup memory and truncates over-limit content with raw String.prototype.slice, so a cutoff inside a surrogate pair can produce a dangling surrogate in the model-visible startup prelude. (src/auto-reply/reply/startup-context.ts:92, 4fa3118049c1)
  • Current heartbeat exec-event behavior: buildExecEventPrompt still truncates long exec event text with raw rawEventText.slice(0, MAX_EXEC_EVENT_PROMPT_CHARS) before appending the truncation marker. (src/infra/heartbeat-events-filter.ts:124, 4fa3118049c1)
  • Existing helper contract: truncateUtf16Safe delegates to sliceUtf16Safe, which adjusts the end boundary when the limit would leave a dangling high surrogate; the helper tests include truncating at a surrogate-pair boundary. (packages/normalization-core/src/utf16-slice.ts:43, 4fa3118049c1)
  • Startup caller path: Bare /new and /reset paths call buildSessionStartupContextPrelude, and buildReplyPromptEnvelopeBase places that prelude into the effective model body for a reset/startup turn. (src/auto-reply/reply/get-reply-run.ts:768, 4fa3118049c1)
  • Heartbeat caller path: Heartbeat preflight collects exec completion events and passes them into buildExecEventPrompt, which becomes the heartbeat run prompt. (src/infra/heartbeat-runner.ts:1262, 4fa3118049c1)
  • PR diff review: The live PR diff changes only the two truncation call sites plus adjacent tests; it adds no dependency, workflow, config, lockfile, storage, auth, or security-boundary surface. (53892cc55d97)

Likely related people:

  • Takhoffman: GitHub path history shows Takhoffman introduced and refined startup-context runtime behavior, including startup memory loading, bounds, and UTC/local date handling. (role: startup context feature history; confidence: high; commits: 94340b959830, 4f00b769251d, 06ff594a3e99; files: src/auto-reply/reply/startup-context.ts)
  • steipete: GitHub path history shows steipete split and maintained heartbeat event filtering and later extracted the normalization-core package that owns the UTF-16 helper used here. (role: heartbeat and normalization area contributor; confidence: high; commits: 5a431f57fc18, e2362d352d14, 65c9eddae87e; files: src/infra/heartbeat-events-filter.ts, packages/normalization-core/src/utf16-slice.ts)
  • GodsBoy: GitHub path history ties GodsBoy to the exec completion payload behavior that feeds the heartbeat prompt path touched by this PR. (role: exec completion prompt contributor; confidence: medium; commits: 349749f73dee; files: src/infra/heartbeat-events-filter.ts)
  • mmaps: GitHub path history shows mmaps extended exec completion detection for local background exec formats, which is adjacent to the prompt path under review. (role: exec completion detection contributor; confidence: medium; commits: 19a2e9ddb5a8; files: src/infra/heartbeat-events-filter.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jul 9, 2026
@steipete

steipete commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Land-ready at exact head b99c82b99e82be99672353d868ed41d0fa2bac9e.

Improvements made during review:

  • kept the startup truncation helper private instead of exporting it only for tests;
  • replaced both original not.toContain("�") checks, which cannot detect lone UTF-16 surrogates, with exact production-path boundary assertions;
  • verified the startup-memory path at its configured per-file cap and the exec-event path at its 8,000-code-unit cap.

Proof:

  • node scripts/run-vitest.mjs src/auto-reply/reply/startup-context.test.ts src/infra/heartbeat-events-filter.test.ts — 50 + 19 tests passed.
  • git diff --check passed.
  • Exact-head CI run 29000841897 passed.
  • Fresh exact-head autoreview found no actionable findings at 0.99 confidence.

No known proof gaps.

@steipete
steipete merged commit f0cc5e5 into openclaw:main Jul 9, 2026
99 checks passed
@steipete

steipete commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 9, 2026
…UTF-16 safe (openclaw#102483)

* fix(auto-reply,infra): keep startup context and heartbeat event text UTF-16 safe

* test: strengthen UTF-16 truncation coverage

---------

Co-authored-by: Peter Steinberger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal backlog priority with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants