fix(active-memory): recall context breaks when recent turns contain emoji by zhangguiping-xydt · Pull Request #102877 · openclaw/openclaw

zhangguiping-xydt · 2026-07-09T14:04:35Z

What Problem This Solves

Fixes two active-memory paths that could pass malformed text onward when a configured UTF-16 code-unit cap landed inside an emoji or another surrogate-pair character:

recent user or assistant turns embedded in the model-visible recall context; and
timeout-recovered partial assistant transcripts.

The timeout path folds the distinct useful work from same-author PR #102920 into this active same-plugin fix after that PR was closed only by the contributor queue limit.

Why This Change Was Made

Active-memory already bounds both surfaces, but their raw string slices could retain one half of a surrogate pair. The fix reuses the existing truncateUtf16Safe contract at each bounded append.

For timeout transcripts, each already-trimmed record and separator is accounted before append. Collection stops at the first truncation or surrogate backoff so later records cannot skip ahead, and the redundant final map/filter/slice path is deleted. The configured limits and recall query shape remain unchanged.

User Impact

Users can include emoji and other non-BMP characters in recent conversation turns and timeout-recovered partial replies without active-memory corrupting those characters before memory search or partial-result recovery.

Evidence

Final exact head: c0d70155b31638f8c4a3a7a359204b904b3e92af (GitHub-verified signed maintainer follow-up; original author credited).

Current-main inspection confirmed raw UTF-16 slicing in both recent recall snippets and partial transcript collection. Caller/callee review traced the resulting text through before_prompt_build, timeout recovery, and the embedded-agent prompt.
Focused RED control: Crabbox run_c9984877a13a. Reversing only the production-file PR diff made all three exact regressions fail with lone-surrogate output.
Exact-head GREEN proof: Crabbox run_2d2505291a1a.
- /usr/local/bin/pnpm test extensions/active-memory/index.test.ts -- -t "keeps recent conversation context UTF-16 well-formed|keeps partial assistant transcript caps UTF-16 safe|keeps joined partial assistant transcript caps UTF-16 safe": 3/3 passed.
- /usr/local/bin/pnpm test extensions/active-memory/index.test.ts: 161/161 passed.
Both isolated AWS runs used public networking, no Tailscale, no instance profile, pinned Node 24/pnpm, and exact-SHA verification. The lease was stopped afterward.
Static proof: node_modules/.bin/oxfmt --check extensions/active-memory/index.test.ts extensions/active-memory/index.ts and git diff --check passed.
Fresh autoreview reported no actionable findings (0.98 confidence); an independent same-plugin review also found the fold to be the best fix.
Exact-head CI: 58 passed, 0 failed, 0 pending.

No known proof gaps.

clawsweeper · 2026-07-09T14:31:02Z

Codex review: needs maintainer review before merge. Reviewed July 9, 2026, 11:32 AM ET / 15:32 UTC.

Summary
The PR replaces raw active-memory UTF-16 slicing in recent recall snippets and timeout partial transcripts with truncateUtf16Safe, plus emoji-boundary regression tests.

PR surface: Source +6, Tests +87. Total +93 across 2 files.

Reproducibility: yes. Current main and the latest release both show raw UTF-16 code-unit slicing in the recent recall and timeout partial transcript paths, and the PR proof includes a RED control for the emoji-boundary regressions.

Review metrics: none identified.

Root-cause cluster
Relationship: canonical
Canonical: #102877
Summary: This PR is the canonical active-memory UTF-16 truncation fix because it covers both recent recall snippets and timeout partial transcripts; the timeout-only PRs overlap the same root cause.

Members:

superseded: fix(active-memory): keep timeout partial replies valid at emoji caps #102940 - The open timeout-only PR covers the partial-transcript half that this PR now includes with broader same-plugin coverage and stronger proof.
superseded: fix(active-memory): keep timeout partial replies valid at emoji caps #102920 - The closed unmerged timeout-only PR proposed the same partial-transcript repair that this PR folded into its final head.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster ✨ media proof bonus
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] The branch is behind current main, so landing should use a rebase, merge queue, or exact merge-result validation rather than relying only on the stale base SHA.

Maintainer options:

Decide the mitigation before merge
Land this plugin-local fix after exact-head or merge-result validation, then close or supersede the timeout-only duplicate at fix(active-memory): keep timeout partial replies valid at emoji caps #102940.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

No ClawSweeper repair job is needed because the PR already contains the narrow implementation, regression coverage, and real behavior proof; remaining action is normal maintainer landing validation.

Security
Cleared: Cleared: the diff only changes active-memory string truncation and regression tests, with no dependency, workflow, credential, package, or code-execution surface added.

Review details

Best possible solution:

Land this plugin-local fix after exact-head or merge-result validation, then close or supersede the timeout-only duplicate at #102940.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main and the latest release both show raw UTF-16 code-unit slicing in the recent recall and timeout partial transcript paths, and the PR proof includes a RED control for the emoji-boundary regressions.

Is this the best way to solve the issue?

Yes. Reusing the existing truncateUtf16Safe helper at the bounded append sites is the narrowest maintainable fix, and folding the timeout-only duplicate avoids two competing active-memory fixes.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 3c048ef05297.

Label changes

Label changes:

add P2: This is a focused user-visible active-memory text-corruption bug with limited blast radius and a narrow plugin-local fix.
add proof: sufficient: Contributor real behavior proof is sufficient. The PR body and maintainer proof comment include exact-head Crabbox linked artifacts, focused terminal test output, a RED control, and successful exact-head CI for this non-visual behavior.
add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (linked_artifact): The PR body and maintainer proof comment include exact-head Crabbox linked artifacts, focused terminal test output, a RED control, and successful exact-head CI for this non-visual behavior.

Label justifications:

P2: This is a focused user-visible active-memory text-corruption bug with limited blast radius and a narrow plugin-local fix.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (linked_artifact): The PR body and maintainer proof comment include exact-head Crabbox linked artifacts, focused terminal test output, a RED control, and successful exact-head CI for this non-visual behavior.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body and maintainer proof comment include exact-head Crabbox linked artifacts, focused terminal test output, a RED control, and successful exact-head CI for this non-visual behavior.

Evidence reviewed

PR surface:

Source +6, Tests +87. Total +93 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	16	10	+6
Tests	1	88	1	+87
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	104	11	+93

What I checked:

Repository policy read: Root AGENTS.md and extensions/AGENTS.md were read; the review applied the extension boundary and whole-path PR review rules. (AGENTS.md:1, 3c048ef05297)
Current main still has the bug surface: Current main still truncates timeout partial assistant text and joined text with raw slice, which can leave a surrogate half at a configured UTF-16 code-unit cap. (extensions/active-memory/index.ts:2254, 3c048ef05297)
Recent recall path still uses raw slices on main: Current main still slices recent user and assistant snippets with raw slice(0, recentUserChars) and slice(0, recentAssistantChars) before embedding them in recall context. (extensions/active-memory/index.ts:2643, 3c048ef05297)
Runtime entry point checked: The before_prompt_build hook extracts recent turns, builds the active-memory query, and passes it into recall resolution, so the malformed snippet would become model-visible context. (extensions/active-memory/index.ts:3693, 3c048ef05297)
Timeout caller checked: Timeout recovery reads partial assistant text from the active-memory session file and turns it into the timeout partial summary, so the partial transcript path is user-visible in timeout recovery. (extensions/active-memory/index.ts:2355, 3c048ef05297)
Helper contract checked: truncateUtf16Safe floors and bounds the limit, then delegates to sliceUtf16Safe, which adjusts cut points away from surrogate-pair halves. (packages/normalization-core/src/utf16-slice.ts:44, 3c048ef05297)

Likely related people:

Takhoffman: The merged Active Memory recall plugin PR added the bundled plugin and its central tests, including the active-memory implementation now being fixed. (role: introduced behavior and feature owner; confidence: high; commits: b83726d13e33; files: extensions/active-memory/index.ts, extensions/active-memory/index.test.ts)
steipete: The live PR is assigned to steipete, and the final PR head includes steipete-authored test hardening and timeout-partial fix commits on this same plugin path. (role: assigned reviewer and recent follow-up owner; confidence: high; commits: 2ebf0d61eff6, 98ae0ec3b337, c0d70155b316; files: extensions/active-memory/index.ts, extensions/active-memory/index.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (1 earlier review cycle)

reviewed 2026-07-09T14:51:23.613Z sha 98ae0ec :: needs maintainer review before merge. :: none

steipete · 2026-07-09T14:51:57Z

Land-ready proof for exact head c0d70155b31638f8c4a3a7a359204b904b3e92af:

The production change applies the existing UTF-16-safe truncation contract to recent user/assistant recall snippets and timeout-recovered partial assistant transcripts. It also folds the distinct useful work from closed PR fix(active-memory): keep timeout partial replies valid at emoji caps #102920 into this same-author, same-plugin fix.
Timeout transcript collection now safely bounds each already-trimmed record, stops at the first truncation/backoff to preserve prefix order, and directly joins the accounted chunks. The redundant final map/filter/slice path was deleted.
Regression coverage exercises the registered before_prompt_build hook plus exact single-record and multi-record timeout boundaries, including the case where a separator leaves only one UTF-16 code unit before an emoji.
Focused RED control: Crabbox run run_c9984877a13a. Reversing only the production-file PR diff made all three regressions fail with lone surrogate output, as expected.
Exact-head GREEN proof: Crabbox run run_2d2505291a1a. /usr/local/bin/pnpm test extensions/active-memory/index.test.ts -- -t "keeps recent conversation context UTF-16 well-formed|keeps partial assistant transcript caps UTF-16 safe|keeps joined partial assistant transcript caps UTF-16 safe" passed 3/3, then /usr/local/bin/pnpm test extensions/active-memory/index.test.ts passed 161/161 tests.
Both isolated AWS runs used public networking, no Tailscale, no instance profile, pinned Node 24/pnpm, and verified the exact PR SHA. The lease was stopped after proof.
Static proof: node_modules/.bin/oxfmt --check extensions/active-memory/index.test.ts extensions/active-memory/index.ts and git diff --check passed.
Fresh autoreview on the final head reported no actionable findings (0.98 confidence); an independent same-plugin review also found the fold to be the best fix.
Exact-head CI is green: 58 passed, 0 failed, 0 pending.
GitHub verifies the final signed maintainer follow-up commit as valid, with the original author credited.

No known proof gaps.

Co-authored-by: 张贵萍0668001030 <[email protected]>

steipete · 2026-07-09T17:03:39Z

Merged via squash.

Prepared head SHA: c0d70155b31638f8c4a3a7a359204b904b3e92af
Landed commit: f1e7081b4d36287026c872dafa5c86f20a712699

…moji (openclaw#102877) * fix(active-memory): keep recall query snippets utf16-safe * test(active-memory): exercise recall hook for utf16 bounds * test(active-memory): detect only unpaired surrogates * fix(active-memory): bound timeout partials safely Co-authored-by: 张贵萍0668001030 <[email protected]> --------- Co-authored-by: Peter Steinberger <[email protected]>

fix(active-memory): keep recall query snippets utf16-safe

15f78e2

openclaw-barnacle Bot added the size: XS label Jul 9, 2026

steipete self-assigned this Jul 9, 2026

test(active-memory): exercise recall hook for utf16 bounds

2ebf0d6

openclaw-barnacle Bot added size: S and removed size: XS labels Jul 9, 2026

test(active-memory): detect only unpaired surrogates

98ae0ec

zhangguiping-xydt mentioned this pull request Jul 9, 2026

fix(active-memory): keep timeout partial replies valid at emoji caps #102920

Closed

zhangguiping-xydt mentioned this pull request Jul 9, 2026

fix(active-memory): keep timeout partial replies valid at emoji caps #102940

Closed

fix(active-memory): bound timeout partials safely

c0d7015

Co-authored-by: 张贵萍0668001030 <[email protected]>

steipete merged commit f1e7081 into openclaw:main Jul 9, 2026
125 of 130 checks passed

github-actions Bot mentioned this pull request Jul 9, 2026

📡 Upstream Digest — 2026-07-09 17:48 UTC curtismercier/openclaw-mods#1244

Open

steipete mentioned this pull request Jul 9, 2026

docs(changelog): credit recent UTF-16 and display fixes #103102

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(active-memory): recall context breaks when recent turns contain emoji#102877

fix(active-memory): recall context breaks when recent turns contain emoji#102877
steipete merged 4 commits into
openclaw:mainfrom
zhangguiping-xydt:fix/problem-active-memory-codepoint-recall-query

zhangguiping-xydt commented Jul 9, 2026 •

edited by steipete

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

steipete commented Jul 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

zhangguiping-xydt commented Jul 9, 2026 • edited by steipete Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

zhangguiping-xydt commented Jul 9, 2026 •

edited by steipete

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

steipete commented Jul 9, 2026 •

edited

Loading