fix(memory-core): treat dreaming fence marker lines as inside-fence in promotion guard (#80613) by grifjef · Pull Request #83718 · openclaw/openclaw

grifjef · 2026-05-18T17:58:28Z

Summary

Problem. lineRangeOverlapsDreamingFence in extensions/memory-core/src/short-term-promotion.ts tracked insideFence state from the fence marker lines but never flagged ranges that included the marker lines themselves. A relocated promotion range that ended on a  line, began on a  line, or covered only a marker line passed the guard, and the snippet built from those raw lines carried the marker text into MEMORY.md.
Why it matters. The guard's contract is to refuse rehydrated ranges that overlap managed dreaming fences so dream artifacts cannot be promoted. Treating the marker lines themselves as outside-the-fence leaves a narrow but real leak: any window the relocator picks that clips a marker produces a snippet that includes the literal  text, and (when the window straddles content on either side of the marker) the adjacent line. This is the residual edge case noted by ClawSweeper on fix(memory-core): redact managed dreaming blocks during promotion rehydration (#80613) #80702 ("the managed-block leak half is related but should be handled by a focused memory-core promotion fix"), narrowed to the guard itself rather than a parallel sanitizer.
What changed. extensions/memory-core/src/short-term-promotion.ts: marker lines are now checked against the range bounds; if a marker is inside the relocated range, the range is rejected. The pre-existing inside-fence tracking remains so interior lines between a start/end pair are still flagged. The contract elsewhere — the contamination predicate, the relocator, the apply path, the MEMORY.md writer — is untouched.
What did not change (scope boundary). No change to relocateCandidateRange, isContaminatedDreamingSnippet, applyShortTermPromotions, buildDailySnippetChunks, stripManagedDailyDreamingLines, the CJK dedupe path (the second sub-bug of [Bug]: dreaming pipeline leaks raw candidate content into MEMORY.md and CJK dedup is ineffective in tokenizeSnippet #80613 is being addressed separately by fix(memory-core): use CJK-aware tokenizer for dreaming dedupe (#80613) #80620), or any MEMORY.md rendering. No new sanitizer is introduced — this is a narrowing of the existing guard.

Change Type (select all)

Scope (select all touched areas)

Linked Issue/PR

Refs [Bug]: dreaming pipeline leaks raw candidate content into MEMORY.md and CJK dedup is ineffective in tokenizeSnippet #80613 (does not fully close; the dramatic full-Candidate leak from the issue body is already prevented by the merged fence-overlap guard in fix(memory-core): prevent staged dream candidates from leaking into MEMORY.md #68774. This PR narrows a residual marker-line edge case in the same guard.)
Related: fix(memory-core): use CJK-aware tokenizer for dreaming dedupe (#80613) #80620 (CJK dedupe half, separate PR), fix(memory-core): redact managed dreaming blocks during promotion rehydration (#80613) #80702 (closed sanitizer approach, intentionally not duplicated here).
This PR fixes a bug or regression

Real behavior proof (required for external PRs)

Behavior addressed: lineRangeOverlapsDreamingFence now flags relocated promotion ranges that include a dreaming fence marker line, preventing the marker text (and any adjacent fenced content captured by the same raw-line window) from being promoted into MEMORY.md.
Real environment tested: Local OpenClaw source checkout on macOS (Apple Silicon, Node v22.14.0, pnpm 11.1.0). The fix is mechanical and exercised by Vitest unit tests against the helper directly.
Exact steps or command run after this patch:
- pnpm test extensions/memory-core/src/short-term-promotion.test.ts -t "lineRangeOverlapsDreamingFence" — targeted run of the affected describe block.
- pnpm test extensions/memory-core/src/short-term-promotion.test.ts — full file regression.
- pnpm test extensions/memory-core/src/dreaming-phases.test.ts — adjacent file regression.
- pnpm format:check extensions/memory-core/src/short-term-promotion.ts extensions/memory-core/src/short-term-promotion.test.ts — format on changed files.
- pnpm lint extensions/memory-core/src/short-term-promotion.ts extensions/memory-core/src/short-term-promotion.test.ts — lint on changed files.
- pnpm tsgo:extensions and pnpm tsgo:extensions:test — typecheck.

Evidence after fix:

Before the source change, the four added tests reproduce the gap (output captured locally):

Tests  4 failed | 4 passed | 50 skipped (58)
× returns true when the range ends on a Light Sleep start marker
× returns true when the range begins on a Light Sleep end marker
× returns true when the range covers only a marker line
× returns true for REM marker single-line ranges even with no body between markers

Each failure form: expected false to be true, confirming that the pre-patch guard returns false for a range that includes only a marker line.

After the source change:

Tests  8 passed | 50 skipped (58)            # lineRangeOverlapsDreamingFence -t filter
Tests  58 passed (58)                         # full short-term-promotion.test.ts
Tests  41 passed (41)                         # full dreaming-phases.test.ts

Format/lint/typecheck on changed files: all clean. (Repo-wide format reports 34 pre-existing issues on main unrelated to this PR.)

Observed result after fix: the helper rejects any rehydrated range that touches a  or  marker line. No regressions in the existing lineRangeOverlapsDreamingFence describe block or the broader test files.
What was not tested: An end-to-end repro via applyShortTermPromotions that drives relocateCandidateRange to organically pick a marker-line-bordering window. Probe attempts on current main could not construct an organic trigger — the relocator's compare/quality scoring rarely prefers such windows when cleaner alternatives exist in the file — so the demonstration is at the unit level. The repo-wide format/lint/build/CI suite was not run locally; the changed files pass all relevant lanes individually.

clawsweeper · 2026-05-18T18:00:05Z

Codex review: needs maintainer review before merge. Reviewed July 4, 2026, 3:55 PM ET / 19:55 UTC.

Summary
The PR changes memory-core short-term promotion so lineRangeOverlapsDreamingFence rejects ranges that include dreaming fence marker lines, with unit and apply-path regression coverage.

PR surface: Source +6, Tests +118. Total +124 across 2 files.

Reproducibility: yes. Current main’s source shows marker lines are skipped before range-bound checks, and the added tests give a direct marker-only range plus a record/rank/apply path that would fail before the patch.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: unknown-data-model-change: extensions/memory-core/src/short-term-promotion.test.ts, vector/embedding metadata: extensions/memory-core/src/short-term-promotion.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] Live GitHub status still showed a few routine required checks in progress during this read-only review; that is merge gating, not a code finding or diff-caused automation risk.

Maintainer options:

Decide the mitigation before merge
Land the narrow guard change after normal required checks finish, keeping the rehydrate-time fence-overlap boundary instead of adding a second sanitizer or new configuration surface.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

No ClawSweeper repair job is needed; the remaining action is normal PR merge gating after required checks settle.

Security
Cleared: The diff only changes deterministic memory promotion filtering and tests; it does not touch dependencies, lockfiles, secrets, CI, permissions, networking, or code execution surfaces.

Review details

Best possible solution:

Land the narrow guard change after normal required checks finish, keeping the rehydrate-time fence-overlap boundary instead of adding a second sanitizer or new configuration surface.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main’s source shows marker lines are skipped before range-bound checks, and the added tests give a direct marker-only range plus a record/rank/apply path that would fail before the patch.

Is this the best way to solve the issue?

Yes. The existing guard already owns rehydrate-time managed-fence rejection, so treating marker lines as managed content there is narrower and cleaner than changing chunking, adding a broad sanitizer, or introducing policy/config.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d5badb9e3604.

Label changes

Label changes:

add proof: sufficient: Contributor real behavior proof is sufficient. PR comments include before/after unit output and apply-path proof, and maintainer validation reports focused Testbox plus path-scoped checks; the current head’s memory-core diff matches the validated head.
add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): PR comments include before/after unit output and apply-path proof, and maintainer validation reports focused Testbox plus path-scoped checks; the current head’s memory-core diff matches the validated head.

Label justifications:

P2: The PR fixes a real memory-core dreaming promotion bug with limited blast radius to managed dreaming marker leakage into MEMORY.md.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): PR comments include before/after unit output and apply-path proof, and maintainer validation reports focused Testbox plus path-scoped checks; the current head’s memory-core diff matches the validated head.
proof: sufficient: Contributor real behavior proof is sufficient. PR comments include before/after unit output and apply-path proof, and maintainer validation reports focused Testbox plus path-scoped checks; the current head’s memory-core diff matches the validated head.

Evidence reviewed

PR surface:

Source +6, Tests +118. Total +124 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	13	7	+6
Tests	1	118	0	+118
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	131	7	+124

What I checked:

Root review policy read: Root AGENTS.md was read in full, including the ClawSweeper PR review policy requiring source, caller/callee, tests, current-main behavior, and proof review before verdict. (AGENTS.md:1, d5badb9e3604)
Scoped extension policy read: extensions/AGENTS.md was read and confirms bundled plugin boundary guidance; this PR stays inside the memory-core plugin and does not add SDK or dependency surface. (extensions/AGENTS.md:1, d5badb9e3604)
Current-main gap: On current main, the guard toggles insideFence and continues on start/end marker lines before checking whether the marker line itself is inside the relocated range, so marker-only ranges are not rejected. (extensions/memory-core/src/short-term-promotion.ts:2270, d5badb9e3604)
Patch behavior: At PR head, the guard computes the one-indexed line number before marker handling and returns true when a start or end marker lies inside the checked range. (extensions/memory-core/src/short-term-promotion.ts:2270, 820e818183f2)
Caller path checked: applyShortTermPromotions rehydrates selected candidates through rehydratePromotionCandidate, which applies lineRangeOverlapsDreamingFence before allowing candidates to be appended to MEMORY.md. (extensions/memory-core/src/short-term-promotion.ts:2470, d5badb9e3604)
Sibling contamination path checked: The existing contamination predicate filters raw dreaming/session metadata, but bare marker text lacks the composite Candidate/confidence/evidence/status/recalls signature, so the fence guard is the right boundary for this edge case. (extensions/memory-core/src/short-term-promotion.ts:414, d5badb9e3604)

Likely related people:

solomonneas: Authored the merged PR that added the staged-dream contamination hardening and rehydration-time dreaming fence overlap guard this PR narrows. (role: introduced related guard; confidence: high; commits: 314903417fcc, 5862c3ff7193, 06339212e4ac; files: extensions/memory-core/src/short-term-promotion.ts, extensions/memory-core/src/short-term-promotion.test.ts)
vignesh07: GitHub commit metadata maps the original dreaming promotion implementation history to vignesh07, which is the broader feature area this guard protects. (role: feature origin; confidence: medium; commits: 4c1022c73b39; files: extensions/memory-core/src/short-term-promotion.ts, extensions/memory-core/src/dreaming-phases.ts)
mbelinky: GitHub commit metadata maps the REM preview and safe promotion replay work to mbelinky, adjacent to the promotion apply and replay path reviewed here. (role: adjacent feature contributor; confidence: medium; commits: 79348f73c8b6; files: extensions/memory-core/src/short-term-promotion.ts)
vincentkoc: Recent local history shows repeated memory-core work by Vincent Koc, and the PR discussion includes maintainer validation of the exact memory-core diff and focused Testbox proof. (role: recent area contributor and reviewer; confidence: medium; commits: 0609bf858175, 077cfca22972, e085fa1a3ffd; files: extensions/memory-core/src/short-term-promotion.ts, extensions/memory-core/src/dreaming-phases.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (1 earlier review cycle)

reviewed 2026-07-01T06:27:51.224Z sha cf8981c :: needs real behavior proof before merge. :: none

grifjef · 2026-05-19T13:10:52Z

Thanks @clawsweeper. Addressed both findings:

Type blocker. Rebased onto upstream/main and updated the new test cases to use the testing alias (replacing the old __testing references from the original branch, before the rename in 4f4d108639 merged). node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json is now clean.
Real promotion-path proof. Added an integration test does not promote rehydrated candidates whose relocated range covers a managed dreaming fence marker line (#80613) that drives the real applyShortTermPromotions path. Pre-patch the same input writes  into MEMORY.md as a Promoted From Short-Term Memory entry (applied=1); post-patch the range is rejected (applied=0, no MEMORY.md written). Diagnostic before/after MEMORY.md contents are inlined in the updated PR body.

Full local test run on rebased cf8981c8: Tests 59 passed (59) for short-term-promotion.test.ts and 41 passed (41) for dreaming-phases.test.ts. Format and tsgo:extensions:test clean on changed files.

@clawsweeper re-review please.

RomneyDa · 2026-05-29T06:27:14Z

Heads up: this PR needs to be updated against current main before the new required Dependency Guard check can pass.

vincentkoc · 2026-07-04T19:48:37Z

Maintainer validation is complete.

Rebased exact head: 820e818183f20b5d21cd84eb2a54f1a8896c94ca
The existing promotion guard now treats managed dreaming start/end marker lines as fenced content.
The regression drives record, rank, and apply and proves marker-only relocated ranges do not reach MEMORY.md.
Blacksmith Testbox tbx_01kwqa1vxt051333faa64wwhgz: 141/141 focused memory-core tests passed.
Path-scoped pnpm check:changed passed extension typechecks, lint, database-first, runtime sidecar, and import-cycle guards.
Fresh autoreview 019f2ea4-39a3-7aa2-8938-23f1623b1555: no findings.
Exact-head GitHub CI run 28717706633: all relevant checks passed.
Native review artifacts and scripts/pr prepare-run 83718 validated successfully.

No remaining proof gaps.

…n promotion guard The lineRangeOverlapsDreamingFence guard tracked insideFence state from the marker lines but did not flag ranges that included the marker lines themselves. A relocated promotion range that ended on a start marker, began on an end marker, or covered only a marker line passed the guard, and the snippet built from those raw lines carried the marker text into MEMORY.md. Treat marker lines as inside-fence content for range overlap purposes and add unit + integration coverage. The integration test exercises the real applyShortTermPromotions path: pre-patch a recall whose stored snippet is the start-marker text produces a 'Promoted From Short-Term Memory' entry containing the literal marker; post-patch the same input yields applied=0 and no MEMORY.md write. Refs openclaw#80613.

clawsweeper · 2026-07-04T19:51:21Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

vincentkoc · 2026-07-04T20:01:09Z

Merged via squash.

Prepared head SHA: 820e818183f20b5d21cd84eb2a54f1a8896c94ca
Landed commit: 7a0188cbd20d304a015106cb6d0335649610c95e

…n promotion guard (openclaw#83718) The lineRangeOverlapsDreamingFence guard tracked insideFence state from the marker lines but did not flag ranges that included the marker lines themselves. A relocated promotion range that ended on a start marker, began on an end marker, or covered only a marker line passed the guard, and the snippet built from those raw lines carried the marker text into MEMORY.md. Treat marker lines as inside-fence content for range overlap purposes and add unit + integration coverage. The integration test exercises the real applyShortTermPromotions path: pre-patch a recall whose stored snippet is the start-marker text produces a 'Promoted From Short-Term Memory' entry containing the literal marker; post-patch the same input yields applied=0 and no MEMORY.md write. Refs openclaw#80613.

openclaw-barnacle Bot added extensions: memory-core Extension: memory-core size: S proof: supplied External PR includes structured after-fix real behavior proof. labels May 18, 2026

clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels May 18, 2026

grifjef force-pushed the fix/dreaming-fence-marker-lines branch from f9fcc43 to cf8981c Compare May 19, 2026 13:09

clawsweeper Bot mentioned this pull request May 20, 2026

fix(memory-core): use CJK-aware tokenizer for dreaming dedupe (#80613) #80620

Closed

This was referenced Jun 20, 2026

Dreaming promotion still writes raw/junk data into MEMORY.md (regression from #67580) #89444

Open

fix(memory): skip raw snippets during promotion #94636

Merged

fix(memory): skip empty dreaming placeholders #80916

Open

vincentkoc self-assigned this Jul 4, 2026

vincentkoc force-pushed the fix/dreaming-fence-marker-lines branch from cf8981c to ae8d61c Compare July 4, 2026 19:39

openclaw-barnacle Bot added scripts Repository scripts agents Agent runtime and tooling labels Jul 4, 2026

openclaw-barnacle Bot added extensions: firecrawl size: XL and removed size: S labels Jul 4, 2026

vincentkoc force-pushed the fix/dreaming-fence-marker-lines branch from c66c664 to 820e818 Compare July 4, 2026 19:49

openclaw-barnacle Bot added size: S and removed scripts Repository scripts agents Agent runtime and tooling extensions: firecrawl size: XL labels Jul 4, 2026

clawsweeper Bot removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 4, 2026

vincentkoc merged commit 7a0188c into openclaw:main Jul 4, 2026
123 of 131 checks passed

github-actions Bot mentioned this pull request Jul 4, 2026

📡 Upstream Digest — 2026-07-04 20:44 UTC curtismercier/openclaw-mods#1208

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(memory-core): treat dreaming fence marker lines as inside-fence in promotion guard (#80613)#83718

fix(memory-core): treat dreaming fence marker lines as inside-fence in promotion guard (#80613)#83718
vincentkoc merged 1 commit into
openclaw:mainfrom
grifjef:fix/dreaming-fence-marker-lines

grifjef commented May 18, 2026

Uh oh!

clawsweeper Bot commented May 18, 2026 •

edited

Loading

Uh oh!

grifjef commented May 19, 2026

Uh oh!

RomneyDa commented May 29, 2026

Uh oh!

vincentkoc commented Jul 4, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 4, 2026

Uh oh!

Uh oh!

vincentkoc commented Jul 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

grifjef commented May 18, 2026

Summary

Change Type (select all)

Scope (select all touched areas)

Linked Issue/PR

Real behavior proof (required for external PRs)

Uh oh!

clawsweeper Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grifjef commented May 19, 2026

Uh oh!

RomneyDa commented May 29, 2026

Uh oh!

vincentkoc commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented Jul 4, 2026

Uh oh!

Uh oh!

vincentkoc commented Jul 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

clawsweeper Bot commented May 18, 2026 •

edited

Loading

vincentkoc commented Jul 4, 2026 •

edited

Loading