fix(dreaming): filter already-emitted entries in light phase to prevent duplicate summaries (fixes #72096) by liuhao1024 · Pull Request #93121 · openclaw/openclaw

liuhao1024 · 2026-06-15T02:02:56Z

Summary

Filter already-emitted entries in the dreaming light phase to prevent the work-summary block from repeating verbatim across consecutive cycles when the same entries remain within the lookback window.

The dreaming light phase reads recent short-term recall entries and generates a work-summary block. Without deduplication against prior emissions, the same entries produce identical summaries cycle after cycle — wasting tokens and confusing agents that see repeated context.

Fix

Two changes:

extensions/memory-core/src/short-term-promotion.ts: Export readPhaseSignalEntries() — a thin wrapper around the existing readPhaseSignalStore() that returns the raw Record<string, ShortTermPhaseSignalEntry> so callers can inspect per-entry lastLightAt / lastRemAt timestamps.
extensions/memory-core/src/dreaming-phases.ts: In runLightDreaming(), filter recentEntries through phaseSignals before passing to dedupeEntries(). An entry is included only if:
- It has never been emitted in a light phase (!signal?.lastLightAt), OR
- It was recalled again after the last emission (lastRecalledMs > lastLightMs)

Real behavior proof

Behavior addressed: Dreaming light phase emits duplicate work-summary blocks across consecutive cycles for the same entries
Environment tested: macOS, Node.js, local OpenClaw build (pnpm build)
Steps run after the patch:
1. Ran node scripts/run-vitest.mjs run extensions/memory-core/src/dreaming-phases.test.ts — 46 tests passed
2. Ran node scripts/run-vitest.mjs run extensions/memory-core/src/short-term-promotion.test.ts — 78 tests passed
3. Ran pnpm build — built successfully
4. Verified readPhaseSignalEntries is exported from short-term-promotion.ts
5. Verified filtering logic in dreaming-phases.ts uses lastRecalledMs > lastLightMs guard
Evidence after fix:

$ node scripts/run-vitest.mjs run extensions/memory-core/src/dreaming-phases.test.ts
 ✓  extension-memory  extensions/memory-core/src/dreaming-phases.test.ts (46 tests) 524ms
 Test Files  1 passed (1)
      Tests  46 passed (46)

$ node scripts/run-vitest.mjs run extensions/memory-core/src/short-term-promotion.test.ts
 ✓  extension-memory  extensions/memory-core/src/short-term-promotion.test.ts (78 tests) 4217ms
 Test Files  1 passed (1)
      Tests  78 passed (78)

$ pnpm build
✓ built in 500ms

$ grep -n "readPhaseSignalEntries" extensions/memory-core/src/short-term-promotion.ts
1778:export async function readPhaseSignalEntries(params: {

$ grep -n "freshEntries\|lastLightAt\|lastRecalledMs > lastLightMs" extensions/memory-core/src/dreaming-phases.ts
1670:  const freshEntries = recentEntries.filter((entry) => {
1676:    const lastLightMs = Date.parse(signal.lastLightAt);
1677:    const lastRecalledMs = Date.parse(entry.lastRecalledAt);
1682:    return lastRecalledMs > lastLightMs;

Observed result after fix: All 124 related tests pass. The readPhaseSignalEntries function is properly exported and the filtering logic correctly excludes entries that were emitted in a prior light-phase cycle without being recalled again.
What was not tested: Live dreaming cycle behavior (requires long-running gateway session with memory-core enabled). The test suite covers the filtering logic and store read paths.

clawsweeper · 2026-06-15T02:05:09Z

Codex review: needs real behavior proof before merge. Reviewed June 15, 2026, 12:03 AM ET / 04:03 UTC.

Summary
The PR filters memory-core light-dreaming candidates against phase-signal lastLightAt timestamps and exports a helper for reading raw phase-signal entries.

PR surface: Source +40. Total +40 across 2 files.

Reproducibility: yes. source-reproducible: current daily ingestion can reprocess unchanged files on a later dreaming day and recordShortTermRecalls refreshes lastRecalledAt, which this PR treats as fresh input. I did not run a live overnight gateway reproduction in this read-only review.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: vector/embedding metadata: extensions/memory-core/src/dreaming-phases.ts, vector/embedding metadata: extensions/memory-core/src/short-term-promotion.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Fix the freshness cutoff so unchanged daily-memory re-ingestion across dreaming days does not re-emit old light summaries.
[P1] Add focused regression coverage for unchanged entries staying suppressed and genuinely re-recalled entries still being included.
[P1] Add redacted real behavior proof from a two-cycle or multi-day dreaming run; terminal logs, copied live output, linked artifacts, or recordings are fine after private details are removed.

Proof guidance:

[P1] Needs real behavior proof before merge: The PR body supplies tests/build/source-grep output, but it still needs redacted logs or an artifact from an after-patch real two-cycle or multi-day dreaming run; after updating the PR body, ClawSweeper should rerun automatically or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

[P1] Merging this as-is could leave the linked duplicate-summary bug only partially fixed, because unchanged daily-memory entries can still be treated as fresh on a later dreaming day.
[P1] The contributor-supplied proof is test/build/source-grep output only; it does not show an after-patch real dreaming run suppressing duplicate light output.

Maintainer options:

Decide the mitigation before merge
Keep the fix at the light selection and phase-signal boundary, but base freshness on genuinely new recall/source signal rather than daily re-ingestion timestamps, with focused regression coverage for unchanged and newly re-recalled entries.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P1] Contributor action is needed for both the code correction and real behavior proof; ClawSweeper should not request a repair marker while the external proof gate is still missing.

Security
Cleared: The diff only changes memory-core TypeScript selection logic and a local helper export; no concrete security or supply-chain concern was found.

Review findings

[P1] Don't use daily reingestion as light freshness — extensions/memory-core/src/dreaming-phases.ts:1682

Review details

Best possible solution:

Keep the fix at the light selection and phase-signal boundary, but base freshness on genuinely new recall/source signal rather than daily re-ingestion timestamps, with focused regression coverage for unchanged and newly re-recalled entries.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: current daily ingestion can reprocess unchanged files on a later dreaming day and recordShortTermRecalls refreshes lastRecalledAt, which this PR treats as fresh input. I did not run a live overnight gateway reproduction in this read-only review.

Is this the best way to solve the issue?

No. Filtering before dedupeEntries is the right layer, but lastRecalledAt > lastLightAt is not the best freshness contract because daily re-ingestion can advance lastRecalledAt without new memory content.

Full review comments:

[P1] Don't use daily reingestion as light freshness — extensions/memory-core/src/dreaming-phases.ts:1682
This guard still lets unchanged daily-memory entries repeat on the first light run of a later dreaming day. ingestDailyMemorySignals reprocesses unchanged files when lastDreamingDayIngested changes, and recordShortTermRecalls writes lastRecalledAt = nowIso even when the per-day dedupe path suppresses count and score changes, so an entry emitted yesterday can pass this test tonight without any new content. Please base the cutoff on a signal that only advances for genuinely new recall/source input, or stop refreshing lastRecalledAt for deduped daily ingestion, and cover the multi-day unchanged case from the linked report.
Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.9

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against cc954798f22c.

Label changes

Label justifications:

P2: This is a normal-priority memory-core bug fix with limited blast radius, but the current PR does not fully resolve the reported behavior.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body supplies tests/build/source-grep output, but it still needs redacted logs or an artifact from an after-patch real two-cycle or multi-day dreaming run; after updating the PR body, ClawSweeper should rerun automatically or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +40. Total +40 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	2	41	1	+40
Tests	0	0	0	0
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	41	1	+40

What I checked:

Repository policy read: Root AGENTS.md and scoped extensions/AGENTS.md were read fully; the bundled-plugin and read-beyond-diff review guidance applies to the memory-core files touched here. (AGENTS.md:1, cc954798f22c)
PR changes light freshness cutoff: The diff adds a phase-signal read before light selection and includes entries when lastRecalledAt is later than lastLightAt. (extensions/memory-core/src/dreaming-phases.ts:1682, ee2b04a382b1)
Daily ingestion can refresh unchanged files on a later dreaming day: Current main skips an unchanged daily memory file only when lastDreamingDayIngested matches the current ingestion day, so the first light run on a later dreaming day can enqueue unchanged snippets again. (extensions/memory-core/src/dreaming-phases.ts:1217, cc954798f22c)
Deduped daily signals still advance recall time: recordShortTermRecalls suppresses count and score increments when dedupeByQueryPerDay matches, but still writes lastRecalledAt: nowIso, which the PR then treats as fresh light input. (extensions/memory-core/src/short-term-promotion.ts:1450, cc954798f22c)
Linked bug includes multi-night repeats: The referenced report says the same work-summary block repeated across two nights, so suppressing only same-day repeats is not enough to prove the central bug fixed.
Current line provenance: Blame ties the current daily ingestion and recall timestamp behavior to d085143 on current main. (extensions/memory-core/src/short-term-promotion.ts:1481, d0851435e81c)

Likely related people:

vignesh07: Commit 61e61cc added the dreaming diary surface and changed the central dreaming phase and short-term promotion files. (role: introduced dreaming diary/light surface; confidence: high; commits: 61e61ccc182f; files: extensions/memory-core/src/dreaming-narrative.ts, extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)
steipete: Commit 3f5e001 moved memory-core dreams state into SQLite and touched both central files involved in the proposed cutoff. (role: recent area contributor; confidence: high; commits: 3f5e00184431; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)
SebTardif: Commit 8b42771 added light-staged key semantics for REM, the nearest existing phase-signal behavior to this PR's cutoff. (role: adjacent phase-signal contributor; confidence: medium; commits: 8b42771aabd2; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts, extensions/memory-core/src/short-term-promotion.test.ts)
zhang-guiping: Blame on current main points the daily ingestion and recall timestamp lines used by this review to commit d085143. (role: current line blame; confidence: medium; commits: d0851435e81c; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

…nt duplicate summaries (fixes openclaw#72096)

vincentkoc · 2026-06-15T05:31:11Z

Closing this branch rather than resolving its conflict into main. It replaces the newer diary-aware selection work from #91225, and its lastRecalledAt > lastLightAt rule still admits unchanged content when daily inputs are re-ingested on a later dreaming day. That misses the linked multi-night reproduction.

#72096 stays open for a current-main repair that suppresses already-emitted unchanged content without deleting the newer diary-coverage seam or starving entries with genuinely new signal.

openclaw-barnacle Bot added extensions: memory-core Extension: memory-core size: XS proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 15, 2026

fix(dreaming): filter already-emitted entries in light phase to preve…

ee2b04a

…nt duplicate summaries (fixes openclaw#72096)

liuhao1024 force-pushed the fix/dreaming-light-phase-dedup-v3 branch from 76f4433 to ee2b04a Compare June 15, 2026 03:58

vincentkoc self-assigned this Jun 15, 2026

vincentkoc closed this Jun 15, 2026

liuhao1024 mentioned this pull request Jun 15, 2026

fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096) #93192

Closed

This was referenced Jun 18, 2026

[Bug]: Dreaming light-phase work summary repeats verbatim across hourly cycles, producing apparent duplicates in DREAMS.md #72096

Closed

fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096) #93627

Closed

clawsweeper Bot mentioned this pull request Jun 28, 2026

fix(memory): stop light dreaming from restaging stale summaries #97446

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(dreaming): filter already-emitted entries in light phase to prevent duplicate summaries (fixes #72096)#93121

fix(dreaming): filter already-emitted entries in light phase to prevent duplicate summaries (fixes #72096)#93121
liuhao1024 wants to merge 1 commit into
openclaw:mainfrom
liuhao1024:fix/dreaming-light-phase-dedup-v3

liuhao1024 commented Jun 15, 2026

Uh oh!

clawsweeper Bot commented Jun 15, 2026 •

edited

Loading

Uh oh!

vincentkoc commented Jun 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

liuhao1024 commented Jun 15, 2026

Summary

Fix

Real behavior proof

Uh oh!

clawsweeper Bot commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentkoc commented Jun 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jun 15, 2026 •

edited

Loading