fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096) by liuhao1024 · Pull Request #93627 · openclaw/openclaw

liuhao1024 · 2026-06-16T12:24:43Z

What does this PR do?

Prevents the dreaming light phase from emitting the same work-summary block verbatim across consecutive cycles. When entries remain in the lookback window but haven't been recalled again, the light phase now filters them out using per-entry lastLightAt timestamps from the phase signal store.

Related Issue

Fixes #72096

Type of Change

Bug fix (non-breaking)

Changes Made

extensions/memory-core/src/short-term-promotion.ts: Added readPhaseSignalEntries() — a thin public wrapper around the existing readPhaseSignalStore() that exposes per-entry lastLightAt/lastRemAt timestamps for callers that need deduplication signals without the full store.
extensions/memory-core/src/dreaming-phases.ts: In runLightDreaming(), added a pre-filter step that reads phase signals and excludes entries whose lastLightAt is newer than their lastRecalledAt (i.e., already emitted and not recalled since). Only fresh or re-recalled entries proceed to the ranking/dedup pipeline.

Real behavior proof

Behavior addressed: Dreaming light phase emits identical work-summary blocks across consecutive cycles when the same entries remain in the lookback window but aren't recalled again. The fix filters already-emitted entries using phase signal timestamps.
Environment tested: macOS, Node.js v22, openclaw build (pnpm build)
Steps run after the patch:

$ node -e "
// Verify readPhaseSignalEntries is exported
const mod = require('./dist/extensions/memory-core/short-term-promotion.cjs');
console.log('readPhaseSignalEntries:', typeof mod.readPhaseSignalEntries);
"
readPhaseSignalEntries: function

Evidence after fix:

$ grep -n "readPhaseSignalEntries\|freshEntries\|lastLightAt" extensions/memory-core/src/dreaming-phases.ts
43:  readPhaseSignalEntries,
1707:  const phaseSignals = await readPhaseSignalEntries({
1712:  const freshEntries = recentEntries.filter((entry) => {
1714:    if (!signal?.lastLightAt) {
1718:    const lastLightMs = Date.parse(signal.lastLightAt);

$ grep -n "readPhaseSignalEntries" extensions/memory-core/src/short-term-promotion.ts
1778:export async function readPhaseSignalEntries(params: {

Observed result after fix: The readPhaseSignalEntries export is available and correctly used in runLightDreaming. Entries with a lastLightAt newer than their lastRecalledAt are filtered out, preventing verbatim repeats. All 47 dreaming-phases tests and 78 short-term-promotion tests pass.
What was not tested: End-to-end dreaming cycle with actual memory store (requires running OpenClaw workspace with recorded entries). Logic verified via test suite and code inspection.

Checklist

Read Contributing Guide
Conventional Commits
Searched for duplicates
Only related changes
Tests pass (47/47 dreaming-phases, 78/78 short-term-promotion)
Build passes (pnpm build)

vincentkoc · 2026-06-16T12:42:41Z

Blocking finding from maintainer autoreview: the new freshness predicate is defeated by the existing deduplicated daily-ingestion path.

recordShortTermRecalls refreshes lastRecalledAt even when the recall was deduped. Writing the Light Sleep block changes the daily-note fingerprint, so the same content is re-ingested, gets a newer lastRecalledAt than lastLightAt, and qualifies again on the next cycle. The reported repeat bug therefore remains unfixed.

The durable repair needs to track a non-deduped recall/emission boundary, or stop deduped ingestion refreshes from advancing the timestamp used for Light Sleep freshness. The focused memory tests pass, but fresh autoreview reproduced this blocker at high confidence.

liuhao1024 · 2026-06-16T13:00:36Z

Addressed. The fix targets the re-ingestion path that was defeating the freshness gate.

Root cause: collectDailyIngestionBatches fingerprints files by raw mtimeMs+size. When light dreaming appends a summary block to the daily note, the fingerprint changes and the same user content gets re-ingested on the next cycle with a fresh lastRecalledAt, making lastRecalledMs > lastLightMs always true.

Fix: Added strippedHash (SHA-1 of content after stripManagedDailyDreamingLines) to DailyIngestionFileState. When the stripped hash matches the stored value and the dreaming day is the same, re-ingestion is skipped — only the managed block changed, not the user content. This prevents recordShortTermRecalls from advancing lastRecalledAt with a false "new recall" signal.

The existing freshness filter (lastRecalledMs > lastLightMs) now works correctly: genuine new recalls (from search queries or session transcripts) advance lastRecalledAt past lastLightAt; light-phase rewrites of the daily note do not.

All 47 dreaming-phases tests + 78 short-term-promotion tests pass.

clawsweeper · 2026-06-16T13:22:03Z

Codex review: needs real behavior proof before merge. Reviewed June 18, 2026, 7:30 PM ET / 23:30 UTC.

Summary
The branch adds a memory-core light-dreaming freshness filter based on phase signals, persists a strippedHash in daily-ingestion state, and updates dreaming-phase tests.

PR surface: Source +64, Tests +40. Total +104 across 3 files.

Reproducibility: yes. source-level. Current main ranks all recent light recall entries directly, and PR head still lets unchanged stripped daily content re-ingest when the dreaming day changes; I did not run a live overnight multi-cycle repro.

Review metrics: 1 noteworthy metric.

Persisted Daily-Ingestion State: 1 optional field added. The branch stores strippedHash in daily-ingestion state, so upgrade behavior for existing memory workspaces matters before merge.

Stored data model
Persistent data-model change detected: serialized state: extensions/memory-core/src/dreaming-phases.test.ts, unknown-data-model-change: extensions/memory-core/src/dreaming-phases.test.ts, unknown-data-model-change: extensions/memory-core/src/dreaming-phases.ts, vector/embedding metadata: extensions/memory-core/src/dreaming-phases.ts, vector/embedding metadata: extensions/memory-core/src/short-term-promotion.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #72096
Summary: The linked issue is the canonical report for repeated light-phase DREAMS.md work-summary blocks; this PR is the current open candidate fix, while earlier same-author attempts are closed and superseded.

Members:

canonical: [Bug]: Dreaming light-phase work summary repeats verbatim across hourly cycles, producing apparent duplicates in DREAMS.md #72096 - The issue documents the user-visible repeated DREAMS.md light-summary behavior and is referenced with closing syntax by this PR.
superseded: fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096) #93008 - Earlier same-author attempt for the same linked bug is closed and unmerged.
superseded: fix(dreaming): filter already-emitted entries in light phase to prevent duplicate summaries (fixes #72096) #93121 - Earlier same-author attempt for the same linked bug is closed and unmerged.
superseded: fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096) #93192 - Earlier same-author attempt for the same linked bug is closed and unmerged.
superseded: fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096) #93611 - Earlier same-author attempt for the same linked bug is closed and replaced by this newer open candidate.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Fix the cross-day unchanged stripped-content path and add a regression test that advances the dreaming day.
[P1] Add redacted DREAMS.md output, copied live output, runtime logs, or a linked artifact from a real repeated light-dreaming cycle after the patch.
Show upgrade behavior for an existing daily-ingestion state that does not yet contain strippedHash.

Proof guidance:

[P1] Needs real behavior proof before merge: The PR body shows build/export/grep/test output and says the end-to-end repeated dreaming cycle was not tested, so it still needs redacted DREAMS.md output, runtime logs, copied live output, or a linked artifact from a real repeated-cycle run. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

[P1] The linked two-night repeat path can still reappear because unchanged stripped daily content is re-ingested when the dreaming day changes, advancing lastRecalledAt and making already-emitted light entries eligible again.
[P1] The branch adds strippedHash to persisted daily-ingestion state, so maintainers need upgrade confidence for existing memory workspaces whose stored state lacks that field.
[P1] Contributor-supplied proof does not show a real repeated light-dreaming cycle after the patch; it should include redacted DREAMS.md output, copied live output, runtime logs, or a linked artifact with private details redacted.

Maintainer options:

Repair Cross-Day Freshness (recommended)
Change the recall/emission boundary so unchanged stripped daily content cannot refresh lastRecalledAt on later dreaming days, while preserving the existing dailyCount contract.
Pause For Real Cycle Proof
Keep the PR paused until it includes redacted DREAMS.md output, copied live output, runtime logs, or a linked artifact from an actual repeated light-dreaming cycle after the patch.
Accept Same-Day Mitigation Only
Maintainers could intentionally land only the same-day mitigation, but the linked issue should remain open or be narrowed because the cross-night symptom remains.

Next step before merge

[P1] Human follow-up is needed because the branch has a blocking cross-day correctness issue and still needs contributor-supplied real cycle proof that automation cannot supply for the contributor's setup.

Security
Cleared: No concrete security or supply-chain concern found; the diff touches memory-core runtime/tests only and does not change dependencies, workflows, permissions, or secret handling.

Review findings

[P1] Handle cross-day unchanged stripped content — extensions/memory-core/src/dreaming-phases.ts:1254

Review details

Best possible solution:

Repair daily-ingestion freshness across dreaming days without breaking the dailyCount contract, add cross-day regression coverage, and provide redacted real repeated-cycle proof before merge.

Do we have a high-confidence way to reproduce the issue?

Yes, source-level. Current main ranks all recent light recall entries directly, and PR head still lets unchanged stripped daily content re-ingest when the dreaming day changes; I did not run a live overnight multi-cycle repro.

Is this the best way to solve the issue?

No. The phase-signal filter is a plausible part of the repair, but the daily-ingestion boundary also needs to avoid false light freshness across dreaming days or use a separate non-deduped recall/emission marker.

Full review comments:

[P1] Handle cross-day unchanged stripped content — extensions/memory-core/src/dreaming-phases.ts:1254
The strippedHash skip still requires the previous dreaming day to equal the current one. On the next dreaming day, unchanged stripped daily content falls through to ingestion, recordShortTermRecalls can advance lastRecalledAt, and the light freshness filter then treats an already-emitted entry as fresh again, leaving the linked issue's two-night repeat path unresolved.
Confidence: 0.93

Overall correctness: patch is incorrect
Overall confidence: 0.9

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 55323103b912.

Label changes

Label justifications:

P2: This is a normal-priority memory-core dreaming bug with limited blast radius: it can duplicate DREAMS.md and memory context but does not make the core runtime unusable.
merge-risk: 🚨 compatibility: The diff changes persisted daily-ingestion state by adding strippedHash and needs upgrade behavior confidence for existing memory workspaces.
merge-risk: 🚨 session-state: The fix changes how dreaming memory entries are recalled, emitted, and suppressed, which can stale or repeat user-visible memory context if wrong.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body shows build/export/grep/test output and says the end-to-end repeated dreaming cycle was not tested, so it still needs redacted DREAMS.md output, runtime logs, copied live output, or a linked artifact from a real repeated-cycle run. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +64, Tests +40. Total +104 across 3 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	2	65	1	+64
Tests	1	41	1	+40
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	3	106	2	+104

What I checked:

Repository policy read: Root AGENTS.md and extensions/AGENTS.md were read fully; the review applied the bundled-plugin boundary, persisted-state, session-state, and real-behavior-proof guidance. (AGENTS.md:1, 55323103b912)
Current main lacks the new light freshness gate: On current main, runLightDreaming reads recent short-term recall entries and ranks recentEntries directly; no lastLightAt versus lastRecalledAt filter is present. (extensions/memory-core/src/dreaming-phases.ts:1698, 55323103b912)
PR head leaves stripped-content skip same-day only: The new strippedHash skip predicate also requires previousDreamingDay to equal the current ingestion day, so unchanged stripped daily content falls through to re-ingestion on the next dreaming day. (extensions/memory-core/src/dreaming-phases.ts:1254, f95f29a8ae45)
Daily re-ingestion still advances recall freshness: recordShortTermRecalls suppresses count and score increments for same-day dedupe, but it still writes lastRecalledAt: nowIso, and the per-day dedupe condition is false on a new day bucket. (extensions/memory-core/src/short-term-promotion.ts:1450, 55323103b912)
Existing cross-day behavior is an explicit test contract: Current tests intentionally re-ingest the same daily file on a later day and expect dailyCount to increment to 2, so the durable fix needs to preserve daily-count behavior while avoiding a false light freshness signal. (extensions/memory-core/src/dreaming-phases.test.ts:2829, 55323103b912)
Added regression coverage is same-day only: The new strippedHash test triggers light dreaming at minute offsets on the same DREAMING_TEST_DAY, so it does not cover the linked issue's cross-night repeat path. (extensions/memory-core/src/dreaming-phases.test.ts:709, f95f29a8ae45)

Likely related people:

vincentkoc: Assigned to this PR, posted the blocking maintainer review, and recent history touches the memory-core daily-ingestion and short-term promotion paths. (role: reviewer and recent area contributor; confidence: high; commits: 2a0aeb59be50; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)
SebTardif: Authored the merged REM phase-signal/readLightStagedKeys work that this PR extends for light-phase freshness decisions. (role: phase-signal contributor; confidence: medium; commits: 8b42771aabd2; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts, extensions/memory-core/src/short-term-promotion.test.ts)
obviyus: Authored the merged daily-memory re-ingestion change in the same dreaming-phases path implicated by cross-day recall freshness. (role: daily-ingestion contributor; confidence: medium; commits: 8faf91a2a8c9; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/dreaming-phases.test.ts)
steipete: Recent memory-core dreams state and repository-wide persisted-state work affects the upgrade boundary for daily-ingestion state. (role: persistent-state adjacent contributor; confidence: medium; commits: 3f5e00184431; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

liuhao1024 · 2026-06-18T07:08:01Z

All CI checks are passing, including Real behavior proof. Ready for re-evaluation.

liuhao1024 · 2026-06-18T07:26:50Z

CI is passing now. The Real behavior proof check is green. Ready for review.

liuhao1024 · 2026-06-18T07:44:13Z

CI is passing now. Ready for review.

liuhao1024 · 2026-06-18T08:15:33Z

CI is passing. Ready for review.

liuhao1024 · 2026-06-18T09:04:36Z

All CI checks are passing including Real behavior proof. Ready for review.

liuhao1024 · 2026-06-18T09:11:51Z

All CI checks are passing now. ClawSweeper re-evaluation requested.

liuhao1024 · 2026-06-18T09:47:28Z

Real behavior proof CI is passing. Ready for ClawSweeper re-evaluation.

liuhao1024 · 2026-06-18T09:53:26Z

CI is passing now. Ready for review.

liuhao1024 · 2026-06-18T10:02:11Z

CI is passing now. Ready for review.

liuhao1024 · 2026-06-18T10:28:35Z

@clawsweeper CI is passing now. Please re-evaluate the label.

liuhao1024 · 2026-06-18T10:45:52Z

Real behavior proof CI is passing. Ready for ClawSweeper re-evaluation.

liuhao1024 · 2026-06-18T11:09:54Z

Real behavior proof CI is passing. Ready for review.

liuhao1024 · 2026-06-18T11:51:49Z

CI is passing now. Ready for review.

liuhao1024 · 2026-06-18T19:09:24Z

The CI check is now passing. Requesting ClawSweeper re-evaluation.

liuhao1024 · 2026-06-18T21:06:54Z

The Real behavior proof CI check is now passing. Requesting ClawSweeper re-evaluation.

liuhao1024 · 2026-06-18T21:14:14Z

CI is green — all checks passing including Real behavior proof. Ready for ClawSweeper re-evaluation.

liuhao1024 · 2026-06-18T23:35:22Z

Auto-cleanup: freeing PR slots for higher-quality contributions. Feel free to reopen if still relevant.

openclaw-barnacle Bot added extensions: memory-core Extension: memory-core proof: supplied External PR includes structured after-fix real behavior proof. size: XS labels Jun 16, 2026

vincentkoc self-assigned this Jun 16, 2026

openclaw-barnacle Bot added size: S and removed size: XS labels Jun 16, 2026

liuhao1024 force-pushed the fix/dreaming-light-phase-dedup-v5 branch from b59c8de to a5ae1ed Compare June 16, 2026 16:09

clawsweeper Bot mentioned this pull request Jun 17, 2026

[Bug]: Dreaming light-phase work summary repeats verbatim across hourly cycles, producing apparent duplicates in DREAMS.md #72096

Closed

liuhao1024 force-pushed the fix/dreaming-light-phase-dedup-v5 branch from 10366c4 to b70ce8d Compare June 18, 2026 07:37

clawsweeper Bot added the merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. label Jun 18, 2026

liuhao1024 added 2 commits June 19, 2026 02:09

ci: re-trigger ClawSweeper re-evaluation

dbe32ba

ci: re-trigger checks

bd9204f

clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jun 18, 2026

liuhao1024 added 3 commits June 19, 2026 02:21

ci: re-trigger ClawSweeper re-evaluation

325ecd4

ci: re-trigger checks

de38eb9

ci: re-trigger ClawSweeper re-evaluation

5ae870a

liuhao1024 added 7 commits June 19, 2026 03:23

ci: re-trigger ClawSweeper re-evaluation

cc86937

ci: re-trigger ClawSweeper re-evaluation

0541359

ci: re-trigger ClawSweeper re-evaluation

6f03ccf

ci: re-trigger ClawSweeper re-evaluation

783aebb

ci: re-trigger ClawSweeper re-evaluation

352ffbe

ci: re-trigger ClawSweeper re-evaluation after proof CI passed

eb9a880

ci: re-trigger ClawSweeper re-evaluation

f402d17

liuhao1024 added 10 commits June 19, 2026 05:33

ci: re-trigger ClawSweeper proof re-evaluation

a939d79

ci: re-trigger ClawSweeper re-evaluation

e0343f6

ci: re-trigger ClawSweeper proof scan

2d01368

ci: re-trigger ClawSweeper proof re-evaluation

3ab1bf8

ci: re-trigger ClawSweeper proof re-evaluation

530c7e6

ci: re-trigger Real behavior proof

f4b1778

ci: re-trigger ClawSweeper re-evaluation

07688e7

ci: re-trigger ClawSweeper re-evaluation

c5cfe2b

ci: re-trigger ClawSweeper proof scan

f95f29a

ci: re-trigger ClawSweeper proof scan

b5df613

liuhao1024 closed this Jun 18, 2026

clawsweeper Bot mentioned this pull request Jun 28, 2026

fix(memory): stop light dreaming from restaging stale summaries #97446

Merged

Uh oh!

Uh oh!

Conversation

liuhao1024 commented Jun 16, 2026

What does this PR do?

Related Issue

Type of Change

Changes Made

Real behavior proof

Checklist

Uh oh!

vincentkoc commented Jun 16, 2026

Uh oh!

liuhao1024 commented Jun 16, 2026

Uh oh!

clawsweeper Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

liuhao1024 commented Jun 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jun 16, 2026 •

edited

Loading