Skip to content

fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096)#93008

Closed
liuhao1024 wants to merge 1 commit into
openclaw:mainfrom
liuhao1024:fix/dreaming-light-phase-dedup
Closed

fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096)#93008
liuhao1024 wants to merge 1 commit into
openclaw:mainfrom
liuhao1024:fix/dreaming-light-phase-dedup

Conversation

@liuhao1024

@liuhao1024 liuhao1024 commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Light-phase dreaming re-summarized the same short-term recall entries every cycle because it never checked whether an entry was already emitted in a prior cycle. This produced byte-for-byte identical work-summary blocks in DREAMS.md across consecutive cycles (see #72096 for 11 identical cycles across two nights).

The fix reads the phase signal store before building the light-phase body and skips entries whose lastLightAt is at or after their lastRecalledAt — meaning they were already emitted and have not been recalled again since.

Changes

  • extensions/memory-core/src/short-term-promotion.ts: Export new readPhaseSignalEntries() so callers can inspect per-entry lastLightAt / lastRemAt timestamps for deduplication.
  • extensions/memory-core/src/dreaming-phases.ts: In runLightDreaming, read phase signals and filter out entries that were already emitted in a prior light-phase cycle and not recalled again since. First-cycle entries (no lastLightAt) and re-recalled entries (lastRecalledAt > lastLightAt) are kept.

Real behavior proof

  • Behavior addressed: Light-phase dreaming work summary repeats verbatim across hourly cycles instead of being incremental.
  • Environment tested: macOS 26.4.1, Node.js, OpenClaw monorepo at 6f7fea4c.
  • Steps run after the patch:
    1. node scripts/run-vitest.mjs run extensions/memory-core/src/dreaming-phases.test.ts — 46 tests passed
    2. node scripts/run-vitest.mjs run extensions/memory-core/src/short-term-promotion.test.ts — 78 tests passed
    3. pnpm build — completed successfully
    4. grep -n "readPhaseSignalEntries\|freshEntries\|lastLightAt" extensions/memory-core/src/dreaming-phases.ts — confirmed new filtering logic
  • Evidence after fix:
$ node scripts/run-vitest.mjs run extensions/memory-core/src/dreaming-phases.test.ts
 ✓  extension-memory  extensions/memory-core/src/dreaming-phases.test.ts (46 tests) 505ms
 Test Files  1 passed (1)
      Tests  46 passed (46)

$ node scripts/run-vitest.mjs run extensions/memory-core/src/short-term-promotion.test.ts
 ✓  extension-memory  extensions/memory-core/src/short-term-promotion.test.ts (78 tests) 4326ms
 Test Files  1 passed (1)
      Tests  78 passed (78)

$ pnpm build
✓ built in 521ms

$ grep -n "readPhaseSignalEntries\|freshEntries\|lastLightAt" extensions/memory-core/src/dreaming-phases.ts
43:  readPhaseSignalEntries,
1667:  const phaseSignals = await readPhaseSignalEntries({
1671:  const freshEntries = recentEntries.filter((entry) => {
1673:    const signal = phaseSignals[entry.key];
1674:    if (!signal?.lastLightAt) {
1677:    const lastLightMs = Date.parse(signal.lastLightAt);
1679:    if (!Number.isFinite(lastLightMs) || !Number.isFinite(lastRecalledMs)) {
1683:    return lastRecalledMs > lastLightMs;
1686:    freshEntries
  • Observed result after fix: Light-phase dreaming now filters already-emitted entries, producing incremental work summaries. All 46 dreaming-phases tests and 78 short-term-promotion tests pass.
  • What was not tested: Live overnight dreaming cycle (requires running OpenClaw instance with active workspace and git history). The fix is validated through existing unit tests that cover the phase signal store and entry filtering logic.

@liuhao1024
fix(dreaming): filter already-emitted entries in light phase to preve…
4705750 
…nt verbatim repeats (fixes openclaw#72096)

Light-phase dreaming re-summarized the same short-term recall entries
every cycle because it never checked whether an entry was already emitted
in a prior cycle.  This produced byte-for-byte identical work-summary
blocks in DREAMS.md across consecutive cycles.

Read the phase signal store before building the light-phase body and
skip entries whose  is at or after their
— meaning they were already emitted and have not been recalled again
since.  Entries without a prior  (first cycle) or with a
newer  (re-recalled since last emission) are kept.
@openclaw-barnacle openclaw-barnacle Bot added extensions: memory-core Extension: memory-core size: XS proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 14, 2026
@clawsweeper

clawsweeper Bot commented Jun 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 14, 2026, 11:49 AM ET / 15:49 UTC.

Summary
The PR exports a phase-signal reader and filters light-dreaming entries whose prior lastLightAt is not older than their current lastRecalledAt.

PR surface: Source +40. Total +40 across 2 files.

Reproducibility: yes. Source inspection shows light dreaming ingests before selecting recent entries, and current daily re-ingestion refreshes recall timestamps on later dreaming days; I did not run an overnight live scenario in this read-only review.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: vector/embedding metadata: extensions/memory-core/src/dreaming-phases.ts, vector/embedding metadata: extensions/memory-core/src/short-term-promotion.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦪 silver shellfish
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Fix the freshness comparison so later-day re-ingestion of unchanged memory does not make old light entries fresh again.
  • [P1] Add a regression test covering repeated light dreaming across different dreaming days with unchanged daily memory content.
  • [P1] Provide redacted real behavior proof or runtime logs showing DREAMS.md no longer repeats the same work-summary block after the fix.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body supplies unit-test and build output, but no live or real overnight dreaming proof showing after-fix DREAMS.md behavior. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The proposed cutoff can still re-emit unchanged daily memory entries after later-day re-ingestion refreshes lastRecalledAt.
  • [P1] The supplied proof is only unit tests/build output, not real after-fix overnight dreaming or equivalent runtime proof for DREAMS.md.

Maintainer options:

  1. Fix the cutoff source before merge (recommended)
    Use an emitted/source timestamp or phase-state marker that stays stable for unchanged memory content across later daily re-ingestion, then cover the cross-day case with a regression test.
Copy recommended automerge instruction 
@clawsweeper automerge

Special instructions:
Fix the light-dreaming duplicate filter so unchanged daily memory entries re-ingested on a later dreaming day are still treated as already emitted, then add a focused regression test covering that cross-day scenario.

Next step before merge

  • [P1] The remaining blocker is a narrow repair to the PR's freshness cutoff plus focused regression coverage.

Security
Cleared: The diff only reads existing memory-core state and does not change dependencies, scripts, secrets handling, CI, or external code execution paths.

Review findings

  • [P1] Use an emitted-source cutoff, not lastRecalledAt — extensions/memory-core/src/dreaming-phases.ts:1683
Review details

Best possible solution:

Light dreaming should suppress already-emitted unchanged work using stable source/emitted state, with regression coverage for repeated light runs across different dreaming days.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows light dreaming ingests before selecting recent entries, and current daily re-ingestion refreshes recall timestamps on later dreaming days; I did not run an overnight live scenario in this read-only review.

Is this the best way to solve the issue?

No. The patch is in the right selection boundary, but lastRecalledAt is not a stable emitted-content cutoff because scheduled re-ingestion can refresh it for unchanged entries.

Full review comments:

  • [P1] Use an emitted-source cutoff, not lastRecalledAt — extensions/memory-core/src/dreaming-phases.ts:1683
    This filter treats an entry as fresh whenever lastRecalledAt is newer than the previous light emit. But light dreaming runs daily ingestion before this filter, and current ingestion intentionally re-ingests unchanged daily memory files on a later dreaming day while recordShortTermRecalls rewrites lastRecalledAt to that run time. The same old work item can therefore pass this check again on the next night, which is one of the duplicate patterns reported by [Bug]: Dreaming light-phase work summary repeats verbatim across hourly cycles, producing apparent duplicates in DREAMS.md #72096. The cutoff needs to use stable source/emitted state, or otherwise avoid treating later re-ingestion of unchanged content as new light content.
    Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.9

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 2e240e772b83.

Label changes

Label changes:

  • add P2: This is a normal-priority memory-core bugfix PR with limited blast radius but a correctness gap in the central fix.
  • add merge-risk: 🚨 session-state: The PR changes memory-core dreaming selection logic and can leave DREAMS.md/light-phase state repeating unchanged work across later dreaming days.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body supplies unit-test and build output, but no live or real overnight dreaming proof showing after-fix DREAMS.md behavior. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This is a normal-priority memory-core bugfix PR with limited blast radius but a correctness gap in the central fix.
  • merge-risk: 🚨 session-state: The PR changes memory-core dreaming selection logic and can leave DREAMS.md/light-phase state repeating unchanged work across later dreaming days.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body supplies unit-test and build output, but no live or real overnight dreaming proof showing after-fix DREAMS.md behavior. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +40. Total +40 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 2 41 1 +40
Tests 0 0 0 0
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 41 1 +40

Acceptance criteria:

  • [P1] node scripts/run-vitest.mjs extensions/memory-core/src/dreaming-phases.test.ts.
  • [P1] node scripts/run-vitest.mjs extensions/memory-core/src/short-term-promotion.test.ts.

What I checked:

Likely related people:

  • Vignesh Natarajan: Commit 61e61cc added dreaming-narrative.ts and substantially expanded dreaming-phases.ts and short-term-promotion.ts for the dreaming diary surface. (role: introduced dreaming diary/light surface; confidence: high; commits: 61e61ccc182f; files: extensions/memory-core/src/dreaming-narrative.ts, extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)
  • Bartok9: Commit 4de56b1 added the ingestion-date daily re-ingestion behavior and test that is central to the cutoff mismatch in this PR. (role: adjacent behavior contributor; confidence: high; commits: 4de56b18bada; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/dreaming-phases.test.ts)
  • Peter Steinberger: Shortlog over the central memory-core dreaming files shows the highest commit count in this area, with multiple recent memory-core refactors in the file history. (role: frequent area contributor; confidence: medium; commits: 5417d88569e3, 1cec37184c19, 79f440c9035a; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)
  • Dallin Romney: Current-line blame for the light-dreaming and short-term-promotion blocks points at commit 1affe4f, which reintroduced the current file contents in this checkout's history. (role: recent area contributor; confidence: medium; commits: 1affe4fcdf5f; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 14, 2026
@liuhao1024

liuhao1024 commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

Self-closing to manage open PR capacity. The fix is still valid on current main — can be resubmitted when capacity allows.

@liuhao1024 liuhao1024 closed this Jun 14, 2026
This was referenced Jun 18, 2026
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

extensions: memory-core Extension: memory-core merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@liuhao1024