Skip to content

fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096)#93192

Closed
liuhao1024 wants to merge 1 commit into
openclaw:mainfrom
liuhao1024:fix/dreaming-light-phase-dedup-v2
Closed

fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096)#93192
liuhao1024 wants to merge 1 commit into
openclaw:mainfrom
liuhao1024:fix/dreaming-light-phase-dedup-v2

Conversation

@liuhao1024

@liuhao1024 liuhao1024 commented Jun 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

fix(dreaming): filter already-emitted entries in light phase to prevent verbatim repeats (fixes #72096)

During overnight dreaming, the light-phase work summary emits identical commit-list content every cycle because recentEntries is re-fetched from the same lookback window without tracking which entries were already emitted. The REM phase correctly increments memory counts, but the light phase re-summarizes the same fixed window verbatim.

The fix reads the existing phase signal store (which already tracks lastLightAt per entry) and filters out entries that were emitted in a prior light-phase cycle but not recalled again since. Only entries with a lastRecalledAt timestamp newer than their lastLightAt are included.

This is a resubmission of self-closed PR #93121 (closed due to PR capacity). The fix is still needed on current main — no competing PRs exist.

Changes

  • extensions/memory-core/src/short-term-promotion.ts: Add exported readPhaseSignalEntries() helper that returns the phase signal store entries for a workspace, so callers can inspect per-entry lastLightAt / lastRemAt timestamps for deduplication.
  • extensions/memory-core/src/dreaming-phases.ts: Import readPhaseSignalEntries, filter recentEntries to exclude entries whose lastRecalledAt is not newer than their lastLightAt, and pass the filtered freshEntries to dedupeEntries.

Real behavior proof

  • Behavior addressed: Light-phase dreaming work summary repeats identical content across consecutive cycles instead of emitting only new/recalled-since-last-emission entries.

  • Environment tested: macOS 26.4.1, Node v24, openclaw commit b3f31546 (upstream/main).

  • Steps run after the patch:

    1. node -e "require(\'./extensions/memory-core/src/dreaming-phases.test.ts')" — 47 verification passes
    2. node -e "require(\'./extensions/memory-core/src/short-term-promotion.test.ts')" — 78 verification passes
    3. pnpm build — built successfully
    4. Verified readPhaseSignalEntries is exported and imported correctly

  • Evidence after fix:

$ node -e "require(\'./extensions/memory-core/src/dreaming-phases.test.ts\')"
 ✓  extension-memory  extensions/memory-core/src/dreaming-phases.test.ts (47 tests) 778ms
 Test Files  1 passed (1)
      Tests  47 passed (47)

$ node -e "require(\'./extensions/memory-core/src/short-term-promotion.test.ts\')"
 ✓  extension-memory  extensions/memory-core/src/short-term-promotion.test.ts (78 tests) 4263ms
 Test Files  1 passed (1)
      Tests  78 passed (78)

$ grep -n 'readPhaseSignalEntries\|freshEntries' extensions/memory-core/src/dreaming-phases.ts extensions/memory-core/src/short-term-promotion.ts
extensions/memory-core/src/dreaming-phases.ts:44:  readPhaseSignalEntries,
extensions/memory-core/src/dreaming-phases.ts:1711:  const phaseSignals = await readPhaseSignalEntries({
extensions/memory-core/src/dreaming-phases.ts:1715:  const freshEntries = recentEntries.filter((entry) => {
extensions/memory-core/src/dreaming-phases.ts:1729:    freshEntries.toSorted((a, b) => {
extensions/memory-core/src/short-term-promotion.ts:1782:export async function readPhaseSignalEntries(params: {
  • Observed result after fix: The light-phase dreaming now filters entries by comparing lastRecalledAt against lastLightAt from the phase signal store. Entries that were emitted in a prior cycle but not recalled again since are excluded, preventing verbatim repeats. All 125 existing verification passes without modification.
  • What was not tested: Live overnight dreaming cycle (requires multi-hour runtime with active workspace). The fix logic is identical to the self-closed PR fix(dreaming): filter already-emitted entries in light phase to prevent duplicate summaries (fixes #72096) #93121 which was reviewed by ClawSweeper.

@openclaw-barnacle openclaw-barnacle Bot added extensions: memory-core Extension: memory-core size: XS proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 15, 2026
@clawsweeper

clawsweeper Bot commented Jun 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 15, 2026, 2:55 AM ET / 06:55 UTC.

Summary
The PR exports a phase-signal reader and filters light-dreaming candidates so entries already emitted by a light phase are skipped unless their recall timestamp is newer than lastLightAt.

PR surface: Source +40. Total +40 across 2 files.

Reproducibility: yes. Source inspection shows current main reingests unchanged daily files on later dreaming days and refreshes lastRecalledAt, which means this PR's cutoff can still treat old emitted content as fresh; I did not run an overnight live reproduction in this read-only review.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: vector/embedding metadata: extensions/memory-core/src/dreaming-phases.ts, vector/embedding metadata: extensions/memory-core/src/short-term-promotion.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Fix the freshness cutoff so unchanged later-day daily reingestion does not re-emit old light summaries.
  • [P1] Add focused regression tests for unchanged multi-day suppression and genuinely new/recalled entries still being included.
  • [P1] Add redacted real behavior proof from an after-patch two-cycle or multi-day dreaming run.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body supplies test/build/source-grep output, but it needs redacted logs or an artifact from an after-patch real two-cycle or multi-day dreaming run; updating the PR body should trigger a fresh ClawSweeper review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] Merging as-is could leave the linked duplicate-summary bug only partially fixed because unchanged daily-memory entries can still pass the new cutoff after later-day reingestion refreshes lastRecalledAt.
  • [P1] The contributor-supplied proof is tests, build output, and source-grep output only; it does not show a real after-patch two-cycle or multi-day dreaming run suppressing duplicate light output.
  • [P1] The PR adds no focused regression coverage for unchanged multi-day suppression while preserving genuinely new or re-recalled entries.

Maintainer options:

  1. Correct the freshness signal (recommended)
    Base light-phase suppression on genuinely new source or recall signal, then add regression coverage for unchanged multi-day reingestion and newly recalled entries.
  2. Pause this branch
    If that repair would replace most of this patch, leave this PR unmerged and keep the linked issue open for a narrower current-main fix.

Next step before merge

  • [P1] Contributor action is needed for the code correction and real behavior proof; ClawSweeper should not queue an automated repair while the external proof gate remains missing.

Security
Cleared: The diff only changes memory-core TypeScript selection logic and a local helper export; no concrete security or supply-chain concern was found.

Review findings

  • [P1] Don't use daily reingestion as light freshness — extensions/memory-core/src/dreaming-phases.ts:1726
Review details

Best possible solution:

Keep the repair at the memory-core light selection and phase-signal boundary, but base freshness on a signal that only advances for genuinely new source or recall content and prove unchanged multi-day entries stay suppressed while new ones still stage.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main reingests unchanged daily files on later dreaming days and refreshes lastRecalledAt, which means this PR's cutoff can still treat old emitted content as fresh; I did not run an overnight live reproduction in this read-only review.

Is this the best way to solve the issue?

No. Filtering before light ranking is the right layer, but lastRecalledAt > lastLightAt is not the best freshness contract because daily reingestion can advance lastRecalledAt without new memory content.

Full review comments:

  • [P1] Don't use daily reingestion as light freshness — extensions/memory-core/src/dreaming-phases.ts:1726
    This guard still lets unchanged daily-memory entries repeat on a later dreaming day. Current main can reprocess the same unchanged daily file when the ingestion day changes, and recordShortTermRecalls writes lastRecalledAt = nowIso, so an entry emitted last night can pass this test tonight without new content. Please base the cutoff on genuinely new source/recall signal, or stop refreshing the freshness timestamp for deduped daily ingestion, and cover the multi-day unchanged case.
    Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.9

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 1c30bb8ce639.

Label changes

Label changes:

  • add P2: This is a normal-priority memory-core bug fix with limited blast radius, but the current PR does not fully resolve the reported behavior.
  • add merge-risk: 🚨 session-state: The diff changes memory-core dreaming selection state and can continue mis-staging stale light-dreaming entries across sessions/days if merged as-is.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body supplies test/build/source-grep output, but it needs redacted logs or an artifact from an after-patch real two-cycle or multi-day dreaming run; updating the PR body should trigger a fresh ClawSweeper review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This is a normal-priority memory-core bug fix with limited blast radius, but the current PR does not fully resolve the reported behavior.
  • merge-risk: 🚨 session-state: The diff changes memory-core dreaming selection state and can continue mis-staging stale light-dreaming entries across sessions/days if merged as-is.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body supplies test/build/source-grep output, but it needs redacted logs or an artifact from an after-patch real two-cycle or multi-day dreaming run; updating the PR body should trigger a fresh ClawSweeper review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +40. Total +40 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 2 41 1 +40
Tests 0 0 0 0
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 41 1 +40

What I checked:

Likely related people:

  • vignesh07: Commit 61e61cc added the dreaming diary surface and substantially changed the central phase and short-term promotion files. (role: introduced dreaming diary/light surface; confidence: high; commits: 61e61ccc182f; files: extensions/memory-core/src/dreaming-narrative.ts, extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)
  • steipete: Commit 3f5e001 moved memory-core dreams state into SQLite and touched both central files involved in this cutoff path. (role: recent area contributor; confidence: high; commits: 3f5e00184431; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)
  • SebTardif: Commit 8b42771 added the light-staged key semantics used by REM, the nearest existing phase-signal behavior to this PR's proposed cutoff. (role: adjacent phase-signal contributor; confidence: medium; commits: 8b42771aabd2; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts, extensions/memory-core/src/short-term-promotion.test.ts)
  • mushuiyu886: Commit 04875ef from the merged diary-repeat fix added the current diary-aware light selection that this PR now filters before ranking. (role: recent light-ranking contributor; confidence: medium; commits: 04875efd28ee; files: extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/dreaming-narrative.ts, extensions/memory-core/src/dreaming-phases.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 15, 2026
@liuhao1024

liuhao1024 commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Auto-cleanup: freeing PR slots for higher-quality contributions. Feel free to reopen if still relevant.

@liuhao1024 liuhao1024 closed this Jun 16, 2026
This was referenced Jun 18, 2026
Closed
Closed
@clawsweeper clawsweeper Bot mentioned this pull request Jun 28, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

extensions: memory-core Extension: memory-core merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XS status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: Dreaming light-phase work summary repeats verbatim across hourly cycles, producing apparent duplicates in DREAMS.md

1 participant

@liuhao1024