investigate(memory-core): root-cause for #84882 dreaming silent daily-file deletion by googlerest · Pull Request #94625 · openclaw/openclaw

googlerest · 2026-06-18T15:34:10Z

Related: #84882

What Problem This Solves

memory-core Dreaming silently deletes daily memory files (memory/YYYY-MM-DD.md) — 46+ files vanished in one incident, then a second incident (May 30) wiped 171 -> 7 files. #94537 already landed atomic-write hardening for the single current-day write path, but that PR explicitly does not identify the deletion mechanism, and its fix shape (single-file crash-truncation) doesn't match the reported symptom (many distinct historical dates vanishing in one run).

This PR is a root-cause hunt that did not find the mechanism yet, plus diagnostic instrumentation to catch it red-handed on the next reproduction.

Why This Change Was Made

What I checked and ruled out:

repairShortTermPromotionArtifacts (the exact log line implicated: "normalized recall artifacts before dreaming") only mutates the recall store (JSON/SQLite via plugin state), never touches memory/*.md.
scrubDreamingNarrativeArtifacts only renames *.jsonl session transcripts, gated by filename suffix — cannot touch .md files.
applyShortTermPromotions only writes MEMORY.md (long-term), never deletes daily files.
writeDailyDreamingPhaseBlock (the function fix(memory-core): harden dreaming daily-file writes and drain dangling recall refs #94537 hardened) only ever touches today's single file per call — cannot explain ~46 distinct historical dates disappearing in one run.
Traced the anomalous memory/2026-05-29-1740.md-style files from the issue thread to src/hooks/bundled/session-memory/handler.ts (the /new//reset session-memory hook) — confirmed intentional behavior, not data loss, but it does pollute memory/ with non-YYYY-MM-DD.md-shaped files.
Read the full @openclaw/fs-safe write path used by that hook (root.write -> pinned write helper -> Python os.unlink(basename, dir_fd=...)) end to end per this repo's dependency-contract-proof rule. Every mutation is single-basename-scoped via dir_fd; no directory-wide glob/cleanup logic anywhere in the chain.

None of the above can produce the reported symptom. Root cause remains unknown.

What this PR adds (diagnostic only): extensions/memory-core/src/dreaming-integrity-guard.ts snapshots top-level memory/*.md filenames + size + mtime before and after each per-workspace dreaming pass (dreaming.ts). If any file present in the "before" snapshot is absent in the "after" snapshot, it logs logger.error with the full list (name, size, mtime), so the next occurrence leaves a log trail pointing at which dreaming run it happened in, rather than relying on the reporter's own external backup/cron diffing.

Never throws: snapshot failures (including "directory does not exist") are treated as non-fatal and just skip the check for that run.
Never mutates anything — read-only fs.readdir/fs.stat.
Does not change dreaming behavior or output in the non-buggy case.

User Impact

No user-visible behavior change. This is read-only diagnostic instrumentation; dreaming runs produce identical output. The only observable difference is an additional logger.error entry in the rare case where a top-level memory/*.md file disappears between dreaming snapshots — which is exactly the bug this is trying to catch.

Evidence

node scripts/run-vitest.mjs extensions/memory-core/src/dreaming-integrity-guard.test.ts extensions/memory-core/src/dreaming.test.ts
# 2 files, 56 tests passed

pnpm tsgo:extensions:test
# clean

New tests cover: missing memory/ dir (ok, empty snapshot), filtering to top-level .md only, detecting a removed file between snapshots, no false positives when nothing changed, and no false positives when either snapshot failed.

Real environment tested: macOS arm64 (M4), Darwin 25.5.0, Node v25.9.0, branch investigate/84882-dreaming-file-deletion, commit 494317b886 (rebased onto upstream/main).

Status

Still draft / WIP. This does not fix #84882 — it's instrumentation to catch the actual mechanism red-handed on the next reproduction. Will convert from draft once either (a) the integrity log fires and points at a real culprit, or (b) enough reproductions accumulate to rule this approach out.

clawsweeper · 2026-06-18T15:48:38Z

Codex review: needs real behavior proof before merge. Reviewed July 1, 2026, 2:40 PM ET / 18:40 UTC.

Summary
The PR adds read-only memory-core Dreaming diagnostics that snapshot top-level memory/*.md files before and after each workspace run and log files that disappear.

PR surface: Source +95, Tests +68. Total +163 across 3 files.

Reproducibility: yes. for the PR defect by source inspection: the after-snapshot is inside the success path and skipped when an earlier awaited Dreaming step throws. No high-confidence current-main reproduction exists yet for the linked mass-deletion root cause.

Review metrics: 1 noteworthy metric.

Diagnostic snapshots: 2 per successful workspace run. The new guard scans memory markdown files around Dreaming, so maintainers need the run-boundary coverage to be correct before merge.

Stored data model
Persistent data-model change detected: serialized state: extensions/memory-core/src/dreaming-integrity-guard.test.ts, unknown-data-model-change: extensions/memory-core/src/dreaming-integrity-guard.test.ts, unknown-data-model-change: extensions/memory-core/src/dreaming-integrity-guard.ts, unknown-data-model-change: extensions/memory-core/src/dreaming.ts, vector/embedding metadata: extensions/memory-core/src/dreaming-integrity-guard.test.ts, vector/embedding metadata: extensions/memory-core/src/dreaming-integrity-guard.ts, and 1 more. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #84882
Summary: This PR is a diagnostic candidate for the canonical memory-core daily-file deletion issue; related repair/hardening PRs overlap the area but do not replace this before/after deletion diagnostic.

Members:

canonical: [Bug]: memory-core Dreaming normalized recall artifacts silently deletes daily memory files (memory/YYYY-MM-DD.md) #84882 - The PR body, closing reference, labels, and issue discussion identify the open daily-memory deletion report as the central problem.
partial_overlap: fix(memory-core): harden dreaming daily-file writes and drain dangling recall refs #94537 - The closed-unmerged hardening PR targeted atomic writes and dangling recall references in the same symptom area, but did not add this per-workspace deletion diagnostic.
adjacent_distinct: fix(memory-core): keep daily ingestion outside session repair #93389 - The open daily-ingestion repair PR touches memory-core Dreaming repair state, but clears SQLite ingestion namespaces rather than detecting top-level daily file deletion during Dreaming.
adjacent_distinct: [Bug]: memory-core dreaming promotion always finds 0 candidates - recallCount never accumulates (normalized recall artifacts resets scores) #64068 - The closed issue involved the same normalization log area and recall-store rewriting, but tracked a distinct promotion-count symptom that was separately fixed.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🦐 gold shrimp
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Move the post-run comparison so it also runs after failed workspace steps.
[P1] Add focused regression coverage for a seeded daily memory file disappearing before a later workspace-step failure.
Post redacted real OpenClaw Dreaming logs or terminal output showing the diagnostic after the patch.

Proof guidance:

[P1] Needs real behavior proof before merge: The PR body/comment provide unit, lint, and typecheck output plus environment details, but no redacted real OpenClaw Dreaming run log or terminal proof after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

[P1] The diagnostic can miss the exact disappearance window if a daily file vanishes and a later awaited Dreaming step throws before the success-path after-snapshot runs.
[P1] The PR still lacks redacted real OpenClaw Dreaming logs, terminal output, recording, or a linked artifact showing the diagnostic after the patch.
[P1] The branch context says the companion hardening PR already landed, but live GitHub shows that PR closed unmerged, and current main has newer split failure handling that this branch needs to preserve.

Maintainer options:

Fix And Prove The Run Boundary (recommended)
Move the after-snapshot comparison so it also runs after failed workspace steps, add focused regression coverage, and require redacted real Dreaming logs before merge.
Accept Success-Path Diagnostics Explicitly
Maintainers may intentionally accept success-path-only logging, but should do so only after real-run proof shows that narrower diagnostic is useful and understood.
Pause Until Root Cause Is Known
If maintainers want a root-cause fix rather than instrumentation, keep the canonical data-loss issue as the active work item and pause this PR.

Next step before merge

[P1] This external PR is blocked on contributor real behavior proof and a run-boundary patch defect, so it needs human review rather than cleanup closure or repair automation.

Security
Cleared: The diff adds read-only filesystem snapshots, logging, and tests under memory-core; no concrete secrets, dependency, workflow, package, or code-execution regression was found.

Review findings

[P2] Check integrity after failed workspace runs — extensions/memory-core/src/dreaming.ts:697-703

Review details

Best possible solution:

Move the comparison into a failure-safe per-workspace boundary while keeping it read-only, then require redacted real Dreaming proof before merge.

Do we have a high-confidence way to reproduce the issue?

Yes for the PR defect by source inspection: the after-snapshot is inside the success path and skipped when an earlier awaited Dreaming step throws. No high-confidence current-main reproduction exists yet for the linked mass-deletion root cause.

Is this the best way to solve the issue?

No. The diagnostic is a plausible investigation aid, but the best version must compare snapshots even when a workspace run fails and then prove that behavior in a real Dreaming run.

Full review comments:

[P2] Check integrity after failed workspace runs — extensions/memory-core/src/dreaming.ts:697-703
The after-snapshot and diff run only after every awaited Dreaming phase, repair, promotion, report, and narrative step succeeds. If a daily file disappears and a later step throws, control jumps to catch before the comparison, so the diagnostic can miss the deletion window it is meant to catch.
Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.9

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 477b27b6f87e.

Label changes

Label changes:

add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body/comment provide unit, lint, and typecheck output plus environment details, but no redacted real OpenClaw Dreaming run log or terminal proof after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
remove status: 🛠️ actively grinding: Current PR status label is status: 📣 needs proof.

Label justifications:

P0: The PR targets investigation of repeated silent deletion of memory-core daily files, which is a user data-loss class failure.
merge-risk: 🚨 session-state: The success-path-only comparison can produce false-negative diagnostics for disappearing memory files when the workspace run later fails.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦐 gold shrimp.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body/comment provide unit, lint, and typecheck output plus environment details, but no redacted real OpenClaw Dreaming run log or terminal proof after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +95, Tests +68. Total +163 across 3 files.

View PR surface stats

Area	Files	Added	Net
Source	2	95	+95
Tests	1	68	+68
Docs	0	0	0
Config	0	0	0
Generated	0	0	0
Other	0	0	0
Total	3	163	+163

What I checked:

Repository policy: Read the full root policy and scoped extensions policy; the review applied the plugin-boundary, deep-review, proof, and memory/storage guidance. (AGENTS.md:1, 477b27b6f87e)
PR head run boundary: At PR head, the after-snapshot and missing-file diff run after all awaited sweep, repair, promotion, report, and narrative work; the catch begins at line 716, so any earlier throw skips the diagnostic comparison. (extensions/memory-core/src/dreaming.ts:697, 494317b88629)
Current main failure handling: Current main has separate sweep and promotion failure paths, including a failed deep-dreaming event on promotion/report/narrative failure; the PR needs to be refreshed without losing that boundary. (extensions/memory-core/src/dreaming.ts:573, 477b27b6f87e)
Callee can throw before the after-check: runDreamingSweepPhases catches light/REM failures, appends failed dreaming events, and rethrows, which would bypass the PR's success-path after-snapshot. (extensions/memory-core/src/dreaming-phases.ts:1907, 477b27b6f87e)
Test coverage gap: The new tests cover helper snapshot/diff behavior and a direct deletion between snapshots, but not a workspace step that removes a daily file and then throws before the after-check. (extensions/memory-core/src/dreaming-integrity-guard.test.ts:34, 494317b88629)
Proof state: The PR body and follow-up comment list unit tests, lint, typecheck, and environment details, but no redacted real OpenClaw Dreaming terminal output or logs showing the diagnostic after the patch. (494317b88629)

Likely related people:

vignesh07: PR 60569 introduced the weighted Dreaming promotion flow and touched dreaming.ts and short-term-promotion.ts, which are central to this diagnostic path. (role: original dreaming promotion flow contributor; confidence: high; commits: 4c1022c73b39; files: extensions/memory-core/src/dreaming.ts, extensions/memory-core/src/short-term-promotion.ts)
vincentkoc: PR 60697 hardened Dreaming and multilingual memory promotion and changed the same Dreaming orchestration and recall-repair area. (role: recent memory-core hardening contributor; confidence: high; commits: 0609bf858175; files: extensions/memory-core/src/dreaming.ts, extensions/memory-core/src/short-term-promotion.ts)
ZengWen-DT: Live PR metadata for PR 95943 maps recent current-line churn in the memory-core Dreaming files to this author, though the PR title is adjacent cron timeout work rather than the original Dreaming feature. (role: recent adjacent area contributor; confidence: low; commits: cb1e822f7aa2; files: extensions/memory-core/src/dreaming.ts, extensions/memory-core/src/dreaming-phases.ts, extensions/memory-core/src/short-term-promotion.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (1 earlier review cycle)

reviewed 2026-06-22T03:18:44.268Z sha 83fd7a2 :: needs real behavior proof before merge. :: [P2] Check integrity after failed workspace runs

…g run to catch openclaw#84882 deletions Issue openclaw#84882 reports daily memory files silently disappearing during the nightly Dreaming run. Static review of memory-core and @openclaw/fs-safe found no deterministic deletion path (see PR openclaw#94625 notes), so this adds a diagnostic-only integrity check: snapshot top-level memory/*.md files before and after each per-workspace dreaming pass and log loudly if any present-before files are gone after, with size/mtime for forensics. Never throws and never mutates files; a failed snapshot just skips the check for that run.

… file deletion)

…g run to catch openclaw#84882 deletions Issue openclaw#84882 reports daily memory files silently disappearing during the nightly Dreaming run. Static review of memory-core and @openclaw/fs-safe found no deterministic deletion path (see PR openclaw#94625 notes), so this adds a diagnostic-only integrity check: snapshot top-level memory/*.md files before and after each per-workspace dreaming pass and log loudly if any present-before files are gone after, with size/mtime for forensics. Never throws and never mutates files; a failed snapshot just skips the check for that run.

googlerest · 2026-06-24T12:08:27Z

Behavior addressed: CI failures on push — check-lint (no-unnecessary-type-arguments on dreaming-integrity-guard.ts:25), plus checks-node-core-tooling and checks-node-agentic-plugin-sdk failures unrelated to this diff.

Real environment tested: macOS arm64 (M4), Darwin 25.5.0, Node v25.9.0, branch investigate/84882-dreaming-file-deletion, commit 494317b886 (rebased onto upstream/main @ 2a484a3ff1).

Exact steps or command run after this patch:

node scripts/run-oxlint.mjs extensions/memory-core/src/dreaming-integrity-guard.ts
node scripts/run-vitest.mjs extensions/memory-core/src/dreaming-integrity-guard.test.ts
node scripts/run-vitest.mjs src/scripts/test-projects.test.ts src/plugin-sdk/provider-catalog-shared.test.ts

Evidence after fix:

oxlint: no output, exit 0
dreaming-integrity-guard.test.ts: 5 passed (5)
test-projects.test.ts: 83 passed (83)
provider-catalog-shared.test.ts: 14 passed (14)

Observed result after fix: lint error fixed by removing the unnecessary <string> type argument from Dirent<string>[] (Dirent's generic already defaults to string). The other two CI failures (test-projects.test.ts, provider-catalog-shared.test.ts) were confirmed unrelated to this PR's diff — the branch was 1968 commits behind upstream/main; both tests passed identically when run against current upstream/main HEAD before this patch was applied. Rebasing onto upstream/main (this push) resolves them with no changes to those files.

What was not tested: full GH Actions CI matrix not re-run locally; relying on this push to re-trigger it. Did not exercise the dreaming runtime integration end-to-end (this PR is diagnostic-only snapshot code; no live dreaming run was executed).

openclaw-barnacle · 2026-07-21T04:37:58Z

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

openclaw-barnacle Bot added the size: XS label Jun 18, 2026

openclaw-barnacle Bot added extensions: memory-core Extension: memory-core size: S triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. and removed size: XS labels Jun 18, 2026

googlerest marked this pull request as ready for review June 18, 2026 17:14

clawsweeper Bot added the merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. label Jun 18, 2026

googlerest added 3 commits June 24, 2026 20:04

investigate: claim root-cause work on openclaw#84882 (dreaming silent…

4ad4df4

… file deletion)

fix(memory-core): drop unnecessary Dirent<string> type argument

494317b

googlerest force-pushed the investigate/84882-dreaming-file-deletion branch from 83fd7a2 to 494317b Compare June 24, 2026 12:07

openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 24, 2026

openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 24, 2026

clawsweeper Bot added status: 🛠️ actively grinding The PR author has acted after the latest ClawSweeper review and work remains. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 26, 2026

clawsweeper Bot added status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed status: 🛠️ actively grinding The PR author has acted after the latest ClawSweeper review and work remains. labels Jul 7, 2026

openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jul 21, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

investigate(memory-core): root-cause for #84882 dreaming silent daily-file deletion#94625

investigate(memory-core): root-cause for #84882 dreaming silent daily-file deletion#94625
googlerest wants to merge 3 commits into
openclaw:mainfrom
googlerest:investigate/84882-dreaming-file-deletion

googlerest commented Jun 18, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 18, 2026 •

edited

Loading

Uh oh!

googlerest commented Jun 24, 2026

Uh oh!

openclaw-barnacle Bot commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

googlerest commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Status

Uh oh!

clawsweeper Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

googlerest commented Jun 24, 2026

Uh oh!

openclaw-barnacle Bot commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

googlerest commented Jun 18, 2026 •

edited

Loading

clawsweeper Bot commented Jun 18, 2026 •

edited

Loading