Skip to content

fix(memory-core): keep daily ingestion outside session repair#93389

Merged
steipete merged 3 commits into
openclaw:mainfrom
Alix-007:fix/dreaming-repair-clear-daily-ingestion
Jul 6, 2026
Merged

fix(memory-core): keep daily ingestion outside session repair#93389
steipete merged 3 commits into
openclaw:mainfrom
Alix-007:fix/dreaming-repair-clear-daily-ingestion

Conversation

@Alix-007

@Alix-007 Alix-007 commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

The dreaming repair audit mixed two independent bookkeeping domains: session-corpus ingestion and daily-memory ingestion. This made daily-only state look like stale session ingestion and encouraged session repair to delete healthy daily-memory bookkeeping.

This corrects the boundary related to #92017 and #92020 without broadening destructive repair behavior.

Why This Change Was Made

The original PR direction was reversed after reading the full dreaming pipeline and repair callers. sessionIngestionExists now reflects only the session-files and seen-session namespaces. Session repair clears those namespaces while preserving dreaming-daily-ingestion, whose lifecycle belongs to daily-memory ingestion.

User Impact

openclaw memory status --fix no longer reports daily-only bookkeeping as session-corpus state and no longer discards healthy daily ingestion records during explicit session repair.

Evidence

  • Blacksmith Testbox tbx_01kww7ned9gavfc1rtwaghsm8y: all 8 focused dreaming-repair tests passed against the real SQLite-backed plugin-state store.
  • Regression coverage proves daily-only audit is false, session namespaces audit true and are cleared, and daily rows survive session repair.
  • Same Testbox: pnpm check:changed passed.
  • Fresh Codex autoreview: no actionable findings.

Credit: @Alix-007 surfaced and authored the original repair; @vincentkoc provided the corrected session-scope direction.

AI-assisted: yes. Maintainer repair, full owner/caller/sibling review, regression proof, and final validation were performed before landing.

@openclaw-barnacle openclaw-barnacle Bot added extensions: memory-core Extension: memory-core size: S triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 15, 2026
@clawsweeper

clawsweeper Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 6, 2026, 12:56 PM ET / 16:56 UTC.

Summary
The branch clears DREAMING_DAILY_INGESTION_NAMESPACE during dreaming artifact repair and adds regression tests for clearing and post-repair re-audit behavior.

PR surface: Source +8, Tests +57. Total +65 across 2 files.

Reproducibility: yes. source-reproducible: current main audits dreaming-daily-ingestion as ingestion state while repair clears only the two session namespaces. The PR proof also includes an independent SQLite readback and negative control, but I did not rerun it locally in this read-only review.

Review metrics: 1 noteworthy metric.

  • Persisted repair clear: 1 SQLite namespace added. The diff changes explicit repair behavior for persisted memory-core plugin state, which is the main maintainer-visible merge consideration.

Stored data model
Persistent data-model change detected: migration/backfill/repair: extensions/memory-core/src/dreaming-repair.test.ts, serialized state: extensions/memory-core/src/dreaming-repair.test.ts, unknown-data-model-change: extensions/memory-core/src/dreaming-repair.test.ts, vector/embedding metadata: extensions/memory-core/src/dreaming-repair.test.ts, vector/embedding metadata: extensions/memory-core/src/dreaming-repair.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: canonical
Canonical: #93389
Summary: This PR is the narrow repair-side fix for the daily-ingestion SQLite namespace symmetry gap created after audit began counting migrated ingestion namespaces.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] Merging intentionally makes explicit memory status --fix and doctor dreaming repair delete one additional workspace-scoped persisted ingestion namespace; maintainers should confirm daily-ingestion bookkeeping is meant to be rebuilt after this repair.

Maintainer options:

  1. Accept the symmetric reset (recommended)
    Land after maintainer acceptance that explicit dreaming repair clears daily-ingestion rows so later sweeps can rebuild the bookkeeping.
  2. Keep daily rows out of this repair
    Request a patch that preserves the daily namespace or exposes it through a separate repair action if daily bookkeeping must survive session-corpus repair.

Next step before merge

  • [P2] The remaining action is maintainer acceptance of clearing one additional persisted ingestion namespace during explicit repair, not an automated code repair.

Maintainer decision needed

  • Question: Should memory status --fix and doctor dreaming repair clear the migrated dreaming-daily-ingestion SQLite namespace whenever they archive the dreaming session corpus or legacy session-ingestion artifact?
  • Rationale: The code change is narrow and symmetric with current audit behavior, but it deliberately deletes persisted workspace ingestion bookkeeping under an explicit repair action, so maintainer intent should confirm the repair semantic before merge.
  • Likely owner: Takhoffman — Takhoffman introduced the dreaming repair surface and is the strongest history-backed owner for the repair semantic choice.
  • Options:
    • Accept Symmetric Repair (recommended): Merge this PR so explicit dreaming repair clears all three audited ingestion namespaces and later sweeps rebuild daily bookkeeping.
    • Preserve Daily Bookkeeping: Ask for a different repair path if daily-ingestion rows should survive session-corpus repair despite audit treating them as ingestion state.

Security
Cleared: The diff uses existing memory-core plugin-state helpers and focused tests, with no dependency, workflow, secret, download, publishing, or broader code-execution change.

Review details

Best possible solution:

Land the narrow repair if maintainers accept that explicit dreaming artifact repair should reset every audited ingestion namespace; otherwise request a repair shape that preserves daily bookkeeping separately.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: current main audits dreaming-daily-ingestion as ingestion state while repair clears only the two session namespaces. The PR proof also includes an independent SQLite readback and negative control, but I did not rerun it locally in this read-only review.

Is this the best way to solve the issue?

Yes, the implementation is the narrowest maintainable code fix if maintainers accept the repair semantic: it updates the existing clear helper and pins the audit/repair symmetry with focused tests. The only safer alternative is a maintainer-directed repair split that intentionally preserves daily bookkeeping.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 06789ca4c2ed.

Label changes

Label justifications:

  • P3: The PR is a small focused bug fix for an explicit repair path rather than an urgent runtime regression.
  • merge-risk: 🚨 session-state: Merging changes repair behavior for persisted memory-core ingestion state by clearing an additional workspace-scoped namespace.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body supplies terminal proof that drives the real audit/repair functions against a real SQLite state DB, verifies persisted rows through an independent read-only SQLite handle, and includes a negative control.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body supplies terminal proof that drives the real audit/repair functions against a real SQLite state DB, verifies persisted rows through an independent read-only SQLite handle, and includes a negative control.
Evidence reviewed

PR surface:

Source +8, Tests +57. Total +65 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 8 0 +8
Tests 1 57 0 +57
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 65 0 +65

What I checked:

Likely related people:

  • Takhoffman: Introduced the dreaming repair surface and CLI/doctor recovery wiring in the merged repair/recovery PR. (role: feature owner; confidence: high; commits: 847739d82c60, 0ffef0e603e7, e1f728d22a26; files: extensions/memory-core/src/dreaming-repair.ts, extensions/memory-core/src/cli.runtime.ts, src/commands/doctor-memory-search.ts)
  • steipete: Moved Memory Core dreaming runtime state from legacy JSON sidecars into SQLite-backed plugin state and added the daily/session namespace model this repair now touches. (role: SQLite migration owner; confidence: high; commits: 3f5e00184431, 427741bc80c2; files: extensions/memory-core/src/dreaming-state.ts, extensions/memory-core/src/dreaming-repair.ts, extensions/memory-core/doctor-contract-api.ts)
  • zenglingbiao: Added the audit fallback that treats all migrated ingestion namespaces, including daily ingestion, as present ingestion state. (role: audit-side contributor; confidence: medium; commits: 15498f88fb76, 2a9ad19e9c06, 2e3b0309060e; files: extensions/memory-core/src/dreaming-repair.ts, extensions/memory-core/src/dreaming-repair.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (8 earlier review cycles)
  • reviewed 2026-07-02T14:08:17.872Z sha ab34a08 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-05T12:23:06.695Z sha da6323a :: needs maintainer review before merge. :: none
  • reviewed 2026-07-05T13:10:44.694Z sha 14ed193 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-06T03:41:19.920Z sha dcae604 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-06T04:09:04.747Z sha 57b9cb4 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-06T04:14:58.054Z sha 57b9cb4 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-06T06:32:03.484Z sha e890741 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-06T06:57:50.806Z sha 084b35d :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jun 15, 2026
@Alix-007

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@Alix-007

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Added real-behavior proof crossing the real SQLite boundary (addressing needs-real-behavior-proof).

Behavior addressed: repairDreamingArtifacts now also clears the dreaming-daily-ingestion SQLite namespace when archiving session-corpus, so a post-repair re-audit no longer sees phantom ingestion state.

Real environment: local worktree, Node v22, node --import tsx, isolated OPENCLAW_STATE_DIR. Writes go through the real exported writeMemoryCoreWorkspaceEntries; readback opens an independent read-only node:sqlite connection to the on-disk openclaw.sqlite and runs SELECT COUNT(*) FROM plugin_state_entries WHERE plugin_id='memory-core' AND namespace='dreaming-daily-ingestion' (business-side DB handle closed before each readback, so the count is the on-disk truth, not an in-process echo). No mocks — createPluginStateKeyedStoreForTests is the production createPluginStateKeyedStore with an injectable state dir, not a stub.

Evidence after fix:

[1] wrote N=3 entries to daily-ingestion namespace
[2] pre-repair readback (independent read-only SQLite conn) = 3
[3] repairDreamingArtifacts -> changed=true archivedSessionCorpus=true
[4] post-repair readback (independent read-only SQLite conn) = 0
RESULT: PASS

Negative control (temporarily removing the 4-line fix, then restored): post-repair readback stays 3 — proving the namespace clear is exactly this change.

What was not tested: no full dreaming scheduler run; the change is the deterministic repair write-path, driven directly via the real exported function against a real SQLite store. extensions/memory-core/src/dreaming-repair.test.ts: 9/9 pass.

@clawsweeper

clawsweeper Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added status: 🔁 re-review loop A fresh ClawSweeper review was explicitly requested after the latest review. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 16, 2026
@Alix-007
Alix-007 force-pushed the fix/dreaming-repair-clear-daily-ingestion branch 3 times, most recently from 28a07b2 to 7416b2a Compare June 18, 2026 02:15
@clawsweeper clawsweeper Bot added status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed status: 🔁 re-review loop A fresh ClawSweeper review was explicitly requested after the latest review. labels Jun 19, 2026
@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 19, 2026
@Alix-007

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review — Added a Real behavior proof section: it drives the real auditDreamingArtifacts/repairDreamingArtifacts against a real SQLite state store and reads back physical disk rows via an independent read-only handle — after repair the dreaming-daily-ingestion namespace rows go to 0, while a negative control reverting just the daily-clear line leaves the row surviving (audit still reports it). Vitest (9 tests) passes.

@clawsweeper

clawsweeper Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 20, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 21, 2026
@Alix-007

Alix-007 commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto latest main to refresh the stale failing CI run. The previous failures were outside this PR surface (src/gateway/server-cron*.test.ts, src/llm/providers/anthropic.test.ts, src/llm/providers/mistral.test.ts), while this PR only touches extensions/memory-core/src/dreaming-repair*.

Focused validation passed locally after rebase:

  • node scripts/run-vitest.mjs extensions/memory-core/src/dreaming-repair.test.ts -- --run (9/9)
  • oxlint extensions/memory-core/src/dreaming-repair.ts extensions/memory-core/src/dreaming-repair.test.ts
  • node scripts/generate-docs-map.mjs --check

@clawsweeper clawsweeper Bot added the merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. label Jul 2, 2026
@Alix-007
Alix-007 force-pushed the fix/dreaming-repair-clear-daily-ingestion branch from ab34a08 to da6323a Compare July 5, 2026 11:53
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. labels Jul 5, 2026
@Alix-007
Alix-007 force-pushed the fix/dreaming-repair-clear-daily-ingestion branch 7 times, most recently from 084b35d to 5597b57 Compare July 6, 2026 16:03
@steipete steipete self-assigned this Jul 6, 2026
@steipete
steipete force-pushed the fix/dreaming-repair-clear-daily-ingestion branch from 5597b57 to d7fb117 Compare July 6, 2026 17:41
@steipete steipete changed the title fix(memory-core): clear daily-ingestion sqlite namespace on dreaming repair fix(memory-core): keep daily ingestion outside session repair Jul 6, 2026
@steipete

steipete commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Land-ready maintainer pass complete at exact head 1e2d7d315ae19d49f2c502a50f247c21ea1da486.

Repair performed:

  • reversed the original broad-deletion direction after full pipeline review;
  • scoped session-ingestion audit and repair to session files/seen namespaces;
  • preserved independent daily-memory ingestion bookkeeping;
  • added daily-only, coexistence, session-clear, and post-repair regression coverage;
  • preserved release-note context and credit to @Alix-007 and @vincentkoc in the PR body for the aggregate release changelog.

Proof:

  • Blacksmith Testbox 28810582395: all 8 focused real SQLite-backed tests passed; pnpm check:changed passed.
  • Exact-head CI 28812414156: hosted gate source.
  • Fresh Codex autoreview: no actionable findings.
  • Review artifacts validated with recommendation READY FOR /prepare-pr.

Known gaps: none for the session-repair owner boundary.

@steipete
steipete force-pushed the fix/dreaming-repair-clear-daily-ingestion branch from d7fb117 to 0736760 Compare July 6, 2026 17:54
Alix-007 and others added 3 commits July 6, 2026 14:00
…repair

repairDreamingArtifacts() cleared the dreaming-session-ingestion-files and
dreaming-session-ingestion-seen namespaces but not the migrated
dreaming-daily-ingestion namespace. After openclaw#92020 taught auditDreamingArtifacts()
to treat all three ingestion namespaces as ingestion state, the repair path
became asymmetric: the memory status --fix re-audit still reported
sessionIngestionExists=true from the surviving daily rows, and the daily
ingestion bookkeeping leaked past repair so daily memory files were not
re-ingested on the next sweep.

Clear DREAMING_DAILY_INGESTION_NAMESPACE alongside the session files/seen
namespaces so repair fully resets dreaming ingestion state, matching the audit.
@steipete
steipete force-pushed the fix/dreaming-repair-clear-daily-ingestion branch from 0736760 to 1e2d7d3 Compare July 6, 2026 18:00
@steipete
steipete merged commit 962d109 into openclaw:main Jul 6, 2026
88 checks passed
@steipete

steipete commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 7, 2026
…aw#93389)

* fix(memory-core): clear daily-ingestion sqlite namespace on dreaming repair

repairDreamingArtifacts() cleared the dreaming-session-ingestion-files and
dreaming-session-ingestion-seen namespaces but not the migrated
dreaming-daily-ingestion namespace. After openclaw#92020 taught auditDreamingArtifacts()
to treat all three ingestion namespaces as ingestion state, the repair path
became asymmetric: the memory status --fix re-audit still reported
sessionIngestionExists=true from the surviving daily rows, and the daily
ingestion bookkeeping leaked past repair so daily memory files were not
re-ingested on the next sweep.

Clear DREAMING_DAILY_INGESTION_NAMESPACE alongside the session files/seen
namespaces so repair fully resets dreaming ingestion state, matching the audit.

* chore: retrigger CI for real behavior proof check

* fix(memory-core): keep daily ingestion outside session repair

Co-authored-by: Vincent Koc <[email protected]>
Co-authored-by: Alix-007 <[email protected]>

---------

Co-authored-by: Peter Steinberger <[email protected]>
Co-authored-by: Vincent Koc <[email protected]>
giodl73-repo pushed a commit to giodl73-repo/openclaw that referenced this pull request Jul 8, 2026
…aw#93389)

* fix(memory-core): clear daily-ingestion sqlite namespace on dreaming repair

repairDreamingArtifacts() cleared the dreaming-session-ingestion-files and
dreaming-session-ingestion-seen namespaces but not the migrated
dreaming-daily-ingestion namespace. After openclaw#92020 taught auditDreamingArtifacts()
to treat all three ingestion namespaces as ingestion state, the repair path
became asymmetric: the memory status --fix re-audit still reported
sessionIngestionExists=true from the surviving daily rows, and the daily
ingestion bookkeeping leaked past repair so daily memory files were not
re-ingested on the next sweep.

Clear DREAMING_DAILY_INGESTION_NAMESPACE alongside the session files/seen
namespaces so repair fully resets dreaming ingestion state, matching the audit.

* chore: retrigger CI for real behavior proof check

* fix(memory-core): keep daily ingestion outside session repair

Co-authored-by: Vincent Koc <[email protected]>
Co-authored-by: Alix-007 <[email protected]>

---------

Co-authored-by: Peter Steinberger <[email protected]>
Co-authored-by: Vincent Koc <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: memory-core Extension: memory-core merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: XS status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants