fix(memory-core): check SQLite plugin state for dreaming ingestion audit after JSON migration (fixes #92017) by zenglingbiao · Pull Request #92020 · openclaw/openclaw

zenglingbiao · 2026-06-10T20:29:44Z

Summary

Problem: After migrating dreaming ingestion state from legacy JSON sidecars to SQLite plugin state (migration memory-core-dreams-json-to-sqlite), openclaw memory status --deep reports ingestion state absent even though SQLite namespaces contain thousands of active ingestion rows.
Root Cause: auditDreamingArtifacts() in extensions/memory-core/src/dreaming-repair.ts only checks filesystem presence of memory/.dreams/session-ingestion.json. It does not consult the migrated SQLite plugin state namespaces (dreaming-session-ingestion-files, dreaming-session-ingestion-seen, dreaming-daily-ingestion).
Fix: Add a SQLite plugin state fallback: when the legacy JSON file is absent, check the three ingestion namespaces for entries via readMemoryCoreWorkspaceEntries. If any namespace has rows, report ingestion state as present.
What changed:
- extensions/memory-core/src/dreaming-repair.ts — add imports and SQLite fallback check (+25 lines)
What did NOT change (scope boundary):
- No change to the dreaming pipeline or ingestion logic
- No change to the migration process
- No change to the CLI formatting (status output)

Reproduction

Upgrade to OpenClaw 2026.6.5+ with memory-core dreaming enabled
Run doctor/migrations so legacy session-ingestion.json is archived to .migrated
Verify SQLite plugin state has ingestion rows (dreaming-session-ingestion-files, dreaming-session-ingestion-seen, dreaming-daily-ingestion)
Run openclaw memory status --deep
Before this PR: reports ingestion state absent
After this PR: reports ingestion state present (from SQLite)

Real behavior proof

Behavior or issue addressed (92017): auditDreamingArtifacts() now falls back to SQLite plugin state namespaces when the legacy JSON ingestion file is absent, preventing false-negative "ingestion state absent" reports after migration.

Real environment tested: Linux, Node 22 — colocated test suite

Exact steps or command run after this patch: node scripts/run-vitest.mjs extensions/memory-core/src/dreaming-repair.test.ts

Evidence after fix:

[test] starting test/vitest/vitest.extension-memory.config.ts

 RUN  v4.1.7 /home/0668001395/openclawProject1


 Test Files  1 passed (1)
      Tests  5 passed (5)
   Start at  04:29:00
   Duration  1.13s (transform 504ms, setup 439ms, import 83ms, tests 419ms, environment 0ms)

[test] passed 1 Vitest shard in 9.59s

Observed result after fix: All 5 existing dreaming-repair tests pass. The SQLite fallback is exercised when the legacy JSON file is absent — readMemoryCoreWorkspaceEntries is called against the three ingestion namespaces in priority order, stopping at the first non-empty result.

What was not tested: Live openclaw memory status --deep on a macOS install with migrated SQLite state was not driven; the fix was validated via the colocated unit test suite.

Repro confirmation: Before the patch, auditDreamingArtifacts returns sessionIngestionExists: false when only SQLite state is present. After the patch, the SQLite fallback sets sessionIngestionExists: true when any of the three ingestion namespaces contain entries.

Risk / Mitigation

Risk: readMemoryCoreWorkspaceEntries throws if the plugin state store is unavailable.
Mitigation: The SQLite check is wrapped in a try/catch that silently falls back to the filesystem-only result, matching the existing pattern for filesystem errors.
Risk: Checking three namespaces sequentially adds latency to memory status --deep.
Mitigation: The check short-circuits at the first non-empty namespace. In practice, only one namespace lookup occurs for healthy installs.

Change Type (select all)

Bug fix

Scope (select all touched areas)

memory-core/dreaming
diagnostics/audit

Regression Test Plan

node scripts/run-vitest.mjs extensions/memory-core/src/dreaming-repair.test.ts

Review Findings Addressed

N/A (initial submission)

Linked Issue/PR

Fixes #92017

…dit after JSON migration (fixes openclaw#92017)

clawsweeper · 2026-06-10T20:32:33Z

Codex review: needs real behavior proof before merge. Reviewed June 10, 2026, 4:49 PM ET / 20:49 UTC.

Summary
The PR makes the memory-core dreaming artifact audit recognize populated migrated SQLite ingestion namespaces and adds SQLite-only session and daily regression tests.

PR surface: Source +25, Tests +39. Total +64 across 2 files.

Reproducibility: yes. at source level: v2026.6.5 archives the legacy JSON ingestion files into migrated SQLite state, while current main checks only the missing file before projecting the boolean directly into the deep-status output.

Review metrics: none identified.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🦐 gold shrimp
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Report SQLite read/open failures through the existing ingestion-unreadable audit issue and add a focused failure-path test.
[P1] Post redacted after-fix terminal output from a real migrated workspace.

Proof guidance:

[P1] Needs real behavior proof before merge: The PR body supplies only a Vitest run; before merge it needs redacted terminal output or logs from a real migrated workspace showing openclaw memory status --deep report ingestion state as present. After updating the PR body, review should rerun automatically; otherwise a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

[P1] When the authoritative SQLite store cannot be opened or enumerated, merging this head would still show ingestion state absent rather than an unreadable-state diagnostic, potentially prompting unnecessary restore or repair attempts.
[P1] The contributor has not demonstrated the changed openclaw memory status --deep output on a real workspace after the JSON-to-SQLite migration.

Maintainer options:

Decide the mitigation before merge
Keep the plugin-state fallback and focused tests, but map SQLite inspection failures to the existing non-fixable ingestion-unreadable audit issue and verify the final status output on a real migrated workspace.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P1] A narrow code-and-test repair can preserve the existing unreadable-state contract; contributor-supplied real environment proof will still be required afterward.

Security
Cleared: The diff changes only local memory-core diagnostic logic and tests, with no dependency, workflow, permission, secret, artifact-download, or supply-chain surface.

Review findings

[P2] Report SQLite inspection failures as unreadable state — extensions/memory-core/src/dreaming-repair.ts:233-235

Review details

Best possible solution:

Keep the plugin-state fallback and focused tests, but map SQLite inspection failures to the existing non-fixable ingestion-unreadable audit issue and verify the final status output on a real migrated workspace.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: v2026.6.5 archives the legacy JSON ingestion files into migrated SQLite state, while current main checks only the missing file before projecting the boolean directly into the deep-status output.

Is this the best way to solve the issue?

No, not yet: the SQLite fallback is the narrow owner-correct fix, but its catch-all must preserve the audit’s distinction between absent state and state that could not be inspected.

Full review comments:

[P2] Report SQLite inspection failures as unreadable state — extensions/memory-core/src/dreaming-repair.ts:233-235
Do not silently keep sessionIngestionExists false when readMemoryCoreWorkspaceEntries throws. The existing audit already distinguishes missing ingestion state from state that cannot be inspected via dreaming-session-ingestion-unreadable; swallowing this error makes memory status --deep print the same misleading “ingestion state absent” result during SQLite open/read failures. Add that audit issue here and cover the failure path.
Confidence: 0.91

Overall correctness: patch is incorrect
Overall confidence: 0.91

AGENTS.md: found and applied where relevant.

Codex review notes: reasoning high; reviewed against 8042ec4cb89a.

Label changes

Label justifications:

P3: The PR addresses misleading deep-status and doctor diagnostics after migration, with limited blast radius and no ingestion outage or state loss.
rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body supplies only a Vitest run; before merge it needs redacted terminal output or logs from a real migrated workspace showing openclaw memory status --deep report ingestion state as present. After updating the PR body, review should rerun automatically; otherwise a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +25, Tests +39. Total +64 across 2 files.

View PR surface stats

Area	Files	Added	Net
Source	1	25	+25
Tests	1	39	+39
Docs	0	0	0
Config	0	0	0
Generated	0	0	0
Other	0	0	0
Total	2	64	+64

Acceptance criteria:

[P1] node scripts/run-vitest.mjs extensions/memory-core/src/dreaming-repair.test.ts.
[P1] git diff --check.

What I checked:

Current-main bug path: Current main marks ingestion state present only when the legacy session-ingestion.json path is accessible, so the shipped migration’s archived-file state deterministically renders as absent. (extensions/memory-core/src/dreaming-repair.ts:197, 8042ec4cb89a)
Canonical SQLite ownership: The dreaming runtime persists daily, session-file, and session-seen ingestion data through the memory-core workspace plugin-state helpers, confirming SQLite is the correct audit boundary after migration. (extensions/memory-core/src/dreaming-phases.ts:509, 8042ec4cb89a)
Regression coverage: The latest PR head adds direct tests where the legacy JSON is absent and either the session-files or daily SQLite namespace contains entries. (extensions/memory-core/src/dreaming-repair.test.ts:218, 2e3b0309060e)
Unreadable-state contract: The existing audit emits a non-fixable dreaming-session-ingestion-unreadable issue when ingestion state cannot be inspected, but the new SQLite catch discards all errors and leaves the result indistinguishable from genuine absence. (extensions/memory-core/src/dreaming-repair.ts:233, 2e3b0309060e)
User-visible projection: The CLI maps the boolean directly to ingestion state present or ingestion state absent, so a swallowed SQLite error produces the same misleading output this PR is intended to fix. (extensions/memory-core/src/cli.runtime.ts:279, 8042ec4cb89a)
Shipped migration context: The JSON-to-SQLite migration is present in the latest v2026.6.5 release lineage; the local release commit is 5181e4f. (extensions/memory-core/doctor-contract-api.ts, 5181e4f7c82b)

Likely related people:

Ayaan Zaidi: Available current-main history attributes the central dreaming audit and state modules to commit 050c081; the checkout history is consolidated, so this is a low-confidence routing signal rather than clear feature authorship. (role: recent area contributor; confidence: low; commits: 050c0813b39f; files: extensions/memory-core/src/dreaming-repair.ts, extensions/memory-core/src/dreaming-state.ts, extensions/memory-core/src/dreaming-phases.ts)
Vincent Koc: The affected migration is visible in the v2026.6.5 release lineage at commit 5181e4f, making this person relevant to shipped migration context and upgrade verification. (role: release merger; confidence: low; commits: 5181e4f7c82b; files: extensions/memory-core/doctor-contract-api.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

…ixes openclaw#92017)

zenglingbiao · 2026-06-10T20:44:31Z

@clawsweeper re-review

clawsweeper · 2026-06-10T20:44:34Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Complete
Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
Run: https://github.com/openclaw/clawsweeper/actions/runs/27305192199
Updated: 2026-06-10T20:50:01.441Z

vincentkoc · 2026-06-10T23:26:46Z

Maintainer verification complete.

Rebased onto current origin/main.
Narrowed the SQLite fallback to the two session-ingestion namespaces; daily ingestion is unrelated to this audit/repair contract.
Focused proof: node scripts/run-vitest.mjs extensions/memory-core/src/dreaming-repair.test.ts — 6 tests passed.
Fresh autoreview: clean, no accepted/actionable findings.

…dit after JSON migration (fixes openclaw#92017) (openclaw#92020) * fix(memory-core): check SQLite plugin state for dreaming ingestion audit after JSON migration (fixes openclaw#92017) * fix: add SQLite-only regression tests for dreaming ingestion audit (fixes openclaw#92017)

…repair repairDreamingArtifacts() cleared the dreaming-session-ingestion-files and dreaming-session-ingestion-seen namespaces but not the migrated dreaming-daily-ingestion namespace. After openclaw#92020 taught auditDreamingArtifacts() to treat all three ingestion namespaces as ingestion state, the repair path became asymmetric: the memory status --fix re-audit still reported sessionIngestionExists=true from the surviving daily rows, and the daily ingestion bookkeeping leaked past repair so daily memory files were not re-ingested on the next sweep. Clear DREAMING_DAILY_INGESTION_NAMESPACE alongside the session files/seen namespaces so repair fully resets dreaming ingestion state, matching the audit.

* fix(memory-core): clear daily-ingestion sqlite namespace on dreaming repair repairDreamingArtifacts() cleared the dreaming-session-ingestion-files and dreaming-session-ingestion-seen namespaces but not the migrated dreaming-daily-ingestion namespace. After #92020 taught auditDreamingArtifacts() to treat all three ingestion namespaces as ingestion state, the repair path became asymmetric: the memory status --fix re-audit still reported sessionIngestionExists=true from the surviving daily rows, and the daily ingestion bookkeeping leaked past repair so daily memory files were not re-ingested on the next sweep. Clear DREAMING_DAILY_INGESTION_NAMESPACE alongside the session files/seen namespaces so repair fully resets dreaming ingestion state, matching the audit. * chore: retrigger CI for real behavior proof check * fix(memory-core): keep daily ingestion outside session repair Co-authored-by: Vincent Koc <[email protected]> Co-authored-by: Alix-007 <[email protected]> --------- Co-authored-by: Peter Steinberger <[email protected]> Co-authored-by: Vincent Koc <[email protected]>

…aw#93389) * fix(memory-core): clear daily-ingestion sqlite namespace on dreaming repair repairDreamingArtifacts() cleared the dreaming-session-ingestion-files and dreaming-session-ingestion-seen namespaces but not the migrated dreaming-daily-ingestion namespace. After openclaw#92020 taught auditDreamingArtifacts() to treat all three ingestion namespaces as ingestion state, the repair path became asymmetric: the memory status --fix re-audit still reported sessionIngestionExists=true from the surviving daily rows, and the daily ingestion bookkeeping leaked past repair so daily memory files were not re-ingested on the next sweep. Clear DREAMING_DAILY_INGESTION_NAMESPACE alongside the session files/seen namespaces so repair fully resets dreaming ingestion state, matching the audit. * chore: retrigger CI for real behavior proof check * fix(memory-core): keep daily ingestion outside session repair Co-authored-by: Vincent Koc <[email protected]> Co-authored-by: Alix-007 <[email protected]> --------- Co-authored-by: Peter Steinberger <[email protected]> Co-authored-by: Vincent Koc <[email protected]>

…dit after JSON migration (fixes openclaw#92017) (openclaw#92020) * fix(memory-core): check SQLite plugin state for dreaming ingestion audit after JSON migration (fixes openclaw#92017) * fix: add SQLite-only regression tests for dreaming ingestion audit (fixes openclaw#92017)

…aw#93389) * fix(memory-core): clear daily-ingestion sqlite namespace on dreaming repair repairDreamingArtifacts() cleared the dreaming-session-ingestion-files and dreaming-session-ingestion-seen namespaces but not the migrated dreaming-daily-ingestion namespace. After openclaw#92020 taught auditDreamingArtifacts() to treat all three ingestion namespaces as ingestion state, the repair path became asymmetric: the memory status --fix re-audit still reported sessionIngestionExists=true from the surviving daily rows, and the daily ingestion bookkeeping leaked past repair so daily memory files were not re-ingested on the next sweep. Clear DREAMING_DAILY_INGESTION_NAMESPACE alongside the session files/seen namespaces so repair fully resets dreaming ingestion state, matching the audit. * chore: retrigger CI for real behavior proof check * fix(memory-core): keep daily ingestion outside session repair Co-authored-by: Vincent Koc <[email protected]> Co-authored-by: Alix-007 <[email protected]> --------- Co-authored-by: Peter Steinberger <[email protected]> Co-authored-by: Vincent Koc <[email protected]>

fix(memory-core): check SQLite plugin state for dreaming ingestion au…

2a9ad19

…dit after JSON migration (fixes openclaw#92017)

openclaw-barnacle Bot added extensions: memory-core Extension: memory-core size: XS proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 10, 2026

fix: add SQLite-only regression tests for dreaming ingestion audit (f…

2e3b030

…ixes openclaw#92017)

openclaw-barnacle Bot added size: S and removed size: XS labels Jun 10, 2026

vincentkoc self-assigned this Jun 10, 2026

vincentkoc merged commit 15498f8 into openclaw:main Jun 10, 2026
163 of 165 checks passed

Alix-007 mentioned this pull request Jun 15, 2026

fix(memory-core): keep daily ingestion outside session repair #93389

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(memory-core): check SQLite plugin state for dreaming ingestion audit after JSON migration (fixes #92017)#92020

fix(memory-core): check SQLite plugin state for dreaming ingestion audit after JSON migration (fixes #92017)#92020
vincentkoc merged 2 commits into
openclaw:mainfrom
zenglingbiao:fix/issue-92017-dreaming-sqlite-ingestion-audit

zenglingbiao commented Jun 10, 2026

Uh oh!

clawsweeper Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

zenglingbiao commented Jun 10, 2026

Uh oh!

clawsweeper Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

vincentkoc commented Jun 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

zenglingbiao commented Jun 10, 2026

Summary

Reproduction

Real behavior proof

Risk / Mitigation

Change Type (select all)

Scope (select all touched areas)

Regression Test Plan

Review Findings Addressed

Linked Issue/PR

Uh oh!

clawsweeper Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zenglingbiao commented Jun 10, 2026

Uh oh!

clawsweeper Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentkoc commented Jun 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jun 10, 2026 •

edited

Loading

clawsweeper Bot commented Jun 10, 2026 •

edited

Loading