fix(memory-core): keep durable index identity visible during safe reindex#90422
fix(memory-core): keep durable index identity visible during safe reindex#90422Alix-007 wants to merge 2 commits into
Conversation
…ndex memory_search intermittently returned disabled/unavailable with 'index metadata is missing' during overlapping calls while the durable index was valid (openclaw#90361). runSafeReindex briefly swaps the live db to a fresh temp db, and a concurrent status()/tool read refreshed index identity from that temp db and failed closed. Track a reindexInProgress flag across the atomic swap and, while it is set, keep returning the last valid identity instead of a transient missing. Real mismatches and missing outside a rebuild still propagate, preserving genuine fail-closed behavior.
|
Thanks for the context here. I swept through the related work, and this is now duplicate or superseded. Close as superseded: the useful status-identity guard is covered by the broader open fix at #90453, which also protects search readers and active DB reads during safe reindex. Canonical path: Close this narrower branch and use #90453 as the canonical landing path for the safe-reindex race. So I’m closing this here and keeping the remaining discussion on #90453. Review detailsBest possible solution: Close this narrower branch and use #90453 as the canonical landing path for the safe-reindex race. Do we have a high-confidence way to reproduce the issue? Yes, source inspection shows the race window: safe reindex swaps Is this the best way to solve the issue? No, this is not the best fix because it only preserves the identity state and does not wait search/vector/FTS readers through the temp-DB window. The broader open PR is the safer owner-boundary fix. Security review: Security review cleared: The diff only changes memory-core TypeScript and tests; no CI, dependency, secret, install, or package-publishing surface was touched. AGENTS.md: found and applied where relevant. What I checked:
Likely related people:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 1a3ce7c2a8da. |
|
@clawsweeper re-review |
|
ClawSweeper applied the proposed close for this PR.
|
Summary
Fixes #90361.
memory_searchintermittently returneddisabled/unavailablewithindex metadata is missingduring parallel or closely overlapping calls, even though the durable builtin index was valid (CLI deep status, direct DB inspection, and fresh-manager checks all showed valid metadata). The cause is a race:MemoryIndexManager.search()can kick off an async sync, andrunSafeReindex()briefly swaps the livethis.dbto a fresh temp db (resetting vector/FTS state before metadata is written). A concurrentstatus()/ tool-wrapper read during that window refreshes index identity from the temp db, finds no metadata, and fails closed withindex metadata is missing— while the real index is fine.What changed
extensions/memory-core/src/memory/manager-sync-ops.ts— add areindexInProgressflag, set for the duration ofrunSafeReindex's atomic temp-db swap and always cleared in afinally.extensions/memory-core/src/memory/manager.ts— inrefreshIndexIdentityDirty, whilereindexInProgressis set, a transientmissingno longer overwrites a previouslyvalididentity; the last stable identity is returned instead. Real mismatches — andmissingoutside a rebuild — still propagate, so genuine provider/model/settings drift keeps failing closed.extensions/memory-core/src/memory/index.test.ts— deterministic regression test: it drives a real safe (atomic) reindex and readsstatus()from inside the reindex build, after the temp db has been swapped in, asserting the durable identity staysvalid.Real behavior proof
Behavior addressed: A
status()read concurrent with a safe reindex's temp-db swap keeps returning the durablevalididentity instead of a transientmissing, somemory_searchno longer fails closed mid-rebuild. Real provider/model/settings mismatches and durable-missing states still fail closed.Real environment tested: Local worktree of
origin/mainat base118060157d, Node v22.22.0. Drove the realMemoryIndexManagerover a real on-disk SQLite index vianode scripts/run-vitest.mjs(the actual production reindex + status paths, no manager mocks). The concurrentstatus()is read from inside the real reindex build, while the real temp db is swapped in.Exact steps or command run after this patch:
Evidence after fix: With the guard reverted (origin
manager.ts), the same test fails — the concurrent read sees the temp db, reproducing the issue:With the fix applied:
Observed result after fix: During a real safe reindex (temp db swapped in),
status().custom.indexIdentitystays{ status: "valid" }instead of{ status: "missing", reason: "index metadata is missing" }. The transientmissingreported by the issue is gone; durable-missing and real mismatch still surface.What was not tested: No live OpenClaw v2026.6.1 end-to-end reproduction across multiple OS/providers; the change is in the deterministic manager identity-refresh path, exercised directly against the real manager and SQLite index. The fix is scoped to the manager identity layer (not a tool-wrapper-only mitigation), preserving real mismatch fail-closed as requested in the issue review.
Validation (supplemental)
index.test.ts+tools.test.ts+manager-reindex-state.test.ts→ 59 passed;manager.async-search.test.ts→ 3 passedtsgo:extensions:test→ no new errors in changed files;oxlint→ clean