-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
memory-core main reindex thrashes, leaks main.sqlite.tmp files, and leaves memory_search paused after repair #90508
Copy link
Copy link
Open
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.ClawSweeper needs live local, crabbox, or manual validation to confirm this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.Good issue quality with a plausible reproduction path needing some confirmation.
Description
Metadata
Metadata
Assignees
Labels
P1High-priority user-facing bug, regression, or broken workflow.High-priority user-facing bug, regression, or broken workflow.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.ClawSweeper needs live local, crabbox, or manual validation to confirm this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.Session, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.Good issue quality with a plausible reproduction path needing some confirmation.
Type
Fields
Priority
None yet
Summary
On
OpenClaw 2026.6.1 (2e08f0f), themainmemory index got into a reindex-thrash loop on a large active index. The live SQLite DB had chunks and cached embeddings, but no identity metadata row, so vector search failed closed withindex metadata is missing. Repeated rebuild attempts left many abandonedmain.sqlite.tmp-*files, and after a successful CLI rebuild the in-session dynamicmemory_searchtool still served the stale paused state until manager/gateway reload.This looks like a combined memory-core failure mode:
Live repro/evidence from local agent
Environment:
2026.6.1 (2e08f0f)mainopenai, modeltext-embedding-3-large, sourcesmemory,sessionsBefore repair:
/home/exedev/.openclaw/memory/main.sqlite2.3GPRAGMA integrity_check:okchunks:5462embedding_cache:34255metarows:0openclaw memory status --agent mainreported:Indexed: 412/2360 files · 5462 chunksDirty: yesIndex identity: index metadata is missingVector search: paused until memory is rebuiltTmp leak/thrash evidence:
2026-06-04 21:46 AST, there were 10 abandonedmain.sqlite.tmp-*DB families totaling roughly40GBbefore cleanup.openclaw-memoryprocess had open FDs tomain.sqlite.tmp-457c93d7-46cc-49e2-87d7-e0812ce8119b{,-wal,-shm}but file mtimes/sizes stayed unchanged during a 30s observation window after about 20 minutes of runtime. It looked stalled, not active.SIGTERM, and deleting its leftover tmp family, memory dir usage fell to about3.2Gafter the clean rebuild.Manual repair that succeeded on disk:
Post-repair CLI verification:
Memory index updated (main).main.sqlite.tmp-*files remained.metarowmemory_index_meta_v1:openaitext-embedding-3-large3072chunks:5522embedding_cache:34318openclaw memory status --agent main --index --jsonreported:dirty: falsevector.enabled: truevector.available: truecustom.indexIdentity.status: validok: trueopenclaw memory search --agent main --query 'Morningstar NHYDY fair value' --max-results 3returned results.Still broken in live manager/tool path after repair:
The dynamic in-session
memory_searchtool still returned:{ "disabled": true, "unavailable": true, "error": "index metadata is missing", "action": "Tell the user to run: openclaw memory status --index or openclaw memory index --force." }That was after the CLI status/index probe already saw the index as valid and active. This suggests the memory-search manager caches the disabled identity state and does not re-read the repaired index metadata.
Expected behavior
main.sqlite.tmp-*files, or mark them recoverable and clean them on next startup/status/index operation.memory_index_meta_v1identity.Related issues
This seems related but not identical to:
The distinct repro here is the combined loop: large active
mainindex + missing meta + repeated abandoned tmp rebuilds + CLI repair succeeds while dynamicmemory_searchremains paused until manager reload.