fix(memory-core): guard searches during safe reindex by 849261680 · Pull Request #90453 · openclaw/openclaw

849261680 · 2026-06-04T19:54:12Z

Summary

What problem does this PR solve?

Fixes an intermittent memory search/status race where safe full reindex temporarily points the manager at a half-built temp DB, allowing readers to see chunks without matching index metadata and fail closed with index metadata is missing.
Adds a reindex latch so search waits for in-flight full reindex work before reading index identity, FTS availability, or query state.
Keeps status reads stable during the temp-DB window by serving the last durable status snapshot.
Rewrites the verified metadata on the canonical DB after the safe swap so the serialized-meta cache from the temp DB cannot skip the durable write.

Why does this matter now?

[Bug]:Intermittent memory_search "index metadata is missing" despite valid builtin memory index; likely search/reindex race. Locally hotfixed. #90361 reports intermittent built-in memory search failures even though the durable index is valid. The failure mode is timing-dependent, so a focused race guard is needed rather than only metadata recovery.

What is the intended outcome?

Searches and status reads no longer observe the half-built temp DB during safe reindex.
Immediate post-reindex reads see valid index identity metadata on the canonical DB.

What is intentionally out of scope?

Remote embedding provider stress testing.
Broader recovery for previously corrupted or stale on-disk memory stores.
Changes to memory config, provider selection, or migration behavior.

What does success look like?

Forced reindex, status, and search on a real CLI setup complete without index metadata is missing.
Regression tests cover concurrent status/search reads, the provider-initialization await gap, and a reader that is already active when full reindex reaches the DB swap.

What should reviewers focus on?

Whether the reindex latch is drained before every search path that reads index identity, FTS availability, or query data.
Whether the stable status snapshot is limited to the safe reindex temp-DB window.
Whether the post-swap metadata rewrite is idempotent and scoped to full safe reindex.

Linked context

Which issue does this close?

Closes #90361

Which issues, PRs, or discussions are related?

Related #90422
Related #90395

Was this requested by a maintainer or owner?

No direct maintainer request; this is a contributor fix for [Bug]:Intermittent memory_search "index metadata is missing" despite valid builtin memory index; likely search/reindex race. Locally hotfixed. #90361.

Real behavior proof (required for external PRs)

Behavior or issue addressed: built-in memory search/status should not fail with index metadata is missing after a safe forced reindex when the durable OpenClaw index is valid.
Real environment tested: local source checkout at 0a9c4d1ef5, Node v24.15.0, real OpenClaw CLI memory commands, temporary OPENCLAW_HOME, temporary workspace, FTS-only memory provider (provider: "none", model: "fts-only") to avoid external credentials.
Exact steps or command run after this patch:

export OPENCLAW_HOME=<tmp-home>
export OPENCLAW_CONFIG_PATH=<tmp-config>/openclaw.json
export OPENCLAW_BUILD_ALL_NO_PNPM=1
export OPENCLAW_RUN_NODE_SKIP_DTS_BUILD=1
export CI=true
node scripts/run-node.mjs memory index --force --agent main --verbose
node scripts/run-node.mjs memory status --deep --index --agent main --json
node scripts/run-node.mjs memory search --agent main --query 'durable index identity' --max-results 5 --json

Temporary config used for the proof:

{
  "agents": {
    "defaults": {
      "workspace": "<tmp-workspace>",
      "memorySearch": {
        "provider": "none",
        "model": "fts-only",
        "cache": { "enabled": false },
        "sync": { "watch": false, "onSessionStart": false, "onSearch": false }
      }
    },
    "list": [{ "id": "main", "default": true }]
  }
}

Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output):

## Real OpenClaw CLI proof for #90361
repo_commit=0a9c4d1ef5
node=v24.15.0

### Force rebuild index
Memory Index (main)
Provider: none (requested: none)
Model: fts-only
Sources: memory (MEMORY.md + <tmp-path>)

[memory] sync: indexing memory files
Memory index updated (main).

### Parsed proof summary
status_files=1
status_chunks=1
status_dirty=false
status_index_identity=valid
status_has_missing_metadata=false
search_result_count=1
search_has_missing_metadata=false
search_contains_durable=true
search_first_path=MEMORY.md
search_first_snippet="Alpha topic about durable index identity.\n\nThis note proves memory search can recall the durable OpenClaw index after safe reindex.\n"

Observed result after fix: forced CLI reindex completed, status reported a valid index identity with two indexed chunks, and CLI search returned a MEMORY.md result. Neither status nor search output contained index metadata is missing.
What was not tested: live gateway tool invocation, remote/OpenAI embedding provider, concurrent stress across separate OS processes, cross-OS proof.
Proof limitations or environment constraints: the real behavior proof uses the real CLI and SQLite/FTS memory path, but intentionally uses FTS-only provider mode so no external API key or embedding service is required. Provider/vector concurrency is covered by focused regression tests, not by live remote-provider proof.
Before evidence (optional but encouraged): the bug is timing-dependent; this PR adds regression coverage that forces searches/status reads into the safe reindex temp-DB window and the provider-initialization await window.

Tests and validation

Which commands did you run?

git diff --check
node scripts/run-vitest.mjs extensions/memory-core/src/tools.test.ts extensions/memory-core/src/memory/manager.reindex-read-race.test.ts
node scripts/run-vitest.mjs src/scripts/test-projects.test.ts
pnpm exec oxlint --tsconfig config/tsconfig/oxlint.extensions.json extensions/memory-core/src/memory/manager.ts extensions/memory-core/src/memory/manager-sync-ops.ts extensions/memory-core/src/memory/manager.reindex-read-race.test.ts
.agents/skills/autoreview/scripts/autoreview --mode branch --base upstream/main

Real CLI proof commands are listed in the Real behavior proof section.

What regression coverage was added or updated?

Added extensions/memory-core/src/memory/manager.reindex-read-race.test.ts.
Covers status() during in-flight full safe reindex.
Covers search() waiting during the temp-DB window.
Covers search() waiting when provider initialization yields and another actor starts a full safe reindex before identity/FTS reads.
Covers full safe reindex waiting for an already-active index reader before swapping the DB.

What failed before this fix, if known?

The race allowed search/status readers to observe the half-built temp DB during safe reindex and fail closed with invalid or missing index identity metadata.

If no test was added, why not?

Tests were added.

Risk checklist

Did user-visible behavior change? (Yes/No)

Yes. Memory search/status now wait through an in-flight full safe reindex instead of reading transient temp-DB state.

Did config, environment, or migration behavior change? (Yes/No)

No.

Did security, auth, secrets, network, or tool execution behavior change? (Yes/No)

No.

What is the highest-risk area?

Search can now wait for an in-flight full reindex before reading index identity or FTS state.
Full safe reindex writes verified metadata again after reopening the canonical DB.

How is that risk mitigated?

The latch is refcounted and cleared in finally.
Active index readers are counted while DB reads run, and safe full reindex waits for them before swapping/closing the canonical DB.
Status uses only a captured stable snapshot while the safe reindex temp DB is active.
The post-swap metadata rewrite uses the same nextMeta built and verified during the full reindex.
Focused regression tests cover the search/status race windows.
Real CLI proof covers forced reindex, status, and search after the patch.

Current review state

What is the next action?

Ready for maintainer review and CI.

What is still waiting on author, maintainer, CI, or external proof?

CI needs to run on the PR.
Maintainers may request additional live remote-provider stress proof if they want coverage beyond the local CLI/FTS proof and focused regression tests.

Which bot or reviewer comments were addressed?

Fresh local autoreview completed cleanly: .agents/skills/autoreview/scripts/autoreview --mode branch --base upstream/main reported autoreview clean: no accepted/actionable findings reported.

clawsweeper · 2026-06-04T19:56:18Z

Codex review: needs maintainer review before merge. Reviewed July 2, 2026, 12:41 AM ET / 04:41 UTC.

Summary
The branch adds a memory-core safe-reindex read latch, stable status snapshots during reindex, post-swap metadata rewriting, and regression coverage for concurrent search/status reads.

PR surface: Source +98, Tests +311. Total +409 across 5 files.

Reproducibility: yes. at source level: current main still swaps the live manager to a temp reindex DB before metadata publication, and the tool still fail-closes when status reports missing identity. I did not run a live concurrent gateway reproduction in this read-only review.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: migration/backfill/repair: src/scripts/test-projects.test.ts, unknown-data-model-change: extensions/memory-core/src/memory/manager-sync-ops.ts, unknown-data-model-change: extensions/memory-core/src/memory/manager.async-search.test.ts, unknown-data-model-change: extensions/memory-core/src/memory/manager.reindex-read-race.test.ts, unknown-data-model-change: extensions/memory-core/src/memory/manager.ts, vector/embedding metadata: extensions/memory-core/src/memory/manager-sync-ops.ts, and 3 more. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #90361
Summary: This PR is the active candidate fix for the canonical safe-reindex memory_search missing-metadata race, while several related memory metadata issues cover adjacent stale-handle, WAL durability, and recovery paths.

Members:

canonical: [Bug]:Intermittent memory_search "index metadata is missing" despite valid builtin memory index; likely search/reindex race. Locally hotfixed. #90361 - The PR body explicitly closes this open bug, and the issue's current review names this PR as the active direct repair path.
partial_overlap: fix(memory-core): checkpoint WAL after publishing index metadata #92509 - This open PR addresses WAL metadata durability and stale cleanup in the same memory missing-metadata cluster but does not replace the in-process safe-reindex read barrier.
partial_overlap: agentmemory__memory_search returns "index metadata is missing" persistently (memory-core manager state cache) #90414 - This open issue reports persistent memory_search missing metadata from manager state/cache behavior, overlapping the same memory identity failure class without being the exact safe-reindex read race.
partial_overlap: memory-core main reindex thrashes, leaks main.sqlite.tmp files, and leaves memory_search paused after repair #90508 - This open issue combines reindex thrash, temp file leakage, and stale paused memory_search state, sharing the memory lifecycle area but requiring broader cleanup than this PR.
superseded: fix(memory-core): keep durable index identity visible during safe reindex #90422 - This closed unmerged PR attempted a narrower durable-identity guard for the same safe-reindex race and is superseded by this broader read-latch branch.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Risk before merge

[P2] Search and status now wait behind in-process full safe reindex work, which can increase memory_search latency or timeout exposure on large indexes or slow embedding rebuilds.
[P1] Adjacent memory metadata work covers WAL durability, stale handles, temp cleanup, and retry/reopen policy, so maintainers should coordinate merge order with fix(memory-core): checkpoint WAL after publishing index metadata #92509 and related memory-core items.

Maintainer options:

Accept Guarded Wait Semantics (recommended)
Land after maintainer review if the intended behavior is that memory search/status wait for in-process full reindex completion instead of observing transient missing metadata.
Request Stronger Live Stress Proof
Ask for live gateway or remote-provider stress proof if maintainers want evidence beyond the real CLI FTS proof and deterministic manager race tests.
Pause For Metadata Cleanup Ordering
Pause this PR if maintainers want the WAL checkpoint, stale-handle, or temp-file cleanup policy in fix(memory-core): checkpoint WAL after publishing index metadata #92509 merged or reconciled first.

Next step before merge

[P2] Maintainers need to decide whether to accept the wait-through-reindex behavior and coordinate adjacent memory metadata cleanup; no narrow automated repair remains.

Security
Cleared: The diff changes memory-core synchronization/search logic and tests, with no dependency, workflow, package-resolution, secrets, permissions, or external code execution changes.

Review details

Best possible solution:

Land or replace this manager-level read barrier after maintainers accept the latency tradeoff and coordinate adjacent metadata/WAL cleanup work.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main still swaps the live manager to a temp reindex DB before metadata publication, and the tool still fail-closes when status reports missing identity. I did not run a live concurrent gateway reproduction in this read-only review.

Is this the best way to solve the issue?

Yes as an implementation direction: the manager-level read barrier and stable status snapshot address the lifecycle race more directly than a tool-wrapper workaround. The remaining question is maintainer acceptance of the latency tradeoff and adjacent metadata cleanup ordering.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 8c7ac9b9b9df.

Label changes

Label justifications:

P1: The PR targets a first-class memory_search regression that can make agent memory recall unavailable even when the durable index is valid.
merge-risk: 🚨 session-state: The diff changes memory index lifecycle coordination and determines which durable memory/session chunks are visible to agent recall during reindex.
merge-risk: 🚨 availability: The new read barrier can hold search and status behind full reindex work, affecting latency and timeout behavior for large or slow indexes.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes after-fix terminal output from a real OpenClaw CLI forced reindex, status, and search flow using SQLite/FTS, and the current PR checks include a passing Real behavior proof lane.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal output from a real OpenClaw CLI forced reindex, status, and search flow using SQLite/FTS, and the current PR checks include a passing Real behavior proof lane.

Evidence reviewed

PR surface:

Source +98, Tests +311. Total +409 across 5 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	2	148	50	+98
Tests	3	311	0	+311
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	5	459	50	+409

What I checked:

Repository policy read: Read the full root AGENTS.md and the scoped extensions/AGENTS.md; the review applied the repo's deep PR review, plugin boundary, storage, and merge-risk guidance. (AGENTS.md:1, 8c7ac9b9b9df)
Current main still exposes the temp reindex DB: Current main's runInPlaceReindex assigns the live manager's db handle to the temp DB before the rebuild and metadata publication are complete. (extensions/memory-core/src/memory/manager-sync-ops.ts:2714, 8c7ac9b9b9df)
Current main still fail-closes after search on missing identity: The memory_search tool calls manager.search(), then checks manager.status() and returns early when index identity is paused or missing. (extensions/memory-core/src/tools.ts:563, 8c7ac9b9b9df)
Current main lacks this PR's reindex read guard: No current-main symbols or test file for reindexing/withIndexRead/stableStatusSnapshot/manager.reindex-read-race exist, so the central PR behavior is not already implemented on main. (extensions/memory-core/src/memory/manager-sync-ops.ts, 8c7ac9b9b9df)
PR adds the read barrier and active-reader drain: The PR head adds drainReindex(), withIndexRead(), active index reader tracking, and reader drain helpers that block reads while a full reindex is active. (extensions/memory-core/src/memory/manager-sync-ops.ts:528, a820c14e4da6)
PR brackets safe reindex and rewrites metadata on the durable handle: The PR captures a status snapshot, sets the reindex latch, drains active readers before swapping to the temp DB, and rewrites the verified metadata after returning to the durable DB handle. (extensions/memory-core/src/memory/manager-sync-ops.ts:2762, a820c14e4da6)

Likely related people:

vincentkoc: Recent merged memory manager and index migration/table work touched the affected manager, sync, and DB lifecycle paths. (role: recent area contributor; confidence: high; commits: 6e6bd5633f1c, 88bc08c124a5, e085fa1a3ffd; files: extensions/memory-core/src/memory/manager.ts, extensions/memory-core/src/memory/manager-sync-ops.ts, extensions/memory-core/src/memory/manager-db.ts)
osolmaz: Authored the memory index identity validation series that introduced the missing/mismatched fail-closed behavior involved in this race path. (role: identity behavior contributor; confidence: high; commits: a4b4fed41287; files: extensions/memory-core/src/memory/manager.ts, extensions/memory-core/src/memory/manager-sync-ops.ts, extensions/memory-core/src/tools.ts)
xydt-tanshanshan: Authored merged work preserving live SQLite indexes during swaps, which overlaps the same atomic reindex and stale-handle lifecycle. (role: adjacent reindex lifecycle contributor; confidence: medium; commits: 865fdab07501; files: extensions/memory-core/src/memory/manager-sync-ops.ts, extensions/memory-core/src/memory/manager-db.ts, extensions/memory-core/src/memory/manager-atomic-reindex.ts)
steipete: Authored the memory manager extraction and atomic reindex helper refactors that define the current owner boundaries around this code path. (role: refactor and area history contributor; confidence: medium; commits: c9f288ceafbf, 8d2daf7ef229; files: extensions/memory-core/src/memory/manager-sync-ops.ts, extensions/memory-core/src/memory/manager-atomic-reindex.ts, extensions/memory-core/src/memory/manager-sync-control.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

849261680 · 2026-06-04T21:50:03Z

CI note for the current head 0a9c4d1ef5:

This branch has been rebased onto the current upstream/main (a3f495eb09). The PR diff is still limited to:

extensions/memory-core/src/memory/manager-sync-ops.ts
extensions/memory-core/src/memory/manager.reindex-read-race.test.ts
extensions/memory-core/src/memory/manager.ts
src/scripts/test-projects.test.ts

The remaining failing CI checks are outside this PR's touched surface and reproduce locally as standalone failures:

node scripts/run-vitest.mjs src/agents/tools/pdf-tool.model-config.test.ts
node scripts/run-vitest.mjs src/agents/workspace-templates.test.ts
node scripts/run-vitest.mjs src/commands/doctor-heartbeat-template-repair.test.ts

Observed failures:

src/agents/tools/pdf-tool.model-config.test.ts: expects openai/gpt-5.4-mini, receives openai/gpt-5.5.
src/agents/workspace-templates.test.ts: runtime HEARTBEAT.md template is no longer considered effectively empty.
src/commands/doctor-heartbeat-template-repair.test.ts: expected repaired template omits the current comments-only heartbeat template header.

These failures look like current-main expectation drift in agent/doctor tests rather than regressions from the memory-core reindex/search-status race fix in this PR.

anyech · 2026-06-08T07:48:36Z

Thanks for working on this. I hit the same failure mode in an affected install and this PR matches the direction that tested well locally.

Additional local evidence from the failure window:

During safe reindex, the live process had an active *.sqlite.tmp-* DB open.
That temp DB already contained chunks, but did not yet have memory_index_meta_v1.
Concurrent memory_searchcould therefore fail withindex metadata is missing or time out while the rebuild was in flight.
Once the temp DB finalized, the tmp files disappeared and searches recovered, so the tmp DB itself was an active staging artifact rather than stale residue.

I also tested a minimal current-main patch that waits for in-flight sync before search reads index identity/query state and forces metadata to be written on the rebuilt DB path. Focused regression tests for repeated safe reindex metadata and in-flight safe reindex search both passed.

Given that, I think this PR's broader reindex latch/status-snapshot approach is likely the better upstream shape than a minimal duplicate PR. The main practical blocker I see is that the branch currently appears stale/dirty against current main. If helpful, I can provide the minimal rebased patch/test as a reference or alternative, but I do not want to open a competing PR while this one is active unless maintainers prefer that.

Addresses #21: memory_search returns 'disabled: true, unavailable: true, error: index metadata is missing' when the on-disk index sidecar is missing or was written by a different embedding provider/model. The deploy pipeline was silently green because the existing health check (/healthz + gateway process) does not look at the memory index — the disable message only surfaces when an agent actually calls memory_search. The root-cause upstream race lives in openclaw/openclaw#90361 and PR #90453 is the mergeable closing PR (owned by upstream maintainers — L3b does not duplicate the fix). This commit adds the L3b-side detection + deploy-gate integration that catches regressions of the same shape at the deploy boundary. What ships: 1. scripts/lib-parse-memory-status.py Pure decision logic. Reads JSON from stdin (the output of 'openclaw memory status --json') and emits TSV rows (agent_id<TAB>verdict<TAB>reason). Verdicts: ok identity=ok warn-fresh identity missing/mismatch but no data yet warn-swap provider != requestedProvider warn-unknown unrecognized identity state (forward-compat) fail identity missing/mismatch AND data present 2. scripts/post-deploy-verify-memory-index.sh Bash wrapper that runs the probe, classifies rows, and decides: exit 0 = healthy (or fresh install) exit 1 = degraded index on an agent that has data (deploy gate fails) exit 2 = probe failure (openclaw not on PATH, JSON parse error) Honors SKIP_POST_DEPLOY_MEMORY_CHECK=1 (opt-out) and MEMORY_CHECK_FAIL_ON_FRESH=1 (promote fresh-install WARN to FAIL). 3. scripts/install/deploy.sh Wires the verify script in as section 7 (post-deploy verification, called immediately before 'OpenClaw Gateway installation complete'). Exit 1 fails the deploy gate; exit 2 is soft-warned (don't block deploy on missing CLI). 4. tests/post-deploy-verify-memory-index.bats 19 cases covering: healthy / fresh / regression / mismatch / multi-agent / MEMORY_CHECK_FAIL_ON_FRESH / provider swap / unknown state / skip / openclaw-not-on-PATH / invalid JSON / empty output / no agents / parser existence / parser inline- python-free / parser exit codes / deploy.sh wiring. 5. docs/troubleshooting/memory-index-disabled.md Symptom / diagnose / recover / why / prevention matrix / forward-compat notes. Cross-links to upstream issue #90361 and the mergeable closing PR #90453. Verification: - bats tests/post-deploy-verify-memory-index.bats: 19/19 pass - shellcheck --severity=warning: clean on both new bash scripts - pre-commit run --files <all>: clean - python3 -m py_compile: clean on the parser lib Refs: - #21 - openclaw/openclaw#90361 (root cause) - openclaw/openclaw#90453 (upstream closing PR)

…ills (#35) * feat(gateway): post-deploy memory-index verification gate Addresses #21: memory_search returns 'disabled: true, unavailable: true, error: index metadata is missing' when the on-disk index sidecar is missing or was written by a different embedding provider/model. The deploy pipeline was silently green because the existing health check (/healthz + gateway process) does not look at the memory index — the disable message only surfaces when an agent actually calls memory_search. The root-cause upstream race lives in openclaw/openclaw#90361 and PR #90453 is the mergeable closing PR (owned by upstream maintainers — L3b does not duplicate the fix). This commit adds the L3b-side detection + deploy-gate integration that catches regressions of the same shape at the deploy boundary. What ships: 1. scripts/lib-parse-memory-status.py Pure decision logic. Reads JSON from stdin (the output of 'openclaw memory status --json') and emits TSV rows (agent_id<TAB>verdict<TAB>reason). Verdicts: ok identity=ok warn-fresh identity missing/mismatch but no data yet warn-swap provider != requestedProvider warn-unknown unrecognized identity state (forward-compat) fail identity missing/mismatch AND data present 2. scripts/post-deploy-verify-memory-index.sh Bash wrapper that runs the probe, classifies rows, and decides: exit 0 = healthy (or fresh install) exit 1 = degraded index on an agent that has data (deploy gate fails) exit 2 = probe failure (openclaw not on PATH, JSON parse error) Honors SKIP_POST_DEPLOY_MEMORY_CHECK=1 (opt-out) and MEMORY_CHECK_FAIL_ON_FRESH=1 (promote fresh-install WARN to FAIL). 3. scripts/install/deploy.sh Wires the verify script in as section 7 (post-deploy verification, called immediately before 'OpenClaw Gateway installation complete'). Exit 1 fails the deploy gate; exit 2 is soft-warned (don't block deploy on missing CLI). 4. tests/post-deploy-verify-memory-index.bats 19 cases covering: healthy / fresh / regression / mismatch / multi-agent / MEMORY_CHECK_FAIL_ON_FRESH / provider swap / unknown state / skip / openclaw-not-on-PATH / invalid JSON / empty output / no agents / parser existence / parser inline- python-free / parser exit codes / deploy.sh wiring. 5. docs/troubleshooting/memory-index-disabled.md Symptom / diagnose / recover / why / prevention matrix / forward-compat notes. Cross-links to upstream issue #90361 and the mergeable closing PR #90453. Verification: - bats tests/post-deploy-verify-memory-index.bats: 19/19 pass - shellcheck --severity=warning: clean on both new bash scripts - pre-commit run --files <all>: clean - python3 -m py_compile: clean on the parser lib Refs: - #21 - openclaw/openclaw#90361 (root cause) - openclaw/openclaw#90453 (upstream closing PR) * feat(skills): add memory-status and memory-rebuild slash commands Brings the recovery flow for #21 out of the SSH-only domain and into Discord. Both skills are wired into the existing skill-sync phase of scripts/install/deploy.sh (section 6) — no extra deploy wiring is needed. Two skills: 1. memory-status (/memory-status) Reads 'openclaw memory status --json' and produces a per-agent table with: agent, identity (ok/missing/mismatch), provider, chunks, notes. Surfaces the L3b deploy gate's verdict matrix (warn-fresh / warn-swap / fail) so the user knows whether a rebuild is required, or whether the index is empty and will rebuild on first-use. 2. memory-rebuild (/memory-rebuild) Runs 'openclaw memory index --force' and reports the result. Lists the documented failure modes (auth error, EACCES, model-not-in-catalog) and the recovery steps for each. Warns before rebuilding blindly — does not auto-rebuild on unknown identity states. Both skills reference the troubleshooting doc and the upstream issue (#90361) so a user hitting this in the future has a single documented path to resolution. Verification: - deploy-skills-sync.bats (existing tests cover the new skills): 11/11 pass — they are picked up by the generic skill-sync test - pre-commit run --files <new SKILL.md>: clean - Frontmatter validates as YAML with name + commands (matches the existing model-management skill pattern) Refs: - #21 - openclaw/openclaw#90361 - 78890b8 (Tier 1: deploy-gate wiring) --------- Co-authored-by: Migration Agent <[email protected]>

849261680 · 2026-07-02T04:27:26Z

@clawsweeper re-review

Repair pushed for the current head a820c14e4da6e98270a7c315e4febd6af29f7860.

Addressed finding:

Restored the FTS-unavailable vector-only fallback in extensions/memory-core/src/memory/manager.ts so empty keyword results do not enter mergeHybridResults and downweight vector hits below minScore.
Added focused regression coverage in extensions/memory-core/src/memory/manager.async-search.test.ts proving the manager returns raw vector-only hits and does not call hybrid merge when keyword/FTS search is unavailable.

Validation completed after the repair/rebase:

pnpm exec oxfmt --check extensions/memory-core/src/memory/manager.ts extensions/memory-core/src/memory/manager.async-search.test.ts
pnpm exec oxlint --tsconfig config/tsconfig/oxlint.extensions.json extensions/memory-core/src/memory/manager.ts extensions/memory-core/src/memory/manager.async-search.test.ts
git diff --check upstream/main...HEAD
.agents/skills/autoreview/scripts/autoreview --mode commit --commit HEAD --prompt-file /tmp/openclaw-90453-autoreview-context.txt

Result: formatting, oxlint, diff check, and autoreview are clean. Local Vitest/tsgo attempts on this host did not complete because the local test runner stayed in a no-output build/startup state; I am not counting those as passed.

clawsweeper · 2026-07-02T04:27:28Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

openclaw-barnacle · 2026-07-17T04:27:00Z

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

clawsweeper · 2026-07-17T04:45:59Z

ClawSweeper status: review started.

I am starting a fresh review of this pull request: fix(memory-core): guard searches during safe reindex This is item 1/1 in the current shard. Shard 2/4.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

openclaw-barnacle · 2026-07-25T04:27:51Z

Closing due to inactivity.
If you believe this PR should be revived, post in #clawtributors on Discord to talk to a maintainer.
That channel is the escape hatch for high-quality PRs that get auto-closed.

openclaw-barnacle Bot added extensions: memory-core Extension: memory-core size: M proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 4, 2026

849261680 force-pushed the fix/90361-memory-reindex-read-race branch from 55ddb80 to 1f4817e Compare June 4, 2026 20:35

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 4, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 4, 2026

849261680 force-pushed the fix/90361-memory-reindex-read-race branch from 1f4817e to 09c4abb Compare June 4, 2026 21:18

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 4, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 4, 2026

849261680 force-pushed the fix/90361-memory-reindex-read-race branch from 09c4abb to 0a9c4d1 Compare June 4, 2026 21:30

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 4, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 4, 2026

clawsweeper Bot mentioned this pull request Jun 5, 2026

fix(memory-core): write meta after sync when identity is missing with existing chunks #90395

Closed

2 tasks

849261680 force-pushed the fix/90361-memory-reindex-read-race branch from 0a9c4d1 to 239c193 Compare June 5, 2026 07:16

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 5, 2026

clawsweeper Bot mentioned this pull request Jun 5, 2026

fix(memory-core): keep durable index identity visible during safe reindex #90422

Closed

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 5, 2026

This was referenced Jun 5, 2026

fix(memory): refresh rebuilt index handles #90727

Closed

[Bug]:Intermittent memory_search "index metadata is missing" despite valid builtin memory index; likely search/reindex race. Locally hotfixed. #90361

Open

clawsweeper Bot mentioned this pull request Jun 7, 2026

fix(memory): self-heal missing index identity during sync #91173

Closed

849261680 force-pushed the fix/90361-memory-reindex-read-race branch from 239c193 to 26b0b99 Compare June 10, 2026 18:39

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 10, 2026

clawsweeper Bot mentioned this pull request Jun 20, 2026

memory_search broken with "index scope changed" after --force rebuild — scopeHash mismatch persists #91592

Closed

clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 20, 2026

849261680 mentioned this pull request Jun 23, 2026

fix(memory): prove live manager recovery after CLI reindex #96094

Merged

clawsweeper Bot mentioned this pull request Jun 26, 2026

memory_search: index identity check blocks use of valid chunks when memory_index_meta is empty (Ubuntu IP change recovery) #97015

Closed

patelmm79 mentioned this pull request Jun 29, 2026

feat(gateway): post-deploy memory-index verification gate + memory skills DarojaAI/openclaw-gateway#35

Merged

clawsweeper Bot mentioned this pull request Jun 30, 2026

Slack socket starved by blocking SQLite VACUUM on bloated main.sqlite; interrupted vacuums leak 53GB of orphaned .tmp files #83712

Closed

849261680 added 2 commits July 2, 2026 12:23

fix(memory-core): guard searches during safe reindex

297e158

fix(memory-core): preserve vector fallback without FTS

a820c14

849261680 force-pushed the fix/90361-memory-reindex-read-race branch from bc317e2 to a820c14 Compare July 2, 2026 04:26

openclaw-barnacle Bot added size: L and removed size: M labels Jul 2, 2026

clawsweeper Bot mentioned this pull request Jul 7, 2026

[Feature]: Memory indexing lacks checkpoint/resume capability for long-running operations #26772

Open

openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jul 17, 2026

openclaw-barnacle Bot closed this Jul 25, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(memory-core): guard searches during safe reindex#90453

fix(memory-core): guard searches during safe reindex#90453
849261680 wants to merge 2 commits into
openclaw:mainfrom
849261680:fix/90361-memory-reindex-read-race

849261680 commented Jun 4, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 4, 2026 •

edited

Loading

Uh oh!

849261680 commented Jun 4, 2026

Uh oh!

anyech commented Jun 8, 2026

Uh oh!

849261680 commented Jul 2, 2026

Uh oh!

clawsweeper Bot commented Jul 2, 2026 •

edited

Loading

Uh oh!

openclaw-barnacle Bot commented Jul 17, 2026

Uh oh!

clawsweeper Bot commented Jul 17, 2026

Uh oh!

openclaw-barnacle Bot commented Jul 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

849261680 commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Linked context

Real behavior proof (required for external PRs)

Tests and validation

Risk checklist

Current review state

Uh oh!

clawsweeper Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

849261680 commented Jun 4, 2026

Uh oh!

anyech commented Jun 8, 2026

Uh oh!

849261680 commented Jul 2, 2026

Uh oh!

clawsweeper Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openclaw-barnacle Bot commented Jul 17, 2026

Uh oh!

clawsweeper Bot commented Jul 17, 2026

Uh oh!

openclaw-barnacle Bot commented Jul 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

849261680 commented Jun 4, 2026 •

edited

Loading

clawsweeper Bot commented Jun 4, 2026 •

edited

Loading

clawsweeper Bot commented Jul 2, 2026 •

edited

Loading