feat(gateway): post-deploy memory-index verification gate + memory skills#35
Merged
Conversation
added 2 commits
June 29, 2026 01:32
Addresses #21: memory_search returns 'disabled: true, unavailable: true, error: index metadata is missing' when the on-disk index sidecar is missing or was written by a different embedding provider/model. The deploy pipeline was silently green because the existing health check (/healthz + gateway process) does not look at the memory index — the disable message only surfaces when an agent actually calls memory_search. The root-cause upstream race lives in openclaw/openclaw#90361 and PR #90453 is the mergeable closing PR (owned by upstream maintainers — L3b does not duplicate the fix). This commit adds the L3b-side detection + deploy-gate integration that catches regressions of the same shape at the deploy boundary. What ships: 1. scripts/lib-parse-memory-status.py Pure decision logic. Reads JSON from stdin (the output of 'openclaw memory status --json') and emits TSV rows (agent_id<TAB>verdict<TAB>reason). Verdicts: ok identity=ok warn-fresh identity missing/mismatch but no data yet warn-swap provider != requestedProvider warn-unknown unrecognized identity state (forward-compat) fail identity missing/mismatch AND data present 2. scripts/post-deploy-verify-memory-index.sh Bash wrapper that runs the probe, classifies rows, and decides: exit 0 = healthy (or fresh install) exit 1 = degraded index on an agent that has data (deploy gate fails) exit 2 = probe failure (openclaw not on PATH, JSON parse error) Honors SKIP_POST_DEPLOY_MEMORY_CHECK=1 (opt-out) and MEMORY_CHECK_FAIL_ON_FRESH=1 (promote fresh-install WARN to FAIL). 3. scripts/install/deploy.sh Wires the verify script in as section 7 (post-deploy verification, called immediately before 'OpenClaw Gateway installation complete'). Exit 1 fails the deploy gate; exit 2 is soft-warned (don't block deploy on missing CLI). 4. tests/post-deploy-verify-memory-index.bats 19 cases covering: healthy / fresh / regression / mismatch / multi-agent / MEMORY_CHECK_FAIL_ON_FRESH / provider swap / unknown state / skip / openclaw-not-on-PATH / invalid JSON / empty output / no agents / parser existence / parser inline- python-free / parser exit codes / deploy.sh wiring. 5. docs/troubleshooting/memory-index-disabled.md Symptom / diagnose / recover / why / prevention matrix / forward-compat notes. Cross-links to upstream issue #90361 and the mergeable closing PR #90453. Verification: - bats tests/post-deploy-verify-memory-index.bats: 19/19 pass - shellcheck --severity=warning: clean on both new bash scripts - pre-commit run --files <all>: clean - python3 -m py_compile: clean on the parser lib Refs: - #21 - openclaw/openclaw#90361 (root cause) - openclaw/openclaw#90453 (upstream closing PR)
Brings the recovery flow for #21 out of the SSH-only domain and into Discord. Both skills are wired into the existing skill-sync phase of scripts/install/deploy.sh (section 6) — no extra deploy wiring is needed. Two skills: 1. memory-status (/memory-status) Reads 'openclaw memory status --json' and produces a per-agent table with: agent, identity (ok/missing/mismatch), provider, chunks, notes. Surfaces the L3b deploy gate's verdict matrix (warn-fresh / warn-swap / fail) so the user knows whether a rebuild is required, or whether the index is empty and will rebuild on first-use. 2. memory-rebuild (/memory-rebuild) Runs 'openclaw memory index --force' and reports the result. Lists the documented failure modes (auth error, EACCES, model-not-in-catalog) and the recovery steps for each. Warns before rebuilding blindly — does not auto-rebuild on unknown identity states. Both skills reference the troubleshooting doc and the upstream issue (#90361) so a user hitting this in the future has a single documented path to resolution. Verification: - deploy-skills-sync.bats (existing tests cover the new skills): 11/11 pass — they are picked up by the generic skill-sync test - pre-commit run --files <new SKILL.md>: clean - Frontmatter validates as YAML with name + commands (matches the existing model-management skill pattern) Refs: - #21 - openclaw/openclaw#90361 - 78890b8 (Tier 1: deploy-gate wiring)
patelmm79
force-pushed
the
fix/memory-index-deploy-gate
branch
from
June 29, 2026 01:34
083afc1 to
311053e
Compare
Contributor
Author
|
Force-push at 01:11 UTC: rebased onto current main after PR #34 (lane-health) merged. Resolved one conflict in scripts/install/deploy.sh — kept both lane-health sections (7+8) and added the memory-index gate as section 9. All 19 memory-index tests + 11 skill-sync tests pass on the rebased branch. shellcheck clean. |
This was referenced Jul 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two commits addressing DarojaAI/openclaw-gateway#21 (
memory_searchreturnsdisabled: true, error: "index metadata is missing"after an embedding model swap or indexer-version mismatch).Commit 1 — Deploy-gate integration (Tier 1)
The deploy pipeline was silently green because the existing health check (
/healthz+ gateway process) does not look at the memory index — the disable message only surfaces when an agent actually callsmemory_search. This commit adds the L3b-side detection + deploy-gate integration that catches regressions of the same shape at the deploy boundary.Files:
scripts/lib-parse-memory-status.py— pure decision logic (parseopenclaw memory status --json, emit TSV rows)scripts/post-deploy-verify-memory-index.sh— bash wrapper that runs the probe, classifies rows, decides exit codescripts/install/deploy.sh— wires the verify script in as section 7 (final deploy step)tests/post-deploy-verify-memory-index.bats— 19 casesdocs/troubleshooting/memory-index-disabled.md— full diagnostic guideCommit 2 — Operator UX skills (Tier 2)
Brings the recovery flow out of the SSH-only domain and into Discord:
config/skills/memory-status/SKILL.md—/memory-statusreports per-agent identity / provider / chunksconfig/skills/memory-rebuild/SKILL.md—/memory-rebuildrunsopenclaw memory index --forceBoth are auto-installed by the existing skill-sync phase (section 6 of
deploy.sh) — no extra deploy wiring needed.Why this is L3b, not upstream
Upstream
openclaw/openclawowns the underlying race fix:L3b does not duplicate the upstream fix. This PR complements it by ensuring future regressions of the same shape are caught at the deploy boundary rather than at the next
memory_searchcall.Behavior matrix
identity=okidentity=missingAND no dataidentity=missingAND has dataidentity=mismatchAND has dataprovider≠requestedProvideropenclaw memory status --jsonerrorsopenclawnot on PATHVerification
bats tests/post-deploy-verify-memory-index.bats: 19/19 passbats tests/deploy-skills-sync.bats: 11/11 pass (covers the new skills via the generic skill-sync test)shellcheck --severity=warning: clean on both new bash scriptspre-commit run --files <all>: cleanpython3 -m py_compile scripts/lib-parse-memory-status.py: cleanFull suite (
bats tests/*.bats): 130/137 pass. The 7 pre-existing failures (cost-monitor.batsandviz-render-server.bats) are test-environment artifacts unrelated to this change — they fail onmaintoo.Risk
Low. The script reads
openclaw memory status --json, which is the same probe the upstream failure surfaces. It does not write to the index, does not modify config, and does not touchoverride.conf. Probe failures (openclaw not on PATH, JSON parse error) exit 2 anddeploy.shtreats that as a soft-warn — the deploy continues. Only exit 1 (regression on an agent with data) blocks the deploy gate, which is exactly the failure mode the issue describes.Opt-out:
SKIP_POST_DEPLOY_MEMORY_CHECK=1. Promote fresh-install WARN to FAIL for prod:MEMORY_CHECK_FAIL_ON_FRESH=1.Out of scope
openclaw/openclaw(owned by PR #90453)restore-prod-tokens.yml-style emergency recovery workflow for memory (the new skills cover the user-facing recovery; the deploy gate covers the regression case)config/openclaw-defaults.json(Tier 3 — deferred until upstreammemory.embeddingconfig schema is confirmed)Refs
.worktrees/fix-memory-index-deploy-gate/)