Skip to content

Memory Index Repeatedly Broken After Gateway Restart (OOM during Rebuild) #92187

Description

@yaorenliang

Memory Index Repeatedly Broken After Gateway Restart (OOM during Rebuild)

Bug Description

Memory search becomes unavailable after every gateway restart. The root cause is an OOM (Out of Memory) issue during index rebuild.

Reproduction Steps

  1. Gateway restarts (manual or automatic)
  2. Memory plugin detects database state anomaly
  3. Clears main.sqlite and starts full index rebuild
  4. Rebuilding embeddings for 415 files requires ~500MB memory
  5. Process is killed by OOM killer (SIGKILL)
  6. .tmp-xxx file never merges back to main database
  7. main.sqlite remains empty
  8. memory_search returns "index metadata is missing" or "timed out"

Evidence

# After gateway restart, main.sqlite is empty
sqlite3 ~/.openclaw/memory/main.sqlite "SELECT COUNT(*) FROM meta, chunks, files;"
# Output: 0|0|0

# Multiple .tmp files are created but never merged
ls -la ~/.openclaw/memory/main.sqlite.tmp-*
# Shows files growing but never completing

# Process is killed by OOM
ps aux | grep openclaw-memory
# Shows high memory usage (~300MB per process)

Expected Behavior

  1. Index rebuild should handle OOM gracefully (e.g., batch processing, checkpoint/resume)
  2. If rebuild fails, restore from backup instead of leaving empty database
  3. Gateway restart should not trigger full rebuild if database is healthy

Environment

  • OpenClaw version: 2026.6.5 (5181e4f)
  • OS: macOS 26.5.1 (arm64)
  • Node: v26.0.0
  • Memory: 8GB
  • Embedding model: bge-m3 (Ollama)
  • Indexed files: 415

Suggested Fixes

  1. Batch processing: Rebuild index in smaller batches to reduce memory usage
  2. Checkpoint/resume: Save progress during rebuild, resume if interrupted
  3. Backup restore: If rebuild fails, restore from last known good backup
  4. Skip rebuild: Don't trigger full rebuild if database exists and is readable
  5. Memory limit: Add --max-memory option to openclaw memory index

Workaround

Currently, manual intervention is required:

openclaw gateway stop
openclaw memory index --force
openclaw gateway start

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions