Skip to content

memory-core: synchronous indexing on session start blocks gateway event loop for 30+ seconds #76890

Description

@huangyingw

Summary

When a new session is created (session key not present in sessions.json), some path in memory-core plugin and/or agents.defaults.memorySearch.sync.onSessionStart causes the gateway Node.js event loop to block synchronously for 30+ seconds, fully saturating one CPU core. During this window the gateway cannot service any WebSocket request — TUI clients see gateway request timeout for connect or gateway closed (1000) even though the gateway process is alive.

Repro

  1. Configure agent with:
    "agents": {
      "defaults": {
        "memorySearch": {
          "enabled": true,
          "sources": ["memory", "sessions"],
          "experimental": { "sessionMemory": true },
          "provider": "ollama",
          "remote": { "baseUrl": "http://localhost:11434" },
          "model": "bge-m3:latest"
        }
      }
    }
  2. Start gateway, open TUI with a brand-new session key (one not yet in sessions.json).
  3. Observe gateway logs:
    [diagnostic] liveness warning: reasons=event_loop_delay,event_loop_utilization,cpu
      interval=39s eventLoopDelayP99Ms=36507.2 eventLoopDelayMaxMs=36507.2
      eventLoopUtilization=1 cpuCoreRatio=1.035
    [ws] ⇄ res ✓ sessions.list 27932ms
    
  4. TUI: stuck at connecting | idle ... unknown | tokens ?, eventually disconnects.

Diagnostic

What I tried Effect
agents.defaults.memorySearch.enabled = false Doesn't help; 30+s block still occurs
agents.defaults.memorySearch.sync.onSessionStart = false Doesn't fully help on its own
plugins.entries.memory-core.enabled = false Fixes the gateway blockagents.list 1s, sessions.list <1s for existing sessions

So the trigger is some path inside memory-core plugin that runs synchronously even when memorySearch.enabled=false. Disabling the entire plugin is the only effective workaround, but it removes the memory feature entirely.

Suggested fix

  • Audit memory-core for synchronous code that runs even with memorySearch.enabled=false and gate it on the master toggle.
  • Wherever indexing must run, ensure it's properly broken into chunks with setImmediate yield (the existing SESSIONS_LIST_YIELD_BATCH_SIZE pattern in src/gateway/session-utils.ts does this elsewhere).
  • Consider running indexing in a worker thread so the main event loop is never blocked.

Workaround

openclaw config set plugins.entries.memory-core.enabled false
systemctl --user restart openclaw-gateway

Loses memory search across sessions but unblocks gateway.

Environment

  • OpenClaw: 2026.4.29 (a448042)
  • Node: 22.22.1
  • OS: Linux 6.17.0-1014-nvidia
  • System under load (load avg ~10), but the 36s block reproduces independent of load

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions