Summary
memorySearch.sync.watch: true accumulates ~16,000 REG file descriptors over 1–3 hours when extraPaths contains a directory with thousands of .md files, leading to ulimit saturation and spawn EBADF for any subsequent child-process spawn from the gateway. Disabling the watcher (sync.watch: false) immediately stops the leak.
Environment
- openclaw
2026.5.4 (npm global, Node v24.14.0)
- macOS Darwin 25.3.0 (arm64)
~/.openclaw/rag-index/lcm-summaries/ contains ~15,000 markdown summaries (heavy LCM user via @martian-engineering/lossless-claw 0.9.4)
- Gateway launched via launchd (
gui/$UID/ai.openclaw.gateway)
Symptom
$ lsof -p $(pgrep -f openclaw-gateway) | awk '{print $5}' | sort | uniq -c
~16000 REG # ~95% in ~/.openclaw/rag-index/lcm-summaries/, rest in workspace docs/
~few KQUEUE
...
After ~1–3h of normal session activity, fds saturate the soft ulimit (raised to 65536 here; default 256 saturates much faster). First externally visible failure: spawn EBADF when the gateway tries to spawn any helper process. /readyz reports degraded.
Repro
- Configure a main agent with:
{
"agents": {
"list": [{
"name": "main",
"memorySearch": {
"enabled": true,
"sync": {"watch": true, "onSessionStart": true, "onSearch": true}
}
}],
"defaults": {
"memorySearch": {
"extraPaths": ["~/.openclaw/rag-index/lcm-summaries", "~/some/docs/tree"]
}
}
}
}
- Ensure one of the
extraPaths directories contains 10k+ .md files.
- Restart gateway. Sample fd count over time — grows from ~50 to 16k+ within a few hours.
Workaround (verified)
Set agents.list[N].memorySearch.sync.watch: false. Keeps onSessionStart + onSearch enabled so recall still works (one-shot scans release fds correctly); only the long-lived watcher is removed.
After applying, fd count stabilizes at 46–69 over 10+ minutes (vs. growth to 16k+ before).
Suspected location
dist/manager-CYeCmxMa.js:1067 (MemoryManagerSyncOps.ensureWatcher):
this.watcher = resolveMemoryWatchFactory()(Array.from(watchPaths), {
ignoreInitial: true,
ignored: (watchPath, stats) => shouldIgnoreMemoryWatchPath(watchPath, stats, this.settings.multimodal),
awaitWriteFinish: {
stabilityThreshold: this.settings.sync.watchDebounceMs,
pollInterval: 100
}
});
watchPaths adds whole directories (e.g. lcm-summaries/), and chokidar walks recursively. With awaitWriteFinish enabled, chokidar polls each tracked file every 100ms to determine write stability. On a 15k-file tree that's a lot of stat/open work, and any close-on-error path that's not airtight will leak fds at scale.
The downstream markDirty → scheduleWatchSync → listMemoryFiles + buildFileEntry re-walks extraPaths on each event. buildFileEntry uses fs.promises.readFile (which auto-closes), so I don't think the indexer itself is the leak — it's the watcher infrastructure.
I cannot pinpoint a single line without dtrace on open/close syscalls, but the lsof evidence (REG fds, all in watched paths, leak gated by sync.watch) puts it firmly in the watcher's lifecycle.
Proposed fixes (in order of preference)
- Per-path watch opt-out. Allow
extraPaths: [{path, watch: false}] (or a sibling sync.watch.ignorePaths array). LCM summaries are append-only and don't benefit from live re-indexing; a session-start scan is sufficient. Today there's no way to watch some extraPaths but not others — it's all-or-nothing per agent.
- Per-tree file-count cap. If
walkDir under an extraPath exceeds a threshold (e.g. 5,000 files), log a warning and skip adding that subtree to the watcher. Users who want it can override.
- Drop
awaitWriteFinish from the memory watcher. The existing scheduleWatchSync debounce is sufficient for index freshness; awaitWriteFinish's polling is the most fd-expensive chokidar mode.
- Self-defense. Periodic fd-count check in
MemoryManagerSyncOps; if growth is monotonic, close and recreate the watcher (mitigation, not root-cause).
I'm happy to PR (1) — the config-shape change is small and backward-compatible.
Why this matters
memorySearch is one of the more compelling features of the agent runtime, and sync.watch: false as a workaround means losing live re-indexing. For users with large LCM corpora (which is exactly the audience that benefits most from semantic recall), the current default cascades into gateway crashes within hours.
Summary
memorySearch.sync.watch: trueaccumulates ~16,000 REG file descriptors over 1–3 hours whenextraPathscontains a directory with thousands of.mdfiles, leading to ulimit saturation andspawn EBADFfor any subsequent child-process spawn from the gateway. Disabling the watcher (sync.watch: false) immediately stops the leak.Environment
2026.5.4(npm global, Node v24.14.0)~/.openclaw/rag-index/lcm-summaries/contains ~15,000 markdown summaries (heavy LCM user via@martian-engineering/lossless-claw0.9.4)gui/$UID/ai.openclaw.gateway)Symptom
After ~1–3h of normal session activity, fds saturate the soft ulimit (raised to 65536 here; default 256 saturates much faster). First externally visible failure:
spawn EBADFwhen the gateway tries to spawn any helper process./readyzreports degraded.Repro
{ "agents": { "list": [{ "name": "main", "memorySearch": { "enabled": true, "sync": {"watch": true, "onSessionStart": true, "onSearch": true} } }], "defaults": { "memorySearch": { "extraPaths": ["~/.openclaw/rag-index/lcm-summaries", "~/some/docs/tree"] } } } }extraPathsdirectories contains 10k+.mdfiles.Workaround (verified)
Set
agents.list[N].memorySearch.sync.watch: false. KeepsonSessionStart+onSearchenabled so recall still works (one-shot scans release fds correctly); only the long-lived watcher is removed.After applying, fd count stabilizes at 46–69 over 10+ minutes (vs. growth to 16k+ before).
Suspected location
dist/manager-CYeCmxMa.js:1067(MemoryManagerSyncOps.ensureWatcher):watchPathsadds whole directories (e.g.lcm-summaries/), and chokidar walks recursively. WithawaitWriteFinishenabled, chokidar polls each tracked file every 100ms to determine write stability. On a 15k-file tree that's a lot of stat/open work, and any close-on-error path that's not airtight will leak fds at scale.The downstream
markDirty → scheduleWatchSync → listMemoryFiles + buildFileEntryre-walksextraPathson each event.buildFileEntryusesfs.promises.readFile(which auto-closes), so I don't think the indexer itself is the leak — it's the watcher infrastructure.I cannot pinpoint a single line without dtrace on
open/closesyscalls, but the lsof evidence (REG fds, all in watched paths, leak gated bysync.watch) puts it firmly in the watcher's lifecycle.Proposed fixes (in order of preference)
extraPaths: [{path, watch: false}](or a siblingsync.watch.ignorePathsarray). LCM summaries are append-only and don't benefit from live re-indexing; a session-start scan is sufficient. Today there's no way to watch someextraPathsbut not others — it's all-or-nothing per agent.walkDirunder anextraPathexceeds a threshold (e.g. 5,000 files), log a warning and skip adding that subtree to the watcher. Users who want it can override.awaitWriteFinishfrom the memory watcher. The existingscheduleWatchSyncdebounce is sufficient for index freshness; awaitWriteFinish's polling is the most fd-expensive chokidar mode.MemoryManagerSyncOps; if growth is monotonic, close and recreate the watcher (mitigation, not root-cause).I'm happy to PR (1) — the config-shape change is small and backward-compatible.
Why this matters
memorySearchis one of the more compelling features of the agent runtime, andsync.watch: falseas a workaround means losing live re-indexing. For users with large LCM corpora (which is exactly the audience that benefits most from semantic recall), the current default cascades into gateway crashes within hours.