Summary

After ~7 hours of runtime, the gateway accumulates ~14,000 open file descriptors (regular files, not memory-mapped). This eventually causes spawn EBADF on all exec operations, event loop saturation (utilization=1.0, P99 delays 15-43s), and complete gateway unresponsiveness.

Environment

• openclaw: 2026.5.2 (also present in 2026.5.3 — pi-* deps unchanged at 0.71.1)
• Node.js: v25.9.0 (Homebrew)
• macOS: Darwin 25.4.0 (ARM64)
• Uptime: ~7h before unresponsive

Symptoms

1. FD count grows linearly: Starting from 0, reaching 13,935 REG file descriptors after ~7h
2. spawn EBADF: All exec operations fail — even echo hello, pwd
3. Event loop saturation: eventLoopUtilization=1.0, P99 delays up to 43s
4. pi-trajectory-flush timeout: Multiple agent cleanup timed out at 30s timeout
5. Gateway unresponsive: Health endpoint times out, WebSocket connections fail with "gateway timeout after 10000ms"
6. Process state 'U': Uninterruptible kernel wait, unresponsive to SIGTERM

Leaked File Analysis

lsof analysis of the stuck process (PID 24231) showed leaked files primarily from:

Most FDs are unique (not duplicate references to the same file), suggesting they accumulate across session initializations.

Code Paths Verified (NOT the cause)

The following file-reading code paths all properly close FDs:

• readWorkspaceFileWithGuards (boundary-file-read → readFileSync + closeSync in finally)
• readSkillFileSync (openVerifiedFileSync → readFileSync + closeSync in finally)
• readFileWithinRoot (openFileWithinRoot → readOpenedFileSafely + handle.close in finally)
• fs.glob async iterator (confirmed doesn't leak on Node.js v25 with early break)
• memory-wiki extension (fs.readFile path-based, no fd exposure)
• pi-coding-agent loadSkills / loadProjectContextFiles (all readFileSync path-based)
• QueuedFileWriter.safeAppendFile (handle.close in finally)

Diagnostic Logs

[diagnostic] liveness warning: eventLoopDelayP99Ms=18941.5 eventLoopUtilization=1
[diagnostic] liveness warning: eventLoopDelayP99Ms=42144.4 eventLoopUtilization=1
[diagnostic] liveness warning: eventLoopDelayP99Ms=43486.5 eventLoopUtilization=1
[tools] exec failed: spawn EBADF raw_params={"command":"echo hello","timeout":5}
[tools] exec failed: spawn EBADF raw_params={"command":"pwd"}
[tools] exec failed: spawn EBADF raw_params={"command":"true","host":"gateway"}
[agent/embedded] agent cleanup timed out: step=pi-trajectory-flush timeoutMs=30000
[bundle-mcp] failed to start server "zhipu-vision": Error: spawn EBADF
Subagent announce failed: GatewayTransportError: gateway timeout after 10000ms

Steps to Reproduce

1. Start OpenClaw gateway: openclaw gateway --port <port>
2. Let it run for 6-8 hours with normal agent activity (feishu channel, cron jobs, wiki operations)
3. Monitor lsof -p <pid> | grep ' REG ' | grep -v ' txt ' | wc -l
4. FD count will grow steadily; exec operations begin failing around ~10k FDs

Impact

• Critical: All exec/spawn operations fail, effectively breaking all agent tool use
• Gateway becomes completely unresponsive, requiring SIGKILL to terminate
• Affects all channels (Feishu, Discord, etc.)

Workaround

Periodic gateway restart (every 4-6 hours) prevents the issue from accumulating to critical levels.

Related

• PR fix: prevent event loop saturation from trajectory flush (setImmediate yield + 10MB cap + 30s timeout) #77133 — related trajectory flush → event loop issue (different root cause, but similar symptoms)