Skip to content

MCP Server Resource Leak - Orphaned processes cause memory exhaustion and hanging commands #65694

Description

@aliahmadaziz

Summary

OpenClaw gateway has a resource leak where MCP (Model Context Protocol) servers launched during agent sessions are not properly cleaned up when sessions end. This leads to memory exhaustion, subprocess hangs, and cron job timeouts.

Environment

  • OpenClaw Version: 2026.4.11 (769908e)
  • Operating System: Ubuntu 22.04 LTS (4GB RAM VM)
  • MCP Config: Custom MCP server defined in gateway config (plugins.entries)

Bug Description

The OpenClaw gateway spawns MCP servers (defined in gateway plugin config) for agent sessions but fails to terminate them when sessions complete. Over time, orphaned MCP server processes accumulate, causing:

  1. Memory Leak: Each orphaned MCP server consumes 15-27MB RAM
  2. Subprocess Hanging: openclaw sessions --active 1440 --json hangs indefinitely under memory pressure
  3. Cron Job Timeouts: Resource exhaustion causes subsequent cron jobs to timeout
  4. Plugin Resolution Errors: Memory pressure causes plugin loading failures

Observed Behavior

  • Initial State: Clean system with no orphaned MCP servers
  • After 1 hour: 15 orphaned MCP server processes running
  • Memory Impact: ~400MB wasted, causing 70% memory usage + heavy swapping
  • System Impact: Session response times increase to 40+ seconds

Steps to Reproduce

  1. Configure a custom MCP server in OpenClaw gateway config (via plugins.entries)
  2. Start the gateway: openclaw gateway start
  3. Run multiple agent sessions that trigger the MCP server
  4. Wait 30-60 minutes for sessions to complete
  5. Check running processes: ps aux | grep <mcp-server-command>
  6. Observe memory pressure: free -h
  7. Try running: openclaw sessions --active 1440 --json

Expected Behavior

  • MCP servers should be automatically terminated when the associated session ends
  • No orphaned processes should accumulate
  • Memory usage should remain stable over time
  • CLI commands should not hang due to resource exhaustion

Current Workaround

A cron-based cleanup script runs every 10 minutes:

#!/bin/bash
# Kill orphaned MCP server processes
pkill -f "<mcp-server-command>"

Root Cause Analysis

The issue appears to be in the OpenClaw gateway MCP lifecycle management:

  1. MCP servers are spawned as child processes when a session needs them
  2. No cleanup/shutdown is triggered when the session ends
  3. No connection pooling or reuse mechanism exists
  4. Orphaned servers persist indefinitely until manually killed or the gateway restarts

Logs

[Mon Apr 13 02:45:17 AM UTC 2026] Running MCP cleanup...
[Mon Apr 13 02:45:17 AM UTC 2026] MCP servers before cleanup: 2
[Mon Apr 13 02:45:19 AM UTC 2026] MCP servers after cleanup: 0
[Mon Apr 13 02:45:19 AM UTC 2026] Killed 2 orphaned MCP servers

Impact Severity

  • High: Causes system instability and cron job failures on resource-constrained hosts
  • Affects: Any OpenClaw deployment using custom MCP servers
  • Frequency: Consistent — occurs every time sessions with MCP tools are created

Suggested Fix

  1. Immediate: Implement proper process cleanup when sessions end (SIGTERM to child MCP servers)
  2. Short-term: Add MCP server lifecycle management with reference counting
  3. Long-term: Implement connection pooling / server reuse instead of spawning per-session

Additional Context

  • Reproducible with any custom MCP server configured via plugins.entries
  • Memory leak accumulates over time until system becomes unstable
  • Gateway restart clears all orphans, but they re-accumulate after new sessions

Timeline

  • First observed: April 12, 2026
  • Workaround deployed: April 13, 2026 (cron cleanup script)
  • Next escalation: System becomes unstable after 4-6 hours without cleanup

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions