Skip to content

Feature: Session memory/cost circuit breaker #16809

Description

@beca-oc

Problem

A single runaway session can consume unbounded memory and API costs with no automatic intervention. The gateway process will crash (or OOM-kill) before any safety mechanism triggers.

Impact (real incident, Feb 14 2026)

One session grew from 800MB to 3,021MB RSS in 20 minutes:

Time (UTC) RSS What happened
20:40 800MB Normal baseline
20:50 2,493MB 3x spike from polling loop
21:00 3,021MB Peak — gateway struggling
21:10 1,004MB Crash + recovery

The server has 6GB heap allocated. One session consumed 50% of total heap.

Estimated API cost for this single session: $150+ in ~2 hours (1,535 polls × 187k cached tokens each).

Proposed Solution

Add configurable per-session resource limits:

1. Memory guard

{
  "agents": {
    "defaults": {
      "session": {
        "maxRssMb": 2048,
        "maxRssAction": "compact-or-kill"
      }
    }
  }
}

When a session pushes total gateway RSS above threshold:

  • First: force compaction on the largest session
  • If RSS still high after compaction: archive the session and start fresh
  • Notify the user on their configured channel

2. Session cost tracking

Track per-session: input tokens, output tokens, cache reads. Surface in session_status. Alert if a single session exceeds a configurable threshold (e.g., $50).

3. Session message count guard

If a session exceeds N messages (e.g., 500) without compaction, force compaction. The Feb 14 session hit 1,535 messages — this should never happen.

Why This Matters

On a resource-constrained server (common for self-hosted OpenClaw), one bad session can take down the entire gateway, affecting all agents and all channels. There is no isolation between sessions for memory.

Environment

  • OpenClaw 2026.2.13
  • Ubuntu 24.04, 6GB Node.js heap (--max-old-space-size=6144)
  • Gateway running as single Node.js process serving all agents/channels

Metadata

Metadata

Assignees

No one assigned

    Labels

    staleMarked as stale due to inactivity

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions