Skip to content

Auto-reset session when all models time out (session bloat death spiral) #20910

Description

@tag-assistant

Problem

When a session accumulates a very large transcript (e.g. 7,069 messages, 205 image references, 20MB on disk, 170k tokens), the gateway enters a death spiral:

  1. Incoming message triggers an embedded agent run
  2. sanitizeSessionMessagesImages() iterates ALL messages, resizing images via Sharp — takes significant time on a large session
  3. The serialized transcript is sent to the LLM API
  4. The 120s timeout fires before the request completes (often during prep, not even network)
  5. Failover tries the next model — same timeout
  6. All 3 fallback models time out → FailoverError: "All models failed (3): ... LLM request timed out"
  7. The error message is returned to the user
  8. The next incoming message triggers another run → goto step 1

This repeats indefinitely. In production, 217 consecutive timeout retries were observed over several hours before manual intervention (clearing the session store and restarting the gateway).

Root Cause

The error "LLM request timed out" does not match any of the existing auto-recovery checks:

  • isLikelyContextOverflowError() — no, it's a timeout not a context overflow API error
  • isCompactionFailureError() — no
  • isTransientHttpError() — no
  • Session corruption / role ordering patterns — no

So the timeout falls through to the generic error path, which just returns a user-facing error message. Since each incoming message starts a fresh run against the same bloated session, the cycle never breaks.

Proposed Fix

In src/auto-reply/reply/agent-runner-execution.ts, detect when ALL models have timed out and auto-reset the session, similar to the existing compaction failure recovery.

The negative lookahead excludes rate-limit timeouts (which should be handled by cooldown logic, not session reset).

Impact

  • Without fix: A single bloated session can render the bot completely unresponsive for hours, consuming resources on every retry
  • With fix: The session auto-resets on the first "all timed out" failure, immediately recovering functionality

Metadata

Metadata

Assignees

No one assigned

    Labels

    staleMarked as stale due to inactivity

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions