Skip to content

[Bug]: [Bug] LLM idle timeout triggers unbounded retry loop, causing massive cost spikes #76293

Description

@ThePuma312

Bug type

Behavior bug (incorrect output/state without crash)

Beta release blocker

No

Summary

Summary

When an LLM API call times out (error: LLM idle timeout (Ns): no response from model), OpenClaw appears to retry the request without any bounds — no maximum retry count, no exponential backoff, no circuit breaker. This causes runaway loops that fire hundreds or thousands of API calls in seconds, generating unexpected cost spikes when using paid providers.

Steps to reproduce

Reproduction evidence

In our deployment, this bug manifested twice in 11 days during normal heartbeat operations:

Incident 1 — April 25, 2026 at 13:02 ET:

  • Single heartbeat fire timed out
  • Retry loop generated 761 API call entries within 60 seconds
  • All retries logged at the same minute timestamp (2026-04-25T13:02)

Incident 2 — April 29, 2026 at 01:02 UTC (21:02 ET on April 28):

  • Single heartbeat fire timed out
  • Retry loop generated 1,384 API call entries within 60 seconds
  • All retries logged at the same minute timestamp (2026-04-29T01:02)

Both incidents triggered when the agent was running on Anthropic Sonnet 4.6 and the API call exceeded the configured idle timeout.

Verification command

grep -oE '"timestamp":"YYYY-MM-DDTHH:MM' ~/.openclaw/agents//sessions/*.jsonl | wc -l

Error signature

The triggering error appears in session JSONL as:

{"type":"custom","customType":"openclaw:prompt-error","data":{
  "error":"LLM idle timeout (60s): no response from model",
  "provider":"anthropic",
  "model":"claude-sonnet-4-6",
  ...
}}

(Also observed with provider: "ollama", model qwen3:14b — confirming provider-agnostic root cause.)

Expected behavior

Expected behavior

On LLM timeout:

  • Retry with exponential backoff (e.g., 1s, 2s, 4s, 8s)
  • Cap retries at a reasonable maximum (e.g., 3-5 attempts)
  • After max retries exhausted, log error and halt the operation
  • Optionally surface alert through configured notification channel

Actual behavior

Actual behavior

Single timeout triggers a tight retry loop that fires until something external stops it. No backoff between retries. No cap on attempts.

OpenClaw version

OpenClaw version: 2026.4.10 (build 44e5b62)

Operating system

mac0S Tahoe 26.4.1

Install method

homebrew

Model

claude-sonnet-4-6 (also reproduced with qwen3:14b — bug is provider-agnostic)

Provider / routing chain

anthropic primary, also reproduced with ollama (bug is in OpenClaw retry layer above provider abstraction)

Additional provider/model setup details

No response

Logs, screenshots, and evidence

Impact and severity

Impact

CRITICAL severity — financial exposure. Each retry processes the full session context (input tokens), so on a large session each retry can cost $0.50-$2.00 in API charges. A 1,384-retry loop running on a moderately-sized session costs $20-30 in 60 seconds with no user awareness until billing reflects it.

For users with auto-reload enabled on their Anthropic account (recommended for reliability), the financial alarm is suppressed — auto-recharge masks the spike, and the user discovers it days later when reviewing usage.

Additional information

Workaround

None at the OpenClaw level. We've implemented external mitigations:

  • Daily session size monitor + auto-rotation (caps blast radius by limiting context size)
  • Daily cost monitor with Telegram alerts (catches spikes within 24 hours)
  • These are damage control, not prevention

Suggested fix

Implement standard retry logic with bounds:

  1. Maximum 3-5 retry attempts per request
  2. Exponential backoff between retries (e.g., 2^n seconds)
  3. After max retries, surface error and halt
  4. Optionally make retry behavior configurable via agents.list[].heartbeat.retry block

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingbug:behaviorIncorrect behavior without a crash

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions