Skip to content

[Bug]: live session model switch blocks failover on overloaded_error — infinite retry loop #58466

Description

@Falyalot

Bug type

Regression (worked before, now fails)

Beta release blocker

No

Summary

When a session's primary model returns overloaded_error (HTTP 503), OpenClaw attempts failover to the next model in the fallback chain. However, the "live session model
switch" mechanism detects that the new model differs from the session's original model and forces a revert back to the original (overloaded) model, creating an infinite retry
loop with no circuit breaker.

Steps to reproduce

  1. Configure agent with primary anthropic/claude-sonnet-4-6 and fallback openai-codex/gpt-5.4
  2. Start a session (any context size — reproduced with both small and ~330K token contexts)
  3. Anthropic API returns overloaded_error (503)
  4. Observe logs: failover decides to try gpt-5.4, then "live session model switch" reverts to sonnet
  5. Sonnet still overloaded → retry → revert → infinite loop

Expected behavior

On overloaded_error, failover to the next model in the fallback chain should succeed. The "live session model switch" guard should not fire during failover scenarios.

Actual behavior

"Live session model switch" treats a legitimate failover as a disallowed model change, reverting back to the overloaded model. This creates an infinite loop: overloaded →
failover → revert → overloaded.

OpenClaw version

OpenClaw v2026.3.28 (f9b1079)

Operating system

Ubuntu 24.04, Node.js v22.22.0, VPS

Install method

No response

Model

Primary model: anthropic/claude-sonnet-4-6

Provider / routing chain

Fallbacks: openai-codex/gpt-5.4, groq-llm/llama-3.3-70b-versatile

Additional provider/model setup details

No response

Logs, screenshots, and evidence

embedded_run_agent_end: isError=true, failoverReason="overloaded", model="claude-sonnet-4-6"                                                                                  
                                                                                                                                                                                
  model_fallback_decision: decision="candidate_failed", candidateModel="claude-sonnet-4-6",                                                                                     
    nextCandidateProvider="openai-codex", nextCandidateModel="gpt-5.4"                                                                                                          
                                                                                                                                                                                
  live session model switch detected before attempt for <session-id>:                                                                                                           
    openai-codex/gpt-5.4 -> anthropic/claude-sonnet-4-6                                                                                                                         
                                                                                                                                                                                
  model_fallback_decision: decision="candidate_failed", candidateModel="gpt-5.4",                                                                                               
    errorPreview="Live session model switch requested: anthropic/claude-sonnet-4-6"                                                                                             
                                                                                                                                                                                
  The third fallback candidate (e.g. groq-llm/llama-3.3-70b-versatile) is never reached — the loop restarts after the model switch reverts the second candidate.

Impact and severity

  • In our incident: 7+ hours of continuous retry loop across multiple agents
  • Anthropic resolved their outage in ~26 minutes — OpenClaw agents continued looping for 6+ more hours
  • Each retry sends the full context to the API server before getting rejected
  • Multiple agents affected simultaneously (all sharing the same primary model)
  • No self-stopping mechanism — loop continues until gateway is manually killed

Additional information

Workaround

External bash watchdog (systemd timer) — counts overloaded_error in gateway logs, stops gateway if threshold exceeded. Prevents token waste but kills all agents rather than
gracefully failing over.

Suggested Fix

  1. Failover path should bypass "live session model switch" guard — overloaded_error is a valid reason to switch models mid-session
  2. Add circuit breaker — max N retries per model per time window before marking it temporarily unavailable
  3. Allow fallback chain to complete — currently the third candidate is never tried because the revert happens before it's reached

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingregressionBehavior that previously worked and now fails

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions