Problem
When all auth profiles for a provider enter cooldown due to rate limiting, runWithModelFallback immediately throws "All models failed" — even if the cooldown expires in just a few seconds. This surfaces as a hard error to end users on messaging channels (Discord, Telegram, etc.) for what is a brief, transient rate limit.
Error example
⚠ Agent failed before reply: All models failed (2):
anthropic/claude-sonnet-4-6: No available auth profile for anthropic (all in cooldown or unavailable). (rate_limit)
| anthropic/claude-sonnet-4-5: Provider anthropic is in cooldown (all profiles unavailable) (rate_limit)
Root cause
In src/agents/model-fallback.ts, runWithModelFallback() iterates over model candidates. When all profiles for a candidate are in cooldown, it skips that candidate immediately. If every candidate is skipped, the function throws without ever waiting for a cooldown to expire.
The existing shouldProbePrimaryDuringCooldown mechanism only probes the primary model (and only when fallback candidates exist). It does not cover the case where ALL candidates across ALL providers have all profiles in cooldown.
Contributing factors
- Cooldown escalation is aggressive:
calculateAuthProfileCooldownMs uses min(1hr, 1min × 5^(n-1)) — after just 3 errors, the cooldown is 25 minutes. After 4+, it's 1 hour.
- Stale error counts compound the problem: If a profile accumulates errors (e.g. from an expired token) and the
failureWindowHours hasn't elapsed, the next transient rate limit immediately escalates to a long cooldown. The clearExpiredCooldowns function addresses this when cooldowns expire naturally, but doesn't help when all profiles are simultaneously in cooldown.
- No "wait for recovery" path: The function has no mechanism to sleep until the soonest cooldown expires when the wait would be short (e.g. under 90 seconds).
Expected behavior
When all model candidates fail due to cooldowns, and the soonest cooldown expiry is within a reasonable window (e.g. 90 seconds), the function should wait for the cooldown to expire, clear it, and retry once before throwing.
Proposed fix
PR #24157 implements this: after exhausting all candidates, if all failures are cooldown-related and the soonest expiry is ≤90s away, sleep until expiry + clear expired cooldowns + retry once.
Environment
- OpenClaw version: 2026.2.21-2
- Auth: Multiple OAuth token profiles on the same provider
- Channels: Discord (most visible impact)
Problem
When all auth profiles for a provider enter cooldown due to rate limiting,
runWithModelFallbackimmediately throws "All models failed" — even if the cooldown expires in just a few seconds. This surfaces as a hard error to end users on messaging channels (Discord, Telegram, etc.) for what is a brief, transient rate limit.Error example
Root cause
In
src/agents/model-fallback.ts,runWithModelFallback()iterates over model candidates. When all profiles for a candidate are in cooldown, it skips that candidate immediately. If every candidate is skipped, the function throws without ever waiting for a cooldown to expire.The existing
shouldProbePrimaryDuringCooldownmechanism only probes the primary model (and only when fallback candidates exist). It does not cover the case where ALL candidates across ALL providers have all profiles in cooldown.Contributing factors
calculateAuthProfileCooldownMsusesmin(1hr, 1min × 5^(n-1))— after just 3 errors, the cooldown is 25 minutes. After 4+, it's 1 hour.failureWindowHourshasn't elapsed, the next transient rate limit immediately escalates to a long cooldown. TheclearExpiredCooldownsfunction addresses this when cooldowns expire naturally, but doesn't help when all profiles are simultaneously in cooldown.Expected behavior
When all model candidates fail due to cooldowns, and the soonest cooldown expiry is within a reasonable window (e.g. 90 seconds), the function should wait for the cooldown to expire, clear it, and retry once before throwing.
Proposed fix
PR #24157 implements this: after exhausting all candidates, if all failures are cooldown-related and the soonest expiry is ≤90s away, sleep until expiry + clear expired cooldowns + retry once.
Environment