Skip to content

[Bug]: Auth profile cooldown is per-profile, not per-model — rate limit on one model blocks fallback to models with available capacity #43879

Description

@haynzz

Summary

Auth profile cooldown is tracked per-profile (usageStats[profileId].cooldownUntil) with no model dimension. When a profile hits a rate limit on Model A, its cooldown blocks it from being tried on Model B — even though Model B may have completely independent rate limits and available capacity.

Steps to reproduce

  1. Configure 3 auth profiles for the same provider (e.g., Anthropic): default (Max Pro), tst (Team), api (API console key)
  2. Configure model fallbacks: opus (primary) → sonnethaiku
  3. Exhaust rate limits on opus for profiles default and tst (but default still has sonnet capacity)
  4. Have the api profile at its monthly spending limit
  5. Send a message

Expected behavior

The model fallback chain should try default + sonnet — a working combination. The default profile's opus rate limit should not prevent it from being used with sonnet.

Actual behavior

All models failed (3): anthropic/claude-opus-4-6: LLM request rejected: You have reached
your specified API usage limits. (rate_limit) | anthropic/claude-sonnet-4-6: LLM request
rejected: You have reached your specified API usage limits. (rate_limit) |
anthropic/claude-haiku-4-5: LLM request rejected: You have reached your specified API
usage limits. (rate_limit)

The default profile was put into cooldown by its opus rate limit and never tried for sonnet.

Root cause

  1. markAuthProfileFailure() in src/agents/auth-profiles/usage.ts sets cooldownUntil per-profile with no model context
  2. isProfileInCooldown() returns true for the entire profile regardless of which model caused the cooldown
  3. The embedded runner's profile-skip loop (src/agents/pi-embedded-runner/run.ts ~L640) checks isProfileInCooldown(authStore, candidate) without model awareness
  4. The model-fallback outer loop (src/agents/model-fallback.ts) has a cooldownProbeUsedProviders partial mitigation, but it only probes once per provider per run — insufficient when different models have independent rate limit pools

Impact

This is most impactful for:

The configured fallback chain provides zero resilience when all profiles get rate-limited on the primary model, even when fallback models have available capacity on one or more profiles.

Related issues and PRs

Suggested fix

Add model context to cooldown tracking so rate-limit cooldowns are scoped to the model that caused them:

  1. Add cooldownModel?: string to ProfileUsageStats
  2. In markAuthProfileFailure, store the model that triggered the failure
  3. In isProfileInCooldown, add an optional forModel parameter — when the cooldown reason is rate_limit and cooldownModel differs from forModel, treat the profile as available
  4. Auth/billing failures remain profile-wide (not model-scoped) since they affect all models

This is surgical, backward-compatible (new field is optional), and preserves existing behavior for non-rate-limit failures.

OpenClaw version

2026.3.2

Operating system

Ubuntu 24.04 (Docker)

Install method

docker

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions