Skip to content

Add optional per-turn model fallback without persisting session override #80502

Description

@ginishuh

Summary

Add an optional mode for model fallbacks to be scoped to a single turn instead of persisting the fallback model as the active session override.

Problem

Model fallbacks are useful when a primary model is temporarily unavailable, rate-limited, or over capacity. However, when a fallback candidate succeeds, the runtime can persist that fallback provider/model into the session state as the active model selection.

That is useful for some workflows, but it can be surprising for agents that should prefer a high-quality primary model whenever it is available:

  1. Primary model fails transiently.
  2. Fallback model succeeds.
  3. Session continues using the fallback model on later turns.
  4. The configured/default primary is not retried unless the session/model state is reset.

For operational bots, the desired behavior is often: keep the reply alive with a fallback for this one turn, but try the configured primary again on the next user turn.

Proposal

Support a configurable fallback persistence policy, for example:

{
  "agents": {
    "list": [
      {
        "id": "example-agent",
        "model": {
          "primary": "openai-codex/gpt-5.5",
          "fallbacks": ["zai/glm-5-turbo"],
          "fallbackPersistence": "turn"
        }
      }
    ]
  }
}

Possible modes:

  • session — current behavior / durable fallback selection where appropriate.
  • turn — fallback is used only for the current run; after a successful fallback turn, restore the prior session model state so the next turn starts from the configured/default primary again.

The default should preserve existing behavior for compatibility.

Expected behavior for turn

Turn N:
  primary fails -> fallback succeeds -> reply is sent with fallback
  session model state is restored after success

Turn N+1:
  configured/default primary is tried again

Notes from local experiment

A local experiment implemented this by rolling back the temporary fallback candidate selection after runWithModelFallback returns a successful fallback result. The relevant area was src/auto-reply/reply/agent-runner-execution.ts, around the existing pendingFallbackCandidateRollback handling.

A regression test asserted that after a successful fallback turn, providerOverride, modelOverride, and modelOverrideSource are not left set to the fallback model.

This issue is intended to discuss making that behavior configurable rather than changing the global default unconditionally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🌊 off-meta tidepoolIssue quality rating does not apply to this item.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions