fix: expire automatic model fallback pins by markus-lassfolk · Pull Request #79678 · openclaw/openclaw

markus-lassfolk · 2026-05-09T06:24:33Z

Summary

add a 30s TTL to automatic model fallback pins (modelOverrideSource: "auto")
clear expired automatic fallback pins before agent/auto-reply model selection so the next turn retries the configured default/primary model
preserve explicit user overrides and legacy auto overrides without an expiry field
document the automatic fallback-pin expiry in docs/concepts/model-failover.md

Real behavior proof

Redacted local proof against the PR branch using OpenClaw's actual createModelSelectionState path and a temporary session store:

$ node --import tsx scripts/proof-model-fallback-ttl.mjs
OpenClaw automatic model fallback TTL proof
tempSessionStore $TMPDIR/sessions.json
before {"providerOverride":"fallback-provider","modelOverride":"fallback-model","modelOverrideSource":"auto","modelOverrideExpiresAtState":"expired timestamp present","updatedAtChanged":false}
selected primary-provider/primary-model
after {"providerOverride":null,"modelOverride":null,"modelOverrideSource":null,"modelOverrideExpiresAtState":null,"updatedAtChanged":true}
result PASS expired auto fallback pin cleared; configured primary retried

What this proves:

the session store started with a persisted auto fallback pin (fallback-provider/fallback-model) and an expired modelOverrideExpiresAt
model selection cleared the auto-owned override fields from the session store
the selected model for the turn returned to the configured primary/default (primary-provider/primary-model)
the persisted session entry was updated after clearing the expired auto pin

Verification

pnpm test -- --run src/sessions/model-overrides.test.ts src/auto-reply/reply/model-selection.test.ts src/auto-reply/reply/agent-runner-execution.test.ts
- src/sessions/model-overrides.test.ts: 9 passed
- src/auto-reply/reply/model-selection.test.ts: 32 passed
- src/auto-reply/reply/agent-runner-execution.test.ts: 68 passed
NODE_OPTIONS=--max-old-space-size=8192 pnpm tsgo:core:test
pnpm format:check docs/concepts/model-failover.md src/sessions/model-overrides.ts src/sessions/model-overrides.test.ts src/agents/agent-command.ts src/auto-reply/reply/model-selection.ts src/auto-reply/reply/model-selection.test.ts src/auto-reply/reply/agent-runner-execution.ts src/auto-reply/reply/agent-runner-execution.test.ts src/config/sessions/types.ts src/plugins/session-entry-slot-keys.ts

chatgpt-codex-connector · 2026-05-09T06:24:37Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

clawsweeper · 2026-05-09T06:27:52Z

Thanks for the context here. I did a careful shell check against current main, and this is already implemented.

Current main and the latest release already solve the central sticky auto-fallback-pin problem with shipped primary recovery probes, so this conflicting TTL branch is no longer needed.

So I’m closing this as already implemented rather than keeping a duplicate issue open.

Review details

Best possible solution:

Keep the shipped primary-probe recovery implementation and close this parallel TTL branch instead of adding another persisted expiry field.

Do we have a high-confidence way to reproduce the issue?

No, not on current main: source, docs, and tests show auto fallback pins are now periodically probed and cleared after primary recovery. The PR body does provide a valid branch-side terminal proof for the old TTL approach.

Is this the best way to solve the issue?

Yes, current main is the better repository solution: it clears auto fallback state after a real primary recovery probe instead of expiring pins after a fixed 30 seconds regardless of provider health.

Security review:

Security review cleared: No concrete security or supply-chain concern was found; the PR changes session model-selection logic, tests, docs, and a reserved session key only.

AGENTS.md: found and applied where relevant.

What I checked:

Repository policy read: Read the full root AGENTS.md and scoped docs/agents/plugins guides; the review applied the fallback/session compatibility guidance to check source, docs, tests, release provenance, and related fallback behavior. (AGENTS.md:1, 23e9bc8c0b61)
Current main implements primary-probe recovery: Current main tracks a five-minute auto-fallback primary probe interval, resolves eligible auto fallback pins back to their recorded origin model, and has a helper that clears the auto-owned override fields after recovery. (src/agents/agent-scope.ts:50, 23e9bc8c0b61)
Agent runtime uses and clears probes: The agent command path switches eligible auto fallback sessions to the recorded primary probe model and clears the auto pin when the probe succeeds, otherwise it preserves/re-pins fallback state with origin metadata. (src/agents/agent-command.ts:1010, 23e9bc8c0b61)
Current docs describe shipped behavior: The model failover docs now say auto fallback overrides record the selected origin, periodically probe it again, clear on recovery, and use a non-configurable five-minute primary-probe interval. Public docs: docs/concepts/model-failover.md. (docs/concepts/model-failover.md:63, 23e9bc8c0b61)
Current tests cover recovery and throttling: Follow-up runner coverage verifies that a successful queued primary probe clears providerOverride, modelOverride, modelOverrideSource, and origin metadata, while a not-yet-due probe keeps the fallback pin and auth profile. (src/auto-reply/reply/followup-runner.test.ts:633, 23e9bc8c0b61)
Git-history provenance: Blame on the current auto fallback primary-probe implementation points at the recent release-line commit carrying the recovery behavior in this checkout; older history also shows the model fallback area was introduced and maintained through dedicated fallback commits. (src/agents/agent-scope.ts:101, d00d0a21c2dd)

Likely related people:

Peter Steinberger: Blame/log history ties the current auto fallback primary-probe implementation, model fallback docs, and recent release-line proof commits to Peter Steinberger. (role: recent area contributor; confidence: high; commits: d00d0a21c2dd, a374c3a5bfd5, fe72474153da; files: src/agents/agent-scope.ts, src/agents/agent-command.ts, docs/concepts/model-failover.md)
Vincent Koc: History shows Vincent Koc authored the safe model fallback feature work that introduced much of the fallback behavior this PR and the shipped recovery path build on. (role: feature-history contributor; confidence: medium; commits: c96a12aeb9e8, 93ce76afe310, 95517edaeba4; files: src/agents/agent-scope.ts, src/auto-reply/reply/agent-runner-execution.ts, docs/concepts/model-failover.md)
Eva H: History shows Eva H worked on preventing fallback persistence from clobbering user model selections, which is adjacent to auto/user override ownership in this review. (role: adjacent fallback persistence contributor; confidence: medium; commits: 3b139862142d; files: src/sessions/model-overrides.ts, src/auto-reply/reply/model-selection.ts)

Codex review notes: model gpt-5.5, reasoning high; reviewed against 23e9bc8c0b61; fix evidence: release v2026.5.22, commit a374c3a5bfd5.

Copilot

Pull request overview

This PR adds an expiry mechanism for automatically-created (“auto”) session model override pins so that temporary fallback selections will fail back to the configured default model after a short TTL, instead of persisting indefinitely across turns.

Changes:

Introduces modelOverrideExpiresAt on SessionEntry plus helpers to compute/clear expiry for auto overrides.
Clears expired auto overrides early in agent/model selection paths so subsequent turns retry the primary/default model.
Extends fallback pin persistence logic and tests to cover TTL behavior and legacy/non-expiring overrides.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/sessions/model-overrides.ts	Adds TTL constant, expiry detection, and clearing logic; writes expiry when applying auto overrides.
src/sessions/model-overrides.test.ts	Adds coverage for expiry creation, expiry clearing, and legacy/user override behavior.
src/plugins/session-entry-slot-keys.ts	Reserves `modelOverrideExpiresAt` to prevent plugin slot collisions.
src/config/sessions/types.ts	Adds `modelOverrideExpiresAt?: number` to `SessionEntry`.
src/auto-reply/reply/model-selection.ts	Clears expired auto overrides before persisted override resolution (and persists store updates when applicable).
src/auto-reply/reply/model-selection.test.ts	Verifies expired auto overrides are cleared and default model is re-selected.
src/auto-reply/reply/agent-runner-execution.ts	Includes `modelOverrideExpiresAt` in fallback selection state snapshots and sets expiry when pinning fallbacks.
src/auto-reply/reply/agent-runner-execution.test.ts	Asserts fallback pins now include a numeric expiry timestamp.
src/agents/agent-command.ts	Clears expired auto overrides before model selection and persists updated session entry.

markus-lassfolk · 2026-05-09T16:47:13Z

Added the requested follow-up in 11776d637a:

documented the auto fallback-pin TTL in docs/concepts/model-failover.md
added redacted real-behaviour proof to the PR body
reran the requested targeted tests, core test typecheck, and focused format check

The proof uses OpenClaw's actual model-selection path with a temporary session store. It starts with an expired persisted modelOverrideSource: "auto" fallback pin and verifies that selection clears the override and returns to the configured primary/default model.

Fresh verification:

$ node --import tsx scripts/proof-model-fallback-ttl.mjs
OpenClaw automatic model fallback TTL proof
tempSessionStore $TMPDIR/sessions.json
before {"providerOverride":"fallback-provider","modelOverride":"fallback-model","modelOverrideSource":"auto","modelOverrideExpiresAtState":"expired timestamp present","updatedAtChanged":false}
selected primary-provider/primary-model
after {"providerOverride":null,"modelOverride":null,"modelOverrideSource":null,"modelOverrideExpiresAtState":null,"updatedAtChanged":true}
result PASS expired auto fallback pin cleared; configured primary retried

pnpm test -- --run src/sessions/model-overrides.test.ts src/auto-reply/reply/model-selection.test.ts src/auto-reply/reply/agent-runner-execution.test.ts
# 109 tests passed across the 3 targeted files

NODE_OPTIONS=--max-old-space-size=8192 pnpm tsgo:core:test
# passed

pnpm format:check docs/concepts/model-failover.md src/sessions/model-overrides.ts src/sessions/model-overrides.test.ts src/agents/agent-command.ts src/auto-reply/reply/model-selection.ts src/auto-reply/reply/model-selection.test.ts src/auto-reply/reply/agent-runner-execution.ts src/auto-reply/reply/agent-runner-execution.test.ts src/config/sessions/types.ts src/plugins/session-entry-slot-keys.ts
# All matched files use the correct format.

@clawsweeper re-review

Re-review progress:

State: Complete
Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
Run: https://github.com/openclaw/clawsweeper/actions/runs/25606393577
Updated: 2026-05-09T16:54:05.776Z

openclaw-barnacle · 2026-05-26T04:45:07Z

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

clawsweeper · 2026-05-26T04:53:11Z

ClawSweeper applied the proposed close for this PR.

Action: closed this PR.
Close reason: already implemented on main.
Evidence: durable ClawSweeper review.

fix: expire automatic model fallback pins

3662142

Copilot AI review requested due to automatic review settings May 9, 2026 06:24

openclaw-barnacle Bot added agents Agent runtime and tooling size: M triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 9, 2026

Copilot started reviewing on behalf of markus-lassfolk May 9, 2026 06:25 View session

Copilot AI reviewed May 9, 2026

View reviewed changes

docs: document auto fallback pin expiry

11776d6

openclaw-barnacle Bot added the docs Improvements or additions to documentation label May 9, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 9, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 9, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 12, 2026

openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 12, 2026

clawsweeper Bot mentioned this pull request May 13, 2026

fix(agent): prevent sticky model fallback #75270

Open

cael-dandelion-cult mentioned this pull request May 18, 2026

Cherry-pick upstream #79678 (model-failback-ttl-30s) to fleet — dissolves codex/gpt snapback disease karmaterminal/openclaw#708

Open

adele-with-a-b mentioned this pull request May 20, 2026

Feature request: gateway.routing.persist_auto_fallback config flag #83731

Closed

clawsweeper Bot mentioned this pull request May 20, 2026

fix: clear auto fallback pins after ephemeral primary failures #84476

Closed

openclaw-barnacle Bot added the stale Marked as stale due to inactivity label May 26, 2026

clawsweeper Bot closed this May 26, 2026

openclaw-barnacle Bot removed the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 26, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix: expire automatic model fallback pins#79678

fix: expire automatic model fallback pins#79678
markus-lassfolk wants to merge 2 commits into
openclaw:mainfrom
markus-lassfolk:maeve/model-failback-ttl-30s

markus-lassfolk commented May 9, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented May 9, 2026

Uh oh!

clawsweeper Bot commented May 9, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

markus-lassfolk commented May 9, 2026 •

edited by clawsweeper Bot

Loading

Uh oh!

openclaw-barnacle Bot commented May 26, 2026

Uh oh!

clawsweeper Bot commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

markus-lassfolk commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Real behavior proof

Verification

Uh oh!

chatgpt-codex-connector Bot commented May 9, 2026

Uh oh!

clawsweeper Bot commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

markus-lassfolk commented May 9, 2026 • edited by clawsweeper Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openclaw-barnacle Bot commented May 26, 2026

Uh oh!

clawsweeper Bot commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

markus-lassfolk commented May 9, 2026 •

edited

Loading

clawsweeper Bot commented May 9, 2026 •

edited

Loading

markus-lassfolk commented May 9, 2026 •

edited by clawsweeper Bot

Loading