fix(agents): probe primary after auto fallback pin by steipete · Pull Request #82707 · openclaw/openclaw

steipete · 2026-05-16T18:55:39Z

Summary

Periodically probe the configured primary for auto fallback pins and clear the pin on recovery.
Keep failed probes pinned to fallback with throttling, auth preservation, queued recheck, concurrent-change guards, and live-switch guards.
Document behavior and add the changelog entry for Session model pinning persists indefinitely after fallback, never reconciles with configured primary — confirmed across two independent occurrences (v2026.4.15 and v2026.5.6) #82544.

Verification

node scripts/run-vitest.mjs src/agents/command/attempt-execution.shared.test.ts src/agents/agent-scope.test.ts src/auto-reply/reply/get-reply.config-override.test.ts src/agents/agent-command.live-model-switch.test.ts src/auto-reply/reply/model-selection.test.ts src/auto-reply/reply/agent-runner-execution.test.ts src/auto-reply/reply/followup-runner.test.ts
node_modules/.bin/tsgo -p test/tsconfig/tsconfig.test.src.json --pretty false --incremental false
git diff --check
/Users/steipete/Projects/agent-scripts/skills/codex-review/scripts/codex-review --mode local --output /tmp/codex-review-82544-final19.txt --parallel-tests "node scripts/run-vitest.mjs src/agents/command/attempt-execution.shared.test.ts src/agents/agent-scope.test.ts src/auto-reply/reply/model-selection.test.ts src/auto-reply/reply/get-reply.config-override.test.ts src/agents/agent-command.live-model-switch.test.ts src/auto-reply/reply/agent-runner-execution.test.ts src/auto-reply/reply/followup-runner.test.ts"

Real behavior proof

Behavior addressed: Auto-pinned fallback sessions periodically retry the configured primary and clear the pin when it succeeds.
Real environment tested: Local OpenClaw source checkout at b098e77 on macOS, Node 26.0.0.
Exact steps or command run after this patch: Ran a production helper probe with node --import tsx --input-type=module against src/agents/agent-scope.ts.
Evidence after fix: Terminal output from the local OpenClaw runtime probe:

{
  "firstProbe": {
    "provider": "openai",
    "model": "gpt-5.5",
    "fallbackProvider": "openrouter",
    "fallbackModel": "minimax/minimax-m2.7",
    "fallbackAuthProfileId": "openrouter:fallback",
    "fallbackAuthProfileIdSource": "auto"
  },
  "throttledProbe": null,
  "recoveredEntry": {
    "sessionId": "session-82544",
    "updatedAt": 70000
  }
}

Observed result after fix: The persisted auto fallback produced a primary probe for openai/gpt-5.5, the immediate retry was throttled, and the recovered primary clear removed provider/model/auth fallback overrides from the session entry.
What was not tested: Live provider recovery against a real remote model backend.

Fixes #82544

clawsweeper · 2026-05-16T18:56:28Z

Codex review: needs real behavior proof before merge.

Summary
The PR adds throttled primary-model probes for auto-pinned fallback sessions, clears matching recovered pins, preserves fallback auth during failed probes, and updates tests, docs, and changelog.

Reproducibility: yes. Current main source shows auto fallback selections are persisted and reused as stored session overrides, and the linked issue supplies concrete production occurrences; I did not run a live provider outage/recovery repro in this read-only review.

Real behavior proof
Needs real behavior proof before merge: The PR body reports focused tests, typecheck, diff check, and codex-review, but no after-fix real runtime proof of a recovered primary clearing a persisted fallback pin. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, ask a maintainer to comment @clawsweeper re-review.

Next step before merge
Protected maintainer labeling and missing real behavior proof make this a human review/validation item rather than an autonomous repair lane.

Security
Cleared: The diff changes TypeScript runtime logic, tests, docs, and changelog only; I found no concrete security or supply-chain regression.

Review details

Best possible solution:

Land a reviewed recovery policy that retries the configured primary for auto-sourced pins, clears only matching auto state, preserves user/live model switches, and tracks any no-visible-reply persistence policy separately if needed.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main source shows auto fallback selections are persisted and reused as stored session overrides, and the linked issue supplies concrete production occurrences; I did not run a live provider outage/recovery repro in this read-only review.

Is this the best way to solve the issue?

Yes, with a proof caveat. A throttled primary probe for auto-sourced pins is a narrow maintainable fix that preserves explicit user selections, but it still needs real behavior proof before merge.

Acceptance criteria:

Inspect contributor-provided real behavior proof showing an auto-pinned fallback session retrying the configured primary and clearing the pin after recovery.
Review reported focused checks from the PR body: node scripts/run-vitest.mjs src/agents/command/attempt-execution.shared.test.ts src/agents/agent-scope.test.ts src/auto-reply/reply/get-reply.config-override.test.ts src/agents/agent-command.live-model-switch.test.ts src/auto-reply/reply/model-selection.test.ts src/auto-reply/reply/agent-runner-execution.test.ts src/auto-reply/reply/followup-runner.test.ts; node_modules/.bin/tsgo -p test/tsconfig/tsconfig.test.src.json --pretty false --incremental false; git diff --check.

What I checked:

Protected label: The provided live PR context shows labels docs, agents, maintainer, and size: XL; the protected maintainer label requires explicit maintainer handling rather than cleanup closing.
Current main source reproduces sticky auto fallback state: Current main persists fallback selections into session override fields and treats legacy non-auto overrides as user state, which explains why auto-created pins can become later-turn starting state without a recovery probe. (src/auto-reply/reply/agent-runner-execution.ts:1344, 1586085c7fcb)
Current main stored override selection: Current main reads stored provider/model overrides before the run is constructed, so a persisted auto fallback can override the configured primary on the next command turn. (src/agents/agent-command.ts:826, 1586085c7fcb)
PR implementation surface: The proposed head commit adds primary probe helpers, routes the probe through reply/command runners, and updates model fallback docs and tests across 19 files. (src/agents/agent-scope.ts, b098e770a5c8)
Related issue discussion: The linked issue reports two production occurrences and an earlier ClawSweeper comment kept it open after source review because current main and v2026.5.12 still preserved auto-created fallback model pins across later turns.
Real behavior proof gap: The PR body lists focused Vitest, test-source tsgo, git diff check, and codex-review output, and explicitly says live provider recovery against a real remote backend was not tested. (b098e770a5c8)

Likely related people:

steipete: Current-main blame and GitHub commit metadata connect Peter Steinberger to selected session model resolution and the central fallback/session files affected here. (role: recent fallback area contributor; confidence: high; commits: 95741daeb4c2, 1fb44f0aadd4; files: src/auto-reply/reply/agent-runner-execution.ts, src/auto-reply/reply/model-selection.ts, src/agents/agent-command.ts)
frankekn: Commit metadata for the merged session reset fix shows related work clearing auto-fallback model overrides while preserving explicit user selections. (role: introduced related reset behavior; confidence: medium; commits: 5d46e4dc4f45; files: src/config/sessions/reset-preserved-selection.ts, src/sessions/model-overrides.ts, src/auto-reply/reply/agent-runner-execution.ts)
joshavant: Commit metadata shows recent fallback provenance work in the same session override and runner execution area. (role: recent adjacent contributor; confidence: medium; commits: 1dac68c0bbe7; files: src/agents/agent-command.ts, src/auto-reply/reply/agent-runner-execution.ts, src/agents/agent-scope.ts)

Remaining risk / open question:

No live or real local provider recovery proof is attached; the PR body explicitly leaves real backend recovery untested.
This read-only sweep did not run tests, so the verdict relies on source inspection, PR metadata, and the contributor's reported focused test commands.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 1586085c7fcb.

steipete · 2026-05-16T19:02:48Z

Verification before merge:

Local proof:

node scripts/run-vitest.mjs src/agents/command/attempt-execution.shared.test.ts src/agents/agent-scope.test.ts src/auto-reply/reply/get-reply.config-override.test.ts src/agents/agent-command.live-model-switch.test.ts src/auto-reply/reply/model-selection.test.ts src/auto-reply/reply/agent-runner-execution.test.ts src/auto-reply/reply/followup-runner.test.ts
node_modules/.bin/tsgo -p test/tsconfig/tsconfig.test.src.json --pretty false --incremental false
git diff --check
/Users/steipete/Projects/agent-scripts/skills/codex-review/scripts/codex-review --mode local --output /tmp/codex-review-82544-final19.txt --parallel-tests "node scripts/run-vitest.mjs src/agents/command/attempt-execution.shared.test.ts src/agents/agent-scope.test.ts src/auto-reply/reply/model-selection.test.ts src/auto-reply/reply/get-reply.config-override.test.ts src/agents/agent-command.live-model-switch.test.ts src/auto-reply/reply/agent-runner-execution.test.ts src/auto-reply/reply/followup-runner.test.ts"

Local result:

Focused Vitest: 10 files passed, 306 tests passed.
Test-source tsgo: passed.
git diff --check: passed.
codex-review: clean, no accepted/actionable findings.

Real behavior proof:

Ran node --import tsx --input-type=module against src/agents/agent-scope.ts in the local OpenClaw source checkout.
Observed primary probe openai/gpt-5.5 for an auto-pinned fallback, immediate throttle returned null, and primary recovery clear removed persisted fallback provider/model/auth fields.

CI:

CI run: https://github.com/openclaw/openclaw/actions/runs/25970133313
Real behavior proof success: https://github.com/openclaw/openclaw/actions/runs/25970277107
Security/CodeQL/OpenGrep checks: passing in PR rollup.

Known proof gap:

No live remote provider recovery test; verified state-machine behavior locally and covered runner paths with focused tests.

fix(agents): probe primary after auto fallback pin

b098e77

openclaw-barnacle Bot added docs Improvements or additions to documentation agents Agent runtime and tooling size: XL maintainer Maintainer-authored PR labels May 16, 2026

clawsweeper Bot added the P1 High-priority user-facing bug, regression, or broken workflow. label May 16, 2026

steipete merged commit c8782d1 into main May 16, 2026
126 of 133 checks passed

steipete deleted the fix-auto-fallback-primary-probe-82544 branch May 16, 2026 19:03

clawsweeper Bot mentioned this pull request May 16, 2026

fix(agents): announce auto model fallback transitions #82676

Merged

github-actions Bot mentioned this pull request May 16, 2026

📡 Upstream Digest — 2026-05-16 20:39 UTC curtismercier/openclaw-mods#875

Open

clawsweeper Bot mentioned this pull request May 16, 2026

[Feature]: Bidirectional fallback notifications — alert on switch AND on recovery, per-session/per-topic #44941

Closed

adele-with-a-b mentioned this pull request May 20, 2026

Feature request: gateway.routing.persist_auto_fallback config flag #83731

Closed

clawsweeper Bot mentioned this pull request May 20, 2026

claude-cli backend silently skipped on systemd-managed root gateway, never spawns subprocess #70279

Closed

galiniliev pushed a commit to galiniliev/openclaw that referenced this pull request May 20, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

3ce2940

SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 24, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

66004a5

SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 24, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

791da27

SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 24, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

67628bc

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

1b25bc8

galiniliev pushed a commit to galiniliev/openclaw that referenced this pull request May 25, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

835b2c7

SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 26, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

9b67e91

SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 26, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

b6fb3d1

SebTardif pushed a commit to SebTardif/openclaw that referenced this pull request May 26, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

ae6312f

jameslcowan pushed a commit to jameslcowan/openclaw that referenced this pull request Jun 2, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

756d9a5

SYU8384 pushed a commit to SYU8384/openclaw that referenced this pull request Jun 3, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

e20b08d

sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026

fix(agents): probe primary after auto fallback pin (openclaw#82707)

f8a8217

This was referenced Jun 15, 2026

Agent always returns "Provider returned error" with auto model after gateway restart #87170

Closed

fix(agent): prevent sticky model fallback #75270

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(agents): probe primary after auto fallback pin#82707

fix(agents): probe primary after auto fallback pin#82707
steipete merged 1 commit into
mainfrom
fix-auto-fallback-primary-probe-82544

steipete commented May 16, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented May 16, 2026 •

edited

Loading

Uh oh!

steipete commented May 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

steipete commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Verification

Real behavior proof

Uh oh!

clawsweeper Bot commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented May 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

steipete commented May 16, 2026 •

edited

Loading

clawsweeper Bot commented May 16, 2026 •

edited

Loading