Skip to content

fix(status): surface fallback model selections in status mismatch detection#96151

Closed
LZY3538 wants to merge 2 commits into
openclaw:mainfrom
LZY3538:fix/status-model-selection-fallback-reason
Closed

fix(status): surface fallback model selections in status mismatch detection#96151
LZY3538 wants to merge 2 commits into
openclaw:mainfrom
LZY3538:fix/status-model-selection-fallback-reason

Conversation

@LZY3538

@LZY3538 LZY3538 commented Jun 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Fixes #96126: openclaw status and session_status collapse the active (fallback) model into the same Model: line as the configured primary, making it impossible to see at a glance whether a session is running its intended model or a fallback.

What Problem This Solves

Operators running multi-model setups with fallback chains cannot tell whether a session is on its configured primary model or an automatic fallback. A session silently running on a fallback model shows no indication in openclaw status output.

Changes (v2 — ClawSweeper rank-up)

4 files, +56/-11:

  • status.summary.ts: Changed mismatch detection from hasUserPinnedModelSelection() to entry?.modelOverride != null so auto-fallback selections are also flagged
  • status.command-sections.ts: Removed modelSelectionReason filter + fallback-specific wording
  • status.summary.test.ts + status.command-sections.test.ts: Added coverage

Real behavior proof

  • Behavior addressed: Auto-fallback model selections hidden from openclaw status mismatch detection.
  • Real environment tested: Linux Node 24, live Gateway (1.8.0), full repository checkout
  • Exact steps or command run after this patch: 
    # Live Gateway status check — command executes against real Gateway:
$ openclaw status
    # Unit tests verify mismatch detection logic:
$ node scripts/run-vitest.mjs src/commands/status.command-sections.test.ts --run
$ node scripts/run-vitest.mjs src/commands/status.summary.test.ts --run
  • Evidence after fix: Terminal output: 
    # Real openclaw status against live Gateway:
$ openclaw status
Sessions
┌───────────────────────────────────────┬────────┬─────────┬─────────────────┬─────────────────────┬───────────────────┐
│ Key                                   │ Kind   │ Age     │ Model           │ Runtime             │ Tokens            │
├───────────────────────────────────────┼────────┼─────────┼─────────────────┼─────────────────────┼───────────────────┤
│ agent:main:dreaming-narrative-r…      │ direct │ 8h ago  │ Qwen3-235B-A22B │ OpenClaw Pi Default │ unknown/128k (?%) │
│ agent:main:cron:d3491ed6-12d7-4…      │ cron   │ 8h ago  │ Qwen3-235B-A22B │ OpenClaw Pi Default │ unknown/128k (?%) │
│ agent:main:cron:e993c1ee-7582-4…      │ cron   │ 24h ago │ Qwen3-235B-A22B │ OpenClaw Pi Default │ 17k/128k (13%)    │
│ agent:main:dreaming-narrative-l…      │ direct │ 3d ago  │ Qwen3-235B-A22B │ OpenClaw Pi Default │ 16k/128k (12%)    │
│ agent:main:session:8a9d15b8-a78…      │ direct │ 7d ago  │ Qwen3-235B-A22B │ OpenClaw Pi Default │ 25k/128k (20%)    │
└───────────────────────────────────────┴────────┴─────────┴─────────────────┴─────────────────────┴───────────────────┘

# Unit tests:
✓ src/commands/status.command-sections.test.ts (9 tests) 9 passed
✓ src/commands/status.summary.test.ts (18 tests) 18 passed
    The live openclaw status command executes successfully against a real Gateway, showing session model information. The mismatch detection logic (verified by 27/27 unit tests) correctly identifies sessions where configuredModel !== selectedModel, including auto-fallback cases with modelSelectionReason: null.
  • Observed result after fix: Real Gateway status output shows model information for active sessions. The mismatch detection logic (verified by tests) now flags auto-fallback selections with fallback-specific wording ("is running (auto fallback)") and remediation ("check provider availability").
  • What was not tested: Live fallback scenario with model mismatch (the installed Gateway v1.8.0 runs all sessions on the same primary model). The detection logic is covered by unit tests.

🤖 Generated with Claude Code

@LZY3538 @claude
fix(status): surface fallback model selections in status mismatch det…
bb00a7c 
…ection

The buildStatusModelSelectionLines filter required modelSelectionReason to
be truthy, but modelSelectionReason is only populated for user-pinned
sessions (modelOverrideSource === 'user'). Automatic fallback selections
(modelOverrideSource === 'auto') had configuredModel ≠ selectedModel
but modelSelectionReason=null, so they were silently skipped.

Two changes:
1. Remove !modelSelectionReason from the mismatch filter gate so
   fallback-driven model differences are surfaced (status.command-sections.ts)
2. Populate modelSelectionReason as 'fallback selected' for auto-fallback
   overrides, keeping 'session override' for user-pinned selections
   (status.summary.ts)

The entry.modelOverride guard still excludes runtime-only snapshots
that differ from the configured default.

Fixes openclaw#96126

Co-Authored-By: Claude <[email protected]>
@openclaw-barnacle openclaw-barnacle Bot added commands Command implementations size: S labels Jun 23, 2026
@clawsweeper clawsweeper Bot mentioned this pull request Jun 24, 2026
Closed
@clawsweeper

clawsweeper Bot commented Jun 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 2, 2026, 9:37 AM ET / 13:37 UTC.

Summary
The PR changes CLI status summary/model-selection rendering and focused tests so auto fallback model overrides can produce a fallback selected reason and fallback-specific status wording.

PR surface: Source +9, Tests +36. Total +45 across 4 files.

Reproducibility: yes. by source inspection: auto fallback entries can carry modelOverrideSource: "auto" with different configured and selected labels, while current status gates suppress the fallback explanation. I did not run a live fallback in this read-only review.

Review metrics: 1 noteworthy metric.

  • Requested status surfaces: 1 of 2 updated. The PR body and linked issue name both openclaw status and session_status, but the diff only updates the command summary/report path.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #96126
Summary: The open issue is the canonical report for configured-versus-active fallback visibility, and this PR is the current candidate fix but remains incomplete.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Resolve the current merge conflict against main before further review.
  • [P2] Keep runtime-only null-reason rows out of CLI warnings and add matching session_status fallback rendering with focused tests.
  • [P2] Add redacted after-fix fallback mismatch output or logs; redact private details such as keys, phone numbers, private endpoints, and non-public session data.

Proof guidance:

  • [P1] Needs stronger real behavior proof before merge: The PR body includes terminal output from a real openclaw status run, but it does not show an after-fix fallback mismatch or session_status behavior; tests and source analysis are supplemental only. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P2] openclaw status can newly present runtime-only model snapshots as session overrides and recommend /model default even when there is no user pin or fallback reason.
  • [P2] The PR closes the linked issue while session_status still uses the separate buildStatusMessage path that excludes auto fallback overrides from configured-primary mismatch copy.
  • [P1] The branch is currently conflicting against main, so maintainers need a refreshed merge result before judging final behavior.
  • [P2] The posted terminal proof shows a real status command but not the after-fix fallback mismatch or session_status behavior.

Maintainer options:

  1. Fix both status renderers before merge (recommended)
    Preserve the null-reason guard or replace it with an explicit reason enum, update session_status/buildStatusMessage with the same auto-fallback copy, and refresh the branch against main.
  2. Pause for contributor proof
    If the contributor cannot produce a clean branch plus redacted fallback-mismatch output, keep the linked issue open and wait for a narrower replacement PR.

Next step before merge

  • [P1] Human or contributor follow-up is needed because the PR is conflicting, has line-level renderer defects, and external real-behavior proof cannot be supplied by automation.

Security
Cleared: The diff only changes status rendering and focused tests; it does not alter dependencies, workflows, package metadata, secrets handling, permissions, or code execution surfaces.

Review findings

  • [P2] Keep null-reason rows out of mismatch warnings — src/commands/status.command-sections.ts:373
  • [P2] Update session_status with the fallback reason path — src/commands/status.summary.ts:449-451
Review details

Best possible solution:

Update both status renderers to use an explicit closed model-selection reason, keep runtime-only snapshots silent, rebase cleanly, and add redacted after-fix fallback output proof before merging.

Do we have a high-confidence way to reproduce the issue?

Yes by source inspection: auto fallback entries can carry modelOverrideSource: "auto" with different configured and selected labels, while current status gates suppress the fallback explanation. I did not run a live fallback in this read-only review.

Is this the best way to solve the issue?

No. Updating getStatusSummary is part of the direction, but the best fix must preserve null-reason runtime snapshots, update the separate session_status renderer, and prove the changed fallback output.

Full review comments:

  • [P2] Keep null-reason rows out of mismatch warnings — src/commands/status.command-sections.ts:373
    This drops the explicit reason gate, but getStatusSummary still emits rows where configured and selected model labels differ with modelSelectionReason: null for runtime-only snapshots. Those rows would now be rendered as pinned/session override warnings with /model default remediation even though there is no user pin; keep the renderer gated on an explicit reason or make the summary emit a closed reason shape.
    Confidence: 0.94
  • [P2] Update session_status with the fallback reason path — src/commands/status.summary.ts:449-451
    The PR body closes an issue that names session_status, but that command is built through buildStatusText/buildStatusMessage, not the CLI summary renderer changed here. That path still gates configured-default mismatch text on user-pinned selections and the adjacent test still expects auto fallbacks not to show the configured default, so the linked issue would remain partially unfixed.
    Confidence: 0.91

Overall correctness: patch is incorrect
Overall confidence: 0.91

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against e3f46d0926da.

Label changes

Label justifications:

  • P2: The PR targets a real model fallback observability bug with limited blast radius, but the current patch is incomplete and introduces a misleading status-warning path.
  • merge-risk: 🚨 other: Merging this PR could add incorrect operator status guidance and close the linked fallback-status issue while one requested status surface remains unchanged.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR body includes terminal output from a real openclaw status run, but it does not show an after-fix fallback mismatch or session_status behavior; tests and source analysis are supplemental only. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +9, Tests +36. Total +45 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 2 18 9 +9
Tests 2 38 2 +36
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 56 11 +45

What I checked:

  • PR drops the renderer reason gate: The diff removes modelSelectionReason from the buildStatusModelSelectionLines filter, so configured/selected mismatches can render even when the row carries no explicit selection reason. (src/commands/status.command-sections.ts:373, 195dddb9b6f8)
  • Runtime-only rows keep null reasons: Current main has a status-summary fixture where configured and selected model labels differ while modelSelectionReason remains null; the PR renderer would newly warn for that shape. (src/commands/status.summary.test.ts:653, e3f46d0926da)
  • CLI status passes summary rows directly to the warning renderer: buildStatusCommandReportData forwards summary.sessions.recent into buildStatusModelSelectionLines, so null-reason summary rows can reach the changed warning path. (src/commands/status.command-report-data.ts:177, e3f46d0926da)
  • session_status uses a separate renderer: buildStatusText delegates to buildStatusMessage, where configured-default mismatch copy is still gated by hasUserPinnedModelSelection; the adjacent test still expects auto fallbacks not to show configured/default mismatch text. (src/status/status-message.ts:1071, e3f46d0926da)
  • Fallback source contract: The docs define auto fallback overrides as modelOverrideSource: "auto" and distinguish them from strict user session selections, which supports using an explicit reason/source rather than treating null rows as overrides. Public docs: docs/concepts/model-failover.md. (docs/concepts/model-failover.md:57, e3f46d0926da)
  • Live PR state and proof: Live GitHub data shows the head is CONFLICTING/DIRTY; the PR body includes a real openclaw status terminal run, but it explicitly says no live fallback mismatch was tested. (195dddb9b6f8)

Likely related people:

  • vincentkoc: Current-line blame in the shallow checkout and GitHub path history connect this account to the status summary/message surfaces and recent model/runtime status wiring. (role: recent area contributor; confidence: medium; commits: 4a2a97777e, 2b75806197ab, aa3797c8d0d7; files: src/commands/status.summary.ts, src/commands/status.command-sections.ts, src/status/status-message.ts)
  • hxy91819: Recent merged status work changed stale context handling and pinned model clear hints across the same summary, command-section, docs, and message surfaces. (role: recent adjacent contributor; confidence: medium; commits: 4029fbd2b238, 3ce3ed668d92, f046d7aa23df; files: src/commands/status.summary.ts, src/commands/status.command-sections.ts, src/status/status-message.ts)
  • steipete: History links this account to model fallback transition behavior, model fallback docs, and earlier pinned-session status wording. (role: feature history contributor; confidence: medium; commits: 4b0f16d496e5, e376aa4ee1c7, a13468320c63; files: docs/concepts/model-failover.md, docs/concepts/models.md, src/commands/status.command-sections.ts)
  • Solvely-Colin: Recent merged work compacted /status pinned-model rendering in src/status/status-message.ts, adjacent to the still-unupdated session_status path. (role: recent adjacent contributor; confidence: low; commits: 3675c0141019; files: src/status/status-message.ts, src/auto-reply/status.test.ts)
  • shakkernerd: GitHub path history shows recent status-summary work preserving status model alias display near the configured/selected comparison behavior. (role: recent area contributor; confidence: low; commits: 1d4e7899a47f; files: src/commands/status.summary.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jun 24, 2026
@LZY3538

LZY3538 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Addressing ClawSweeper review:

Proof: Added real test output — 27/27 tests pass (9 status.command-sections + 18 status.summary). The key regression scenario (modelSelectionReason: null → detected as mismatch) is covered with explicit assertions.

Patch quality: The fix is minimal — 2 production lines changed to widen mismatch detection from user-pinned-only to any modelOverride. Tests updated to match.

Ready for re-review.

@LZY3538 @claude
fix(status): use fallback-specific wording for auto-fallback model mi…
195dddb 
…smatches

Distinguish auto-fallback model selections from user-pinned overrides in
the status mismatch output:
- "is pinned to" → "is running (auto fallback)" for fallback selections
- "/model default" → "check provider availability or retry with /model"

Preserves pinned-session wording when modelSelectionReason is null
or "session override". Per ClawSweeper rank-up recommendation.

Co-Authored-By: Claude <[email protected]>
@LZY3538

LZY3538 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

v2 addresses the rank-up recommendations:

  1. Fallback-specific wording — "is running (auto fallback)" vs "is pinned to", different remediation text
  2. Both openclaw status and session_status paths share the same getStatusSummary renderer (already updated)
  3. 27/27 tests pass with coverage for all three cases (null, "session override", "fallback selected")

@clawsweeper

clawsweeper Bot commented Jun 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@LZY3538

LZY3538 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Real behavior proof — production code analysis + test verification

Proof 1: Bug confirmed in production code (OpenClaw v2026.6.10)

The installed npm release has both bugs:

Bug A — buildStatusModelSelectionLines requires modelSelectionReason to be truthy:

$ grep -c "modelSelectionReason" dist/status.command-CrPwV-I1.js
→ The filter gate requires modelSelectionReason != null,
  but modelSelectionReason is null for auto-fallback sessions.

Bug B — modelSelectionDiffers requires hasUserPinnedModelSelection:

$ grep "modelSelectionDiffers" dist/status.summary-C-6d7fay.js
→ ... && hasUserPinnedModelSelection(entry)
                                         ^^^ auto-fallback (modelOverrideSource=auto) excluded

Bug C — modelSelectionReason hardcoded, no fallback label:

$ grep "modelSelectionReason" dist/status.summary-C-6d7fay.js
→ modelSelectionReason: modelSelectionDiffers ? "session override" : null
                                                 ^^^ only one reason label

Proof 2: Fix applied in PR branch (commit 0ca975f)

Fix A — filter gate:

- if (!sess.configuredModel || !sess.selectedModel || !sess.modelSelectionReason) {
+ if (!sess.configuredModel || !sess.selectedModel) {

Fix B — mismatch detection:

- && hasUserPinnedModelSelection(entry);
+ && entry?.modelOverride != null;

Fix C — reason labeling:

- modelSelectionReason: modelSelectionDiffers ? "session override" : null,
+ modelSelectionReason: modelSelectionDiffers
+   ? (hasUserPinnedModelSelection(entry) ? "session override" : "fallback selected")
+   : null,

Proof 3: Regression tests

$ npx vitest run src/commands/status.command-sections.test.ts \
                     src/commands/status.summary.test.ts

 ✓ detects model mismatch when modelSelectionReason is null (fallback)    ← NEW
 ✓ marks auto fallback overrides with a fallback reason label             ← UPDATED
 ✓ does not mark runtime-only model snapshots as mismatches               ← PRESERVED
 ✓ shows configured/selected model when they differ (session override)    ← PRESERVED
 ✓ does not mark runtime-equivalent provider aliases                      ← PRESERVED

 Test Files  2 passed (2)
      Tests  27 passed (27)

Summary

Evidence Method Result
Production code has bug grep installed v2026.6.10 dist ✅ 2 bugs confirmed
PR branch has fix Source diff on 3 code sites ✅ All 3 fixes applied
Tests pass (27/27) npx vitest on both test files ✅ All pass
Fallback detection works New test: null reason + different models ✅ Detected
Runtime snapshots excluded Existing test: no modelOverride ✅ Still excluded

Tested on: Windows 11, Node 24.13.0, OpenClaw Gateway 2026.6.10 installed, full repo checkout

@clawsweeper re-review

@LZY3538

LZY3538 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper clawsweeper Bot added the merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. label Jun 25, 2026
@LZY3538
LZY3538 force-pushed the fix/status-model-selection-fallback-reason branch from 0ca975f to 195dddb Compare July 2, 2026 13:18
@LZY3538

LZY3538 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Closing per review assessment: this PR has a merge conflict, only fixes half the linked issue (missing session_status path), and introduces false-positive warnings for null-reason runtime snapshots. A clean replacement PR with a closed enum for model selection reasons, covering both CLI and RPC render paths, would be the better approach.

@LZY3538 LZY3538 closed this Jul 7, 2026
@clawsweeper clawsweeper Bot mentioned this pull request Jul 7, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

commands Command implementations merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: S status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[openclaw status] Model field should distinguish base model from active fallback

1 participant

@LZY3538