Skip to content

feat: expose resolved backend model in session_status and agent runtime #51441

Description

@Kyzcreig

Problem

When using LiteLLM (or any routing proxy) as the LLM provider, agents only see the requested model alias (e.g. litellm/complex) — not the actual backend model that was used (e.g. openai/gpt-5.4 or anthropic/claude-sonnet-4-6).

This creates a blind spot: agents can't answer "did a fallback happen?" or "what model actually served this response?" without log-diving.

Current behavior

session_status returns the configured/requested model. The agentMeta.model in runResult.meta is populated from CLI runners (claude-cli, codex, etc.) but not from the OpenAI-compatible path — so when LiteLLM routing is used, agentMeta.model falls through to fallbackModel ?? defaultModel (the configured alias, not the resolved backend).

persistSessionUsageUpdate stores modelUsed and providerUsed from agentMeta, so this gap flows through to the session store.

Requested feature

  1. Capture response.model from OpenAI-compatible responses in the provider transport layer and expose it as agentMeta.resolvedModel / agentMeta.resolvedProvider

  2. Add resolvedModel, resolvedProvider, fallbackUsed fields to GatewaySessionRow and persist them via persistSessionUsageUpdate

  3. Expose in session_status tool output:

    model: litellm/complex (requested)
    resolvedModel: openai/gpt-5.4 (actual)
    fallbackUsed: true
    
  4. Include in agent runtime text injection so agents can introspect routing decisions without tool calls

  5. Optional: pass session/agent metadata as LiteLLM request tags so /spend/logs entries can be correlated to specific agent sessions

Workaround in use

We currently query http://localhost:4000/spend/logs directly (LiteLLM's spend log API) to surface resolved model info. The fields model_group (= requested tier) and model (= actual backend) are both present there. This works but requires a separate API call and is not surfaced to the agent in-band.

Files involved

  • dist/agents/steerable-provider-transport.js — capture response.model from API responses
  • dist/auto-reply/reply/session-usage.js + .d.ts — add new fields to persistSessionUsageUpdate
  • dist/auto-reply/reply/agent-runner.js + followup-runner.js — pass resolved fields to usage update
  • dist/gateway/session-utils.types.d.ts + session store — add resolvedModel, resolvedProvider, fallbackUsed to GatewaySessionRow
  • dist/agents/tools/session-status-tool.js — expose in tool output
  • dist/commands/status.summary.js — expose in CLI status output

Impact

High — any deployment using LiteLLM, OpenRouter, or any routing proxy is blind to actual model routing. Debugging fallbacks requires log access rather than in-band observability.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.maturity:stableIssue affects a taxonomy feature currently scored M4/M5.staleMarked as stale due to inactivity

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions