-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
Model Aliases in system prompt cause LLM to substitute providers unexpectedly #61949
Copy link
Copy link
Open
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.Auth, provider routing, model choice, or SecretRef resolution may break.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
Environment
Summary
When multiple providers are configured for equivalent models (e.g.,
github-copilot/claude-sonnet-4.6andamazon-bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0), the Model Aliases section injected into the system prompt causes the LLM to treat them as interchangeable. This leads to unexpected provider substitution.Observed Behavior
github-copilot/claude-sonnet-4.6Model: "github-copilot/claude-sonnet-4.6"sessions_spawn, it substitutesamazon-bedrock/...instead of passing through the requestedgithub-copilot/...Why This Is a Problem
Different providers for the "same" model are not equivalent:
In our case, sub-agents were crashing immediately because Bedrock Sonnet 4 with thinking enabled requires assistant messages to start with thinking blocks — a constraint the LLM violated after receiving tool results. The crash happened silently with no advance/retreat, causing infinite dispatch loops.
Root Cause
The "Model Aliases" section in the system prompt teaches the LLM that different provider paths are interchangeable:
This invites the LLM to "helpfully" translate a specific provider/model request to whatever alias it recognizes, even when an explicit provider is specified.
Suggested Fix
Options (not mutually exclusive):
provider/modelstring is given, the LLM should pass it through verbatim, not interpret itWorkaround
We're removing Bedrock from our config entirely to prevent this. The aliases will no longer be injected, and the LLM won't have alternative providers to choose from.
Config That Triggered This
{ "agents": { "defaults": { "model": { "primary": "github-copilot/claude-opus-4.5", "fallbacks": ["amazon-bedrock/us.anthropic.claude-opus-4-5-20251101-v1:0"] }, "models": { "amazon-bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0": {"alias": "Bedrock Sonnet 4"}, ... } } }, "models": { "providers": { "amazon-bedrock": { ... }, "github-copilot": { ... } } } }