Skip to content

Add Copilot auto routing via /models/session + /models/session/intent - #1

Merged
m0wer merged 1 commit into
m0wer:masterfrom
avishefi:feat/session-intent-routing
Jun 12, 2026
Merged

m0wer merged 1 commit into
m0wer:masterfrom
avishefi:feat/session-intent-routing

Conversation

@avishefi

Copy link
Copy Markdown
Contributor

What

Makes github-copilot/auto follow Copilot's own auto selection — including the per-turn intent router that VS Code uses — rather than a static availability pick.

The flow (reverse-engineered from microsoft/vscode-copilot-chat + @vscode/copilot-api, verified against the lib's _mixinHeaders):

  1. POST /models/session — per-conversation: returns the candidate pool + a Copilot-Session-Token (attaching it to chat requests is what puts them in the auto billing/rate-limit pool + discount).
  2. POST /models/session/intent — the router: given the prompt, returns ranked candidate_models (reasoning vs. fast). The only call that sees the prompt.

Wired into opencode hooks: provider.models (inject Auto), chat.message (capture prompt), chat.params (route + override model id), chat.headers (attach session token), event (re-eval on compaction).

Routing behavior (mirrors VS Code)

  • Cache-stable cadence: route on turn 0, stick for the rest of the conversation (keeps the prompt KV cache warm), re-evaluate only after session.compacted — matching VS Code's automode router (turnCount > 0 skip + invalidateRouterCache).
  • Within-family: the cloned auto model has a single fixed endpoint (npm + url), so routing stays in the anchored family (a reasoning tier vs. a fast tier). A guard anchors on a family with ≥2 tiers so the router always has a choice. (VS Code re-instantiates the endpoint per turn and can cross families; opencode can only rewrite the model id, hence the constraint.)
  • Default selection mirrors VS Code's _selectDefaultModel — first same-family model in available_models — instead of the undocumented selected_model.
  • Reasoning effort: needs_reasoning raises the routed model's effort (reasoning_effort: high for OpenAI/Copilot, an extended-thinking budget for Anthropic); fast turns stay at the model default.

UI

  • Picker/status bar read Auto; a toast surfaces each (re)route (e.g. Auto → gpt-5.4 · needs_reasoning).

Headers / gating

CAPI headers verified against @vscode/copilot-api's _mixinHeaders. The Copilot-Integration-Id / Editor-Version / Editor-Plugin-Version lines are left commented so the minimal header set can be tested first; the honest non-impersonating floor is Copilot-Integration-Id: code-oss (what a plain OSS build sends). Note the intent router is a Team-Internal/Panel-only experiment in shipped VS Code, so this tracks internal VS Code behavior.

Tests

bun test (7 pass), bun run typecheck, bun run build all green.

Adds a github-copilot/auto model that follows Copilot's own auto selection:
opens a per-conversation /models/session (token + pool), calls the
/models/session/intent router per VS Code's cadence (route on turn 0, stick
after, re-eval on compaction), and attaches Copilot-Session-Token to chat
requests for the auto billing/rate-limit pool.

- Within-family routing: the cloned auto model has one fixed endpoint, so
  routing stays in the anchored family (reasoning-vs-fast tier), with a guard
  ensuring the family has >=2 tiers to choose between.
- Default selection mirrors VS Code's _selectDefaultModel (first same-family
  model in available_models), not the undocumented selected_model.
- Reasoning-effort mapping: needs_reasoning raises the routed model's effort
  (reasoning_effort:high for OpenAI/Copilot, extended-thinking budget for
  Anthropic); fast turns left at default.
- Picker label 'Auto'; a toast surfaces each (re)route.
- CAPI headers verified against @vscode/copilot-api _mixinHeaders; the
  integration-id/editor-version lines are left commented (code-oss is the
  honest floor) to test the minimal set first.
@avishefi
avishefi force-pushed the feat/session-intent-routing branch from b7c47e0 to 1fd442a Compare June 12, 2026 07:15
@m0wer
m0wer merged commit 1fd442a into m0wer:master Jun 12, 2026
@m0wer

m0wer commented Jun 12, 2026

Copy link
Copy Markdown
Owner

Thanks! Added some other improvements afterwards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants