Skip to content

fix(session): preserve reported usage for AI SDK V2 providers - #50540

Closed
Veld101 wants to merge 1 commit into
anomalyco:devfrom
Veld101:fix-v2-provider-usage
Closed

Veld101 wants to merge 1 commit into
anomalyco:devfrom
Veld101:fix-v2-provider-usage

Conversation

@Veld101

@Veld101 Veld101 commented Sep 22, 2026

Copy link
Copy Markdown

Issue for this PR

Closes #50338

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Dynamically loaded AI SDK V2 providers persisted zero token usage, which silently disabled automatic compaction.

session/llm.ts passed every resolved model through wrapLanguageModel(...) with a specificationVersion: "v3" middleware. That helper does not convert a V2 model; it returns a new object that hardcodes specificationVersion: "v3" and forwards doStream unchanged. streamText then calls resolveLanguageModel, sees "v3", returns the model as-is, and skips the AI SDK's asLanguageModelV3 compatibility proxy — which is what performs convertV2StreamToV3 on the finish chunk. Usage (and finishReason) never reached the session.

The middleware was only doing one thing: rewriting params.prompt with ProviderTransform.message(...). That transform is a plain ModelMessage[] -> ModelMessage[] function, and streamText already accepts the same input shape through its messages option, so the transform can run at the call site and the model can be passed unwrapped. ProviderTransform.message handles both string and part-array content, so it does not depend on AI SDK's earlier normalization step.

I confirmed the failure mode and the fix with a standalone probe against [email protected]:

bare V2 model        finish-step usage = { inputTokens: 14, ..., outputTokens: 4, totalTokens: 18 }   finishReason "stop"
V2 + v3 middleware   finish-step usage = { inputTokenDetails: {}, outputTokenDetails: {} }            finishReason undefined
bare V3 model        finish-step usage = { inputTokens: 21, ..., outputTokens: 7, totalTokens: 28 }   finishReason "stop"
V3 + v3 middleware   finish-step usage = { inputTokens: 21, ..., outputTokens: 7, totalTokens: 28 }   finishReason "stop"

V3 is unaffected by the wrapper, so unwrapping is safe for V3 and necessary for V2. finishReason was lost alongside usage, which also broke finish === "content-filter" handling and any other finish-reason branch for these providers.

How did you verify your code works?

  • bun test test/session/llm.test.ts — 30 pass, 0 fail. This file asserts the outgoing payload for Anthropic-compatible, Gemini, and OpenAI Responses models, so it covers the moved ProviderTransform.message call for the providers it exercises.
  • bun test test/session/ — 397 pass / 6 fail, identical to the base commit with this change stashed. The 6 failures are pre-existing 5s timeouts on Windows (stops quickly when aborted during retry backoff, glob tool keeps instance context during prompt runs, two revert + compact workflow cases, tool execution produces non-empty session diff, stop after token overflow requests compaction) and are unrelated.
  • bun typecheck — clean.
  • prettier --write applied to the changed file.

I did not add an automated regression test. Every provider fixture in the test suite resolves to a V3 SDK, so a test there passes with or without this fix — I verified that by stashing the change and watching the test still pass. Exercising the bug requires injecting a real LanguageModelV2, which needs a custom provider loader the harness does not have yet. I would rather leave the suite honest than add a test that cannot fail.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actions

Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

Potential duplicate found:

This appears to be addressing the exact same issue (dynamically loaded AI SDK V2 providers not preserving token usage). The titles and scopes are nearly identical. You may want to check if PR #50414 is already merged or if it covers the same fix.

@Veld101

Veld101 commented Sep 28, 2026

Copy link
Copy Markdown
Author

Superseded by #50797 on �2. dev is not where merges land (recent merges are overwhelmingly into v2), so this fix is carried in the v2 PR instead. Closing to keep the queue focused; reopen or cherry-pick from that branch if needed.

@Veld101 Veld101 closed this Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: AI SDK V2 providers silently lose usage and disable auto-compaction

1 participant