Repository navigation
Conversation
Provider.getLanguage() is typed as returning LanguageModelV3, but a dynamically loaded provider can return a runtime V2 model. Routing every model through wrapLanguageModel() unconditionally labels the result "v3" (ai/src/middleware/wrap-language-model.ts), which skips AI SDK's own V2->V3 compat conversion (asLanguageModelV3 inside resolveLanguageModel) and silently drops finish-step usage. Because session/overflow.ts relies on that usage to detect context pressure, affected providers never trigger automatic compaction. Route V2 models around the wrapper instead: apply the same message transform directly to the ModelMessage[] input and pass the raw model to streamText, so AI SDK's built-in compat layer still runs. Fixes anomalyco#50338 Co-Authored-By: Claude Sonnet 5 <[email protected]>
Contributor
|
The following comment was made by an LLM, it may be inaccurate: |
Contributor
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
3 of 6 tasks
Member
|
This branch has been upgraded to V2. We're no longer taking PRs for V1. If you think this is still relevant, please port it over and open it for V2. — from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #50338
Type of change
What does this PR do?
Provider.getLanguage()is typed as returningLanguageModelV3, but a dynamically loaded provider can hand back a runtimeLanguageModelV2instead.session/llm.tsalways passed that model throughwrapLanguageModel(). Looking atwrapLanguageModel's implementation in theaipackage, the object it returns is hardcoded tospecificationVersion: "v3"and calls the wrapped model'sdoGenerate/doStreamdirectly — it doesn't itself do any V2→V3 conversion. AI SDK has its own compatibility shim for this (asLanguageModelV3, used insideresolveLanguageModel), but that shim only runs whenstreamTextsees a model whosespecificationVersionis still"v2". SincewrapLanguageModelalready relabels it"v3"beforestreamTextever looks at it, the shim never fires and the V2 usage numbers (a different shape than V3) never get converted — they come backundefined.That matters beyond the usage indicator:
session/overflow.tsuses that usage to decide when to auto-compact, so with itundefinedcompaction never triggers and long sessions on these providers just keep growing until the provider hard-rejects the request.Fix: check the runtime
specificationVersionbefore wrapping. V3 models go throughwrapLanguageModel()exactly as before. V2 models skip the wrapper — the message transform that used to run insidetransformParamsgets applied directly to theModelMessage[]input instead, and the raw V2 model is passed straight tostreamText, so AI SDK's own compat layer does the conversion the way it's meant to.How did you verify your code works?
wrapLanguageModel/streamTextdirectly (not opencode code) with a hand-built V2 model to confirm the mechanism: wrapping it drops usage, passing it unwrapped preserves it. This matches the reporter's own differential probe in the issue.test/session/llm.test.tswith a small fixture provider (test/session/fixtures/fake-v2-provider.ts) that implementsLanguageModelV2and is loaded vianpm: "file://...", the same mechanism any dynamically loaded provider uses. It goes through the realProvider.getLanguage()→LLM.Service.stream()path. Checked it fails on the pre-fix code (usageundefined) and passes with the fix.bun run typecheckinpackages/opencode— clean.bun test test/session/llm.test.ts— 31/31 pass.bun test test/session/(full directory) — 413 pass, 0 fail, no regressions.Screenshots / recordings
Not a UI change.
Checklist