Repository navigation
Conversation
|
Heads up on the overlap with #36861. Both PRs touch the same cache-write fallback, but they fix different paths, and #36861 does not cover the case in #36749. #36861 reads the counts out of The count does survive, just not there: I checked this rather than assumed it. Applying #36861 on its own and running the gateway scenario from #36749 against it ( So the two look complementary rather than duplicate. #36861 covers providers that expose cache counts through metadata and also adds cache read; this one covers the openai-compatible SDK path, where metadata has nothing to offer. They do collide textually, so whichever lands first, the other needs a rebase. Happy to rebase on top of #36861 if that one goes first. |
|
@ZYRAXON-AI help |
|
Automated PR Cleanup Thank you for contributing to opencode. Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions. This PR was closed because it matched the following cleanup criteria:
PRs created within the last month are not affected by this cleanup. If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate. Thanks again for taking the time to contribute. |
Issue for this PR
Closes #36749
Type of change
What does this PR do?
Anthropic models served through an OpenAI-compatible gateway always recorded
cache.write: 0, so cache writes were billed as plain input (1.0x instead of 1.25x) and cost came out understated.The OpenAI usage shape has no cache-write field, so
@ai-sdk/openai-compatiblesetscacheWrite: void 0and hands the provider payload through asLanguageModelUsage.raw. Gateways fronting Anthropic do report the count there (cache_creation_input_tokens), butusage()only readinputTokenDetails.cacheWriteTokens, and the fallbacks inSession.getUsageonly cover nativeanthropic/vertex/bedrockmetadata, so the value ended at 0.This reads the count from
rawwhen the SDK has nothing to offer. Native providers are unaffected: the normalized field still wins.Session.getUsagealready derives non-cached input asinputTokens - cacheRead - cacheWrite, so the breakdown lines up once the write count is populated.#18440 covered the same symptom and was closed by #22224, which made
getUsagereadusage.cacheWriteInputTokens. That fixed providers that populate the field, but@ai-sdk/openai-compatiblenever does, so gateways on that path stayed broken.How did you verify your code works?
Ran a real session against a gateway fronting Bedrock Anthropic, first turn of the session, and read the tokens back from the message store.
Before:
{"total": 19428, "input": 19428, "output": 4, "cache": {"write": 0, "read": 0}}After:
{"total": 19432, "input": 2, "output": 4, "cache": {"write": 19426, "read": 0}}which matches what the gateway reported for that request (
prompt_tokens: 19428,cache_creation_input_tokens: 19426).Also ran
bun typecheckandbun test test/sessionfrompackages/opencode: typecheck clean, 368 pass / 0 fail.To confirm it yourself, point a provider with
npm: "@ai-sdk/openai-compatible"at any gateway fronting an Anthropic model with caching on, send a prompt long enough to be cached, and checkcache.writeon the assistant message.Screenshots / recordings
Not a UI change.
Checklist