NanoGPT cache-write usage and billed cost are lost in session accounting #48478

Activity

1. github-actions

   assigned

   fwang

   on Sep 11, 2026
2. github-actions commented on Sep 11, 2026

   

   github-actionsBot

   on Sep 11, 2026 – with GitHub Actions

   Contributor

   More actions

   This issue might be a duplicate of existing issues. Please check:

   • OpenAI cache writes always reported as 0 #37745: OpenAI cache writes always reported as 0 (same symptom of cache-write tokens lost via the OpenAI-compatible SDK)
   • DeepSeek provider: prompt_cache_miss_tokens not mapped to cache.write #35974: DeepSeek provider: prompt_cache_miss_tokens not mapped to cache.write (same root pattern of cache write tokens not being mapped)
   • OpenAI-compatible providers report zero tokens_cache_read despite upstream returning cached_tokens #34022: OpenAI-compatible providers report zero tokens_cache_read despite upstream returning cached_tokens (related usage mapping issue for OpenAI-compatible providers)

   The NanoGPT-specific x_nanogpt_pricing cost override aspect may be unique to this report, but the cache-write token loss overlaps with the above.
3. huggix commented on Sep 14, 2026

   

   huggix

   on Sep 14, 2026

   Author

   More actions

   Thanks—checked all three.

   #37745 overlaps with the missing cache-write count. #35974 concerns DeepSeek’s prompt_cache_miss_tokens, whereas this report concerns NanoGPT’s explicit usage.cache_creation_input_tokens. #34022 concerns missing cache reads; our reproduction preserves reads but loses writes.

   The additional issue here is that OpenCode ignores NanoGPT’s settled USD charge in x_nanogpt_pricing and recomputes an estimate from catalog rates. Fixing cache-write mapping alone would leave that discrepancy unresolved.

   #48477 includes a proposed fix and regression tests for both aspects, including streaming. Happy to consolidate the cache-write portion if preferred, while retaining coverage of the reported-cost handling.
4. opencodejr commented on Oct 28, 2026

   

   opencodejrBot

   on Oct 28, 2026

   Contributor

   More actions

   Thanks for the detailed report. I can reproduce this on the latest dev (1e7ba46).

   I replayed your usage payload as a streamed chat-completions response through OpenCode's OpenAI-compatible protocol (packages/ai), with the provider set to nano-gpt. The parsed usage came back as:

   • input 1000, cache read 100: correct
   • cache write: missing, expected 800
   • non-cached input: 900, expected 100, so the cache-write tokens are counted as ordinary input
   • cost: missing, so session accounting falls back to the catalog estimate instead of the reported $0.0123

   The cause is in mapUsage in packages/ai/src/protocols/openai-chat.ts. It reads cache writes only from prompt_tokens_details.cache_write_tokens and cache_created_input_tokens, never from the top-level cache_creation_input_tokens. Nothing reads x_nanogpt_pricing.

   The test below reproduces it from packages/ai/test/. I only tested the streaming path. A related gap is the TODO for #35765 in packages/core/src/session/usage.ts, which is about using Copilot's reported billed amount in the same way.

   Reproduction script (nanogpt-repro.test.ts)

   // Reproduction for anomalyco/opencode#48478: NanoGPT usage through the OpenAI-compatible chat protocol.
   import { describe, expect } from "bun:test"
   import { Effect } from "effect"
   import { LLM } from "../src/effect.js"
   import { OpenAICompatible } from "../src/providers/openai-compatible.js"
   import { LLMClient } from "../src/route.js"
   import { it } from "./lib/effect.js"
   import { fixedResponse } from "./lib/http.js"
   import { sseEvents } from "./lib/sse.js"
   
   const model = OpenAICompatible.configure({ baseURL: "https://nano-gpt.com/api/v1", apiKey: "k", provider: "nano-gpt" }).model(
     "claude-sonnet-4-6",
   )
   
   describe("NanoGPT usage (#48478)", () => {
     it.effect("keeps cache writes and the billed cost", () =>
       Effect.gen(function* () {
         const response = yield* LLMClient.generate(LLM.request({ model, prompt: "Say OK" })).pipe(
           Effect.provide(
             fixedResponse(
               sseEvents(
                 { id: "c", object: "chat.completion.chunk", created: 1, model: "m",
                   choices: [{ index: 0, delta: { content: "OK" }, finish_reason: null }] },
                 { id: "c", object: "chat.completion.chunk", created: 1, model: "m",
                   choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
                   usage: { prompt_tokens: 1000, completion_tokens: 20, total_tokens: 1020,
                     cache_creation_input_tokens: 800, prompt_tokens_details: { cached_tokens: 100 } },
                   x_nanogpt_pricing: { amount: 0.0123, currency: "USD" } },
                 "[DONE]",
               ),
             ),
           ),
         )
         const u = response.usage!
         console.log("USAGE", JSON.stringify({ input: u.inputTokens, nonCached: u.nonCachedInputTokens,
           cacheRead: u.cacheReadInputTokens, cacheWrite: u.cacheWriteInputTokens, cost: u.cost }))
         expect(u.cacheReadInputTokens).toBe(100)
         expect(u.cacheWriteInputTokens).toBe(800)
         expect(u.nonCachedInputTokens).toBe(100)
         expect(u.cost).toBe(0.0123)
       }),
     )
   })

   Run it: bun nanogpt-repro.test.ts
5. opencodejr

   added

   reproduced

   on Oct 28, 2026