Skip to content

[Bug]: gemini *-latest aliases price a cache read at the deprecated 2.0 rate (25% of input, should be 10%) #38062

Closed

Description

@xyzs996

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

Four entries in model_prices_and_context_window.json still bill a Gemini cache read at 25% of input, which is the Gemini 2.0 rate. Google shut 2.0 Flash down on 2026-06-01; every 2.5 and 3.x model is 10%.

The two *-latest aliases are the clearest case, because the same alias is already in the file twice under the gemini provider with identical input and output prices and different cache-read prices:

line key provider cache_read_input_token_cost as % of input
21731 gemini/gemini-flash-latest gemini 7.5e-08 25%
48220 gemini-flash-latest gemini 3e-08 10%
21777 gemini/gemini-flash-lite-latest gemini 2.5e-08 25%
48266 gemini-flash-lite-latest gemini 1e-08 10%

Both members of each pair carry the same input_cost_per_token and output_cost_per_token, so this is not two different products — it is one product with two costs, and which one you get depends on whether you wrote the model as gemini-flash-latest or gemini/gemini-flash-latest.

Google's pricing page settles which is right:

  • Gemini 2.5 Flash — input $0.30, context caching $0.03 → 10%
  • Gemini 2.5 Flash-Lite — input $0.10, context caching $0.01 → 10%
  • Gemini 2.0 Flash (deprecated, shut down 2026-06-01) — input $0.10, context caching $0.025 → 25%

The -latest entries carry input_cost_per_token of 3e-07 and 1e-07, i.e. the 2.5 Flash and 2.5 Flash-Lite rates, so 10% is the matching figure. The 7.5e-08 / 2.5e-08 values are the 2.0-era ratio left behind.

The file agrees with itself on this everywhere else. Across all 32 gemini-provider entries that have both an input price and a cache-read price, the ratio is 10% on every 2.5/3.x model; 25% appears only on the four Gemini 2.0 entries and on the stragglers listed here. The vertex_ai side is identical.

Two more entries look like the same miss, though I have not been able to verify what these previews were priced at while they were previews, so please treat them as lower confidence than the -latest pair:

line key provider value sibling at 10%
21683 gemini/gemini-2.5-flash-preview-09-2025 gemini 7.5e-08 gemini/gemini-2.5-flash = 3e-08, same in/out
19967 gemini-2.5-flash-preview-09-2025 vertex_ai-language-models 7.5e-08 gemini-2.5-flash = 3e-08, same in/out
21824 gemini/gemini-2.5-flash-lite-preview-06-17 gemini 2.5e-08 gemini/gemini-2.5-flash-lite = 1e-08, same in/out
20103 gemini-2.5-flash-lite-preview-06-17 vertex_ai-language-models 2.5e-08 gemini-2.5-flash-lite = 1e-08, same in/out

This is the same bug class as #19488, which fixed gemini-2.5-flash-lite and gemini-2.5-flash-lite-preview-09-2025 in January. Those two are correct today. The -latest aliases and the two previews above were not part of that pass.

Impact

cache_read_input_token_cost feeds completion_cost, so this is reported spend, not just routing. Mix-free version: 1M cache-read tokens on gemini/gemini-flash-latest are reported at $0.075; Google bills $0.03.

That gap is small in absolute terms and large in relative terms, and it lands hardest exactly where caching is used most. For a workload that is 95% cache reads, reported spend on that model comes out roughly 1.9x actual.

Suggested fix

Four values, no logic change:

line 21731  gemini/gemini-flash-latest                    7.5e-08  ->  3e-08
line 21777  gemini/gemini-flash-lite-latest               2.5e-08  ->  1e-08
line 21683  gemini/gemini-2.5-flash-preview-09-2025       7.5e-08  ->  3e-08   (see note above)
line 21824  gemini/gemini-2.5-flash-lite-preview-06-17    2.5e-08  ->  1e-08   (see note above)

plus lines 19967 and 20103 on the vertex_ai side if Vertex's own page agrees with the Gemini API page for those two previews — I checked the Gemini API page, not the Vertex one.

I have not opened a PR because I am not able to sign the CLA. Every figure above is from main as of today; line numbers are from the same fetch.

Relevant log output

$ python3 - <<'EOF'
import json, urllib.request
d = json.load(urllib.request.urlopen(
    "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"))
for k in ("gemini-flash-latest", "gemini/gemini-flash-latest",
          "gemini-flash-lite-latest", "gemini/gemini-flash-lite-latest"):
    v = d[k]
    print(f"{k:<34} in={v['input_cost_per_token']} out={v['output_cost_per_token']} "
          f"cache={v['cache_read_input_token_cost']} "
          f"({v['cache_read_input_token_cost']/v['input_cost_per_token']:.0%} of input)")
EOF

gemini-flash-latest                in=3e-07 out=2.5e-06 cache=3e-08 (10% of input)
gemini/gemini-flash-latest         in=3e-07 out=2.5e-06 cache=7.5e-08 (25% of input)
gemini-flash-lite-latest           in=1e-07 out=4e-07 cache=1e-08 (10% of input)
gemini/gemini-flash-lite-latest    in=1e-07 out=4e-07 cache=2.5e-08 (25% of input)

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

main, checked today

Twitter / LinkedIn details

No response

Activity

  1. Jokasa7 commented on Aug 24, 2026

    @Jokasa7

    I'd like to take a focused first PR for this.

    I rechecked litellm_internal_staging: the two high-confidence *-latest aliases still use the 25% cache-read ratio while their canonical Gemini aliases use the current 10% ratio. My proposed first scope is deliberately narrow:

    • update only gemini/gemini-flash-latest (7.5e-08 → 3e-08) and gemini/gemini-flash-lite-latest (2.5e-08 → 1e-08);
    • add a deterministic local regression test proving each slash-prefixed alias has the same cache-read cost / completion_cost behavior as its canonical alias;
    • cite the official Gemini pricing source in the PR and leave the preview and Vertex entries out until their historical pricing is independently confirmed;
    • run the targeted pricing tests and make lint.

    I understand the PR must target litellm_internal_staging and that the CLA must be completed before opening it. Would a maintainer be comfortable assigning this issue with that limited scope?

    AI assistance disclosure: I am using Codex to assist with repository analysis and implementation planning. I will review the patch myself and run/report all tests before submitting anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions