Skip to content

fix(wandb): 13 rows priced 100,000x too high (gpt-oss-120b bills at $15,000/1M) - #38015

Closed
xyzs996 wants to merge 1 commit into
BerriAI:mainfrom
xyzs996:wandb-units-2026-08-23
Closed

xyzs996 wants to merge 1 commit into
BerriAI:mainfrom
xyzs996:wandb-units-2026-08-23

Conversation

@xyzs996

@xyzs996 xyzs996 commented Aug 23, 2026

Copy link
Copy Markdown

What's wrong

13 of the 16 wandb/* rows store the price per 1M tokens divided by 10, in a field that is read as dollars per token. wandb/openai/gpt-oss-120b currently prices at 0.015/token — $15,000 per 1M.

The tell is inside the same provider block: three rows already use per-token units.

row stored reads as
wandb/MiniMaxAI/MiniMax-M2.5 3e-07 $0.30 / 1M — matches the published rate
wandb/moonshotai/Kimi-K2.5 6e-07 $0.60 / 1M
wandb/deepseek-ai/DeepSeek-V3.1 0.055 $55,000 / 1M

Check that doesn't depend on trusting me

Divide the 13 rows by 100,000 and the ones W&B still lists land on the published rate exactly, to the cent — nothing rounded, nothing nudged (pricing, read 2026-08-23):

row this PR W&B publishes
deepseek-ai/DeepSeek-V3.1 5.5e-07 / 1.65e-06 $0.55 / $1.65
meta-llama/Llama-3.1-8B-Instruct 2.2e-07 / 2.2e-07 $0.22 / $0.22
meta-llama/Llama-3.3-70B-Instruct 7.1e-07 / 7.1e-07 $0.71 / $0.71
Qwen/Qwen3-Coder-480B-A35B-Instruct 1e-06 / 1.5e-06 $1.00 / $1.50

Four independent rows landing exactly is not what a wrong divisor looks like.

The remaining rows (GLM-4.5, both Qwen3-235B, DeepSeek R1-0528 and V3-0324, Llama-4-Scout, Phi-4-mini) are no longer on the pricing page, so they keep the price they were entered with, only in the right unit.

Two rows where the price itself also moved

gpt-oss-120b and gpt-oss-20b are the only rows where the unit fix alone would still be wrong — W&B has lowered both since they were added. They are set to today's published numbers: $0.03 / $0.17 and $0.03 / $0.13 per 1M. This is the only place a number comes from anywhere but the existing row. If you'd rather keep this PR mechanical, revert those four values and the rest still stands.

Scope

Both copies of the registry, 13 rows, 26 values. No other provider, no key added or removed, no schema change.

Note on the earlier PRs

#37930 was closed as superseded by #37902. I re-read main today: the 13 rows are unchanged, and #37902's diff to the registry doesn't contain them. Re-filing against current main rather than reopening a branch that had drifted 86 commits behind.

@xyzs996
xyzs996 requested a review from mateo-berri as a code owner August 23, 2026 09:57
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Corrects inflated WandB model pricing by storing costs in dollars per token and updates two GPT-OSS models to current published rates.

  • Updates input and output costs for 13 WandB model entries.
  • Keeps the primary and bundled backup pricing registries synchronized.

Confidence Score: 5/5

The PR appears safe to merge because both pricing registries consistently apply the corrected per-token values.

The changed values use the registry’s dollars-per-token convention, documented active-model rates corroborate the corrections, and the runtime fallback copy remains synchronized with the primary registry.

Important Files Changed

Filename Overview
model_prices_and_context_window.json Corrects 13 WandB pricing entries to per-token units, including current GPT-OSS rates; no actionable issues identified.
litellm/model_prices_and_context_window_backup.json Mirrors the primary registry’s WandB pricing corrections exactly for bundled fallback use.

Reviews (1): Last reviewed commit: "fix(wandb): 13 rows priced 100,000x too ..." | Re-trigger Greptile

@xyzs996

xyzs996 commented Aug 23, 2026

Copy link
Copy Markdown
Author

Cross-check that doesn't need the vendor page: models.dev carries the same provider, in $/1M. Every overlapping row agrees with this PR to the cent, including the two I refreshed.

row this PR, as $/1M models.dev
openai/gpt-oss-120b 0.03 / 0.17 0.03 / 0.17
openai/gpt-oss-20b 0.03 / 0.13 0.03 / 0.13
meta-llama/Llama-3.1-8B-Instruct 0.22 / 0.22 0.22 / 0.22
meta-llama/Llama-3.3-70B-Instruct 0.71 / 0.71 0.71 / 0.71
deepseek-ai/DeepSeek-V3.1 0.55 / 1.65 0.55 / 1.65
Qwen/Qwen3-Coder-480B-A35B-Instruct 1.00 / 1.50 1.00 / 1.50

MiniMaxAI/MiniMax-M2.5 is the control: it's one of the three rows already in per-token units here, and it reads 0.30 / 1.20 in both.

Paths are providers/wandb/models/<vendor>/<model>.toml in anomalyco/models.dev.

@codecov

codecov Bot commented Aug 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing xyzs996:wandb-units-2026-08-23 (e34ae50) with main (947dbbf)

Open in CodSpeed

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Rolled into the single rolling registry PR #37902 (maintainers keep one open model_prices_and_context_window.json PR at a time). Your values were re-verified against the official provider docs and are on that branch, so closing this one. Thanks for the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants