Repository navigation
fix(studio/pricing): long-context tier starts strictly above the threshold, not at it - #9527
Merged
danielhanchen merged 1 commit intoAug 25, 2026
Conversation
…shold OpenAI prices "prompts with >272K input tokens" at 2x input / 1.5x output for the full session, so a prompt of exactly 272,000 tokens is still on the standard rate. calculate_cost used `>=`, which flipped the whole turn one token early -- gpt-5.5 at exactly 272,000 in was billed $2.72 instead of $1.36. All five long_context_threshold entries in pricing.py are OpenAI 272_000, so `>=` -> `>` is the complete fix with no other vendor affected. The boundary test asserted the old behaviour and is updated to match the vendor wording.
This was referenced Aug 22, 2026
Member
|
Confirmed this hits studio/backend/core/inference/pricing.py where the long-context flip uses greater-or-equal while the emitted label and the vendor wording both say strictly above, so an exactly-272,000-token prompt is priced at the higher rate. Will get this reviewed. |
danielhanchen
pushed a commit
that referenced
this pull request
Aug 25, 2026
…shold (#9527) OpenAI prices "prompts with >272K input tokens" at 2x input / 1.5x output for the full session, so a prompt of exactly 272,000 tokens is still on the standard rate. calculate_cost used `>=`, which flipped the whole turn one token early -- gpt-5.5 at exactly 272,000 in was billed $2.72 instead of $1.36. All five long_context_threshold entries in pricing.py are OpenAI 272_000, so `>=` -> `>` is the complete fix with no other vendor affected. The boundary test asserted the old behaviour and is updated to match the vendor wording.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
calculate_costflips a turn onto the long-context tier one token early:OpenAI's own wording is strictly-above:
— https://developers.openai.com/api/docs/models/gpt-5.4 (read 2026-08-22). The
tooltip on the pricing page this table cites in its header comment agrees:
≤272K input tokensfor the standard column.So a prompt of exactly 272,000 tokens is standard-rate at OpenAI, and
double-rate here. On
gpt-5.5, 272,000 in / 1,000 out:mainbillsTwo internal signals point the same way. The label this branch emits is
f"{model} (long-context >{lc_thresh})"— it already says>, on a requestthat is not
>. Andtest_openai_gpt55_short_context_under_272k_uses_base_ratesis named "under 272k" but only ever exercises 100k, so the naming convention in
the suite reads exclusive too.
Scope of the fix
All five
long_context_thresholdentries inpricing.pyare OpenAI272_000(
gpt-5.6-sol,gpt-5.6-terra,gpt-5.6-luna,gpt-5.5,gpt-5.4). There isno Google or Anthropic threshold in that table, so
>=→>is the completefix here with no counter-vendor to worry about. (Worth knowing for later: this
is not a safe global convention. xAI documents the opposite — "requests whose
prompt reaches the listed token threshold are billed at the higher rate for
all tokens in the request", https://docs.x.ai/docs/models — so if a non-OpenAI
threshold is ever added to this table it will need a per-entry flag rather than
another blanket comparison.)
About the test I changed
test_openai_long_context_threshold_boundary_inclusiveasserted the currentbehaviour with the comment
# Threshold is inclusive (>=)., so this is not acase of an untested edge — the convention was pinned deliberately. I've renamed
it to
..._exclusive, flipped it to 272,000 → standard / 272,001 → long-context,and put the vendor quote in the comment so the next reader sees where the
convention comes from. If that inclusive reading was deliberate for a reason not
in the repo, say so and I'll close this — but the vendor sentence and the
(long-context >{n})label both read the other way.The cache-inflation test (
250k + 50k cache_creation→ 300,000 billable) isunaffected; it stays comfortably above the line.
Verification
Ran
studio/backend/tests/test_pricing.py(37 passed) andstudio/backend/tests/test_pricing_edge.py(24 passed; the 25th,test_build_usage_chunk_forwards_anthropic_cache_creation_breakdown, importscore.inference.external_provider, which I didn't have wired up locally — itdoesn't touch pricing).