Skip to content

Tier boundary is exclusive for every provider, but xAI and OpenAI document it as inclusive #587

Closed

Description

@xyzs996

calc_unit_price picks a tier with a single, global comparison:

https://github.com/pydantic/genai-prices/blob/main/packages/python/genai_prices/types.py#L839-L842

for tier in reversed(price.tiers):
    if total_input_tokens > tier.start:

Strictly greater, for every provider. But providers do not agree about the token at the threshold, and two of them use the same threshold number and give opposite answers.

Google — exclusive. The pricing table spells it out in prose:

$2.00, prompts <= 200k tokens $4.00, prompts > 200k tokens

— Gemini API pricing, read 2026-08-22. A prompt of exactly 200,000 tokens bills at the cheap rate. > is correct here.

xAI — inclusive. The models table splits each model into < 200k prompt tokens and ≥ 200k prompt tokens, with the footnote:

requests whose prompt reaches the listed token threshold are billed at the higher rate for all tokens in the request

— xAI models & pricing, read 2026-08-22. A prompt of exactly 200,000 tokens bills at the expensive rate. > undercharges by 2× at that token.

OpenAI — inclusive, but only implied. The pricing table labels rows gpt-5.5 (<272K context length), gpt-5.4 (<272K context length) and so on, with separate long-context columns. Strictly less than, so exactly 272,000 falls outside the short-context bracket. Worth noting this one is inferred from a row label — that page never states the boundary in prose, unlike the other two.

Reproduction

On main plus #583 (which adds xAI's tiers — before that PR the xAI rows are flat, so the boundary question does not arise there yet; #586 was a duplicate of it and is closed):

from genai_prices import calc_price
from genai_prices.types import Usage

for pid, mid in [('x-ai', 'grok-4.6'), ('google', 'gemini-3.1-pro-preview')]:
    for n in (199_999, 200_000, 200_001):
        p = calc_price(Usage(input_tokens=n, output_tokens=0), model_ref=mid, provider_id=pid)
        print(f'{pid}/{mid} {n:>7,} -> ${p.input_price}')
x-ai/grok-4.6                   199,999 -> $0.399998
x-ai/grok-4.6                   200,000 -> $0.4          <- xAI bills $0.80
x-ai/grok-4.6                   200,001 -> $0.800004

google/gemini-3.1-pro-preview   199,999 -> $0.399998
google/gemini-3.1-pro-preview   200,000 -> $0.4          <- correct
google/gemini-3.1-pro-preview   200,001 -> $0.800004

Same number, same output, one right and one wrong. No single global operator can be correct for both.

Scope

Narrow in probability, not in magnitude: it only bites on a prompt of exactly the threshold, but when it does the whole request is mispriced by ~2×, silently, in the direction of under-reporting. Anyone reconciling estimates against a real invoice sees an unexplained gap and has no way to find it — the threshold number in the data looks right, because it is.

It also affects any tier boundary added in future, so the cost of leaving it grows with the dataset.

Possible shapes of a fix

I have not sent a PR because this is a schema decision that's yours to make. Rough options:

  1. Per-tier inclusivity flag — Tier.inclusive: bool = False, with >= when set. Backwards compatible; default keeps today's behaviour, so only entries that opt in change. Most explicit, one more field to get right per entry.
  2. Provider-level default — a tier_boundary: exclusive | inclusive on the provider, since in practice this seems to be a house convention rather than per-model. Fewer places to set it, but wrong the day a provider is inconsistent with itself.
  3. Encode it in the number — store xAI's threshold as 199999. No schema change, but it makes the data disagree with the vendor's published number, which seems worse than the bug.

I'd lean to (1) for explicitness, defaulting to today's behaviour so nothing moves until an entry opts in.

Happy to implement whichever you prefer, or to close this if you'd rather accept the boundary token as out of scope — in which case a note on TieredPrices saying so would still help, since its docstring currently presents > as the industry-standard cliff model:

Example: For a tier starting at 200K tokens:

  • Using 199,999 tokens: all tokens pay base rate
  • Using 200,001 tokens: all tokens pay tier rate

which quietly skips the one token in question.

Activity

  1. pydanty commented on Aug 26, 2026

    @pydanty

    pydanty runtime failure

    Run df_run_da5982db60fb43769c5b started from the GitHub webhook, but the Dogfooding runtime
    failed before pydanty completed the workflow. The pydanty status comment above
    (if any) is from a completed run and still stands.

    Next step: maintainer attention needed; rerun after fixing the runtime failure.

    Failure detail (tail)
    text
    20:37:11.635         running tool: search_repository_text
    20:37:11.717         running tool: search_repository_text
    20:37:12.000         chat fallback:glm-5.3
    20:38:07.613         running tool: read_repository_file
    20:38:07.621         running tool: search_repository_text
    20:38:07.638         running tool: search_repository_text
    20:38:07.650         running tool: search_repository_text
    20:38:07.661         running tool: search_repository_text
    20:38:07.919         chat fallback:glm-5.3
    20:39:05.729         running tool: search_repository_text
    20:39:05.735         running tool: search_repository_text
    20:39:05.843         chat fallback:glm-5.3
    gh-token refreshed
    20:40:19.779         running output function: final_result
    20:40:19.852     chat fallback:glm-5.3,gpt-5.6-luna
    20:40:43.840     running tool: ask_reproducer
    AGENT_ERROR (relevant leg):
      File "/home/pydanty/dogfooding/.runner.py", line 204, in <module>
        asyncio.run(main())
      File "/home/pydanty/dogfooding/.runner.py", line 169, in main
        result = await result
                 ^^^^^^^^^^^^
      File "/home/pydanty/dogfooding/agents/pydanty/triage_agent.py", line 657, in run
        result = await TriageWorkflowController(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/pydanty/dogfooding/agents/pydanty/workflow.py", line 873, in run
        execution = await self.executor(phase, inputs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/pydanty/dogfooding/agents/pydanty/triage_agent.py", line 539, in __call__
        result = await phase_agent.run(
                 ^^^^^^^^^^^^^^^^^^^^^^
    UnexpectedModelBehavior: Tool 'ask_reproducer' exceeded max retries count of 1. Consider raising the retry limit, or see the docs on tool retries: https://pydantic.dev/docs/ai/tools-toolsets/tools-advanced/#tool-retries
    
    
  2. AmirF194 commented on Aug 31, 2026

    @AmirF194

    Option 1 (a per-tier inclusive flag, defaulting to today's exclusive > behavior) looks like the safer shape to me: it is backwards compatible, keeps every existing entry unchanged, and avoids guessing at a provider-level convention that could turn out inconsistent across a provider's own models.

    If that direction works for you, I am happy to open a PR that adds the flag and wires it into the xAI and OpenAI tiers this issue found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions