Repository navigation
Conversation
There was a problem hiding this comment.
1 issue found across 8 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="prices/providers/deepseek.yml">
<violation number="1" location="prices/providers/deepseek.yml:144">
P1: On Beijing weekends covered by this policy, requests in either listed UTC peak window still receive peak rates because both constraints test time only. Add weekday-aware constraint handling or equivalent price-selection logic; documenting an upper bound does not implement off-peak billing.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| cache_read_mtok: 0.007 | ||
| output_mtok: 0.66 | ||
| - constraint: | ||
| start_time: 01:00:00Z |
There was a problem hiding this comment.
P1: On Beijing weekends covered by this policy, requests in either listed UTC peak window still receive peak rates because both constraints test time only. Add weekday-aware constraint handling or equivalent price-selection logic; documenting an upper bound does not implement off-peak billing.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At prices/providers/deepseek.yml, line 144:
<comment>On Beijing weekends covered by this policy, requests in either listed UTC peak window still receive peak rates because both constraints test time only. Add weekday-aware constraint handling or equivalent price-selection logic; documenting an upper bound does not implement off-peak billing.</comment>
<file context>
@@ -127,11 +127,33 @@ models:
+ cache_read_mtok: 0.007
+ output_mtok: 0.66
+ - constraint:
+ start_time: 01:00:00Z
+ end_time: 04:00:00Z
+ prices:
</file context>
There was a problem hiding this comment.
Confirmed, and I do not think it can be fixed in this PR.
Every constraint type available today tests time-of-day or start-date and nothing else, so there is no data-only way to say "off-peak all day on Saturday and Sunday, Beijing time". Adding weekday-aware selection means a new or widened constraint type, which lands in the published v2 payload and has to ship identically in both engines — a contract change, not a price update.
So I have raised it as #582 (option B there is exactly your suggestion: an optional weekdays field on TimeOfDateConstraint) and left the disclosure in price_comments in the meantime. Happy to implement it as soon as a maintainer picks a shape.
Worth noting on impact: the rule starts 2026-08-23, and the error is bounded and one-directional — at most 2x on 6 of 168 hours per week (the two peak windows on Saturday and Sunday), always an overestimate, never an underestimate. The alternative available today is dropping the peak entries entirely, which would understate every weekday peak request by the same 2x. Between the two, overstating a weekend bill seemed the safer default, but I would rather it be neither.
There was a problem hiding this comment.
All reported issues were addressed across 6 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
5aa01ed to
6024fb4
Compare
Fixes pydantic#571. Since 2026-08-17 both models are billed at two rates depending on the time of day, with peak hours in two disjoint windows: 01:00-04:00 and 06:00-10:00 UTC. Source: https://api-docs.deepseek.com/quick_start/pricing/, read 2026-08-22. The pre-2026-08-17 flat rate stays as the unconstrained first price, the new off-peak rate is gated on start_date, and the two peak windows come last, so under last-to-first resolution the history is preserved for the 17 hours a day that fall outside the peak windows. Two limitations are stated in price_comments rather than guessed at, and raised as pydantic#582: pre-2026-08-17 requests inside the peak windows still resolve to the peak rate, because `constraint` is a union and one entry cannot carry both a date and a daily window; and the weekend off-peak rule starting 2026-08-23 needs a day-of-week condition, so on Saturdays and Sundays the peak prices are an upper bound. Adds parametrized tests on both engines: eight hours across the window edges and the gap for the current rates, and four hours for the preserved history.
6024fb4 to
05442ca
Compare
Only tests/test_price_calc.py conflicted: main (pydantic#536) and this branch each appended a block of tests at the same spot. Neither block references a name from the other, so both are kept, main's first. Regenerated afterwards to check the auto-merged generated files were still consistent -- 'prices build' reports both data files unchanged. pytest: 1168 passed, 42 xfailed. packages/js: 1883 passed, 1 skipped.
Fixes #571.
What this changes
deepseek-v4-flashanddeepseek-v4-prostill carried flat rates checked on 2026-05-12 and 2026-06-02. Since 2026-08-17 DeepSeek bills both at two rates depending on the time of day.Source: https://api-docs.deepseek.com/quick_start/pricing/, read 2026-08-22. Peak hours are 01:00–04:00 UTC and 06:00–10:00 UTC; every other hour is off-peak.
input_mtokcache_read_mtokoutput_mtokThe old flat rate stays as the unconstrained first price, the new off-peak rate is gated on
start_date: 2026-08-17, and the two peak windows come last. Both engines try conditional prices last-to-first and fall back to the first entry, so:V4 needs two peak entries rather than the one
deepseek-chatuses, because its peak hours are two disjoint windows. Neither shadows the other.deepseek-v4-flash-vision-expmatchesstarts_with: deepseek-v4-flashand DeepSeek publishes exactly the flash rates for it, so it is covered without a separate entry.Two things I could not express, flagged rather than guessed
1. Price history is preserved for 17 hours of the day, not all 24.
constraintisStartDateConstraint | TimeOfDateConstraint | None, a union, so one entry cannot be both "from 2026-08-17" and "during 01:00–04:00 UTC". The dated off-peak entry recovers every pre-repricing hour outside the peak windows, but a request from May at 02:00 UTC still matches the time-of-day peak entry and gets billed at August peak rates. That residual gap is stated inprice_commentson both models and pinned by a test, so the data does not look authoritative for dates it isn't. Raised as #582.(An earlier revision of this PR overwrote the old rates outright and claimed interleaving the two constraint kinds was strictly worse. That was wrong — cubic caught it. The layout above is its suggestion.)
2. The weekend rule. From midnight Beijing time on 2026-08-23, DeepSeek bills off-peak all day on Saturdays and Sundays.
start_time/end_timealone cannot express a day-of-week condition, so on weekends the peak entries here are an upper bound. The error is bounded and one-directional: at most 2x on 6 of 168 hours per week, always an overestimate. Dropping the peak entries instead would understate every weekday peak request by the same 2x. Noted inprice_commentson both models and raised as #582. (I raised the same limitation on simonw/llm-prices, so it is not specific to this repo's schema.)One thing worth a second look, not changed here
deepseek-v4-pro's previous0.435 / 0.003625 / 0.87matches LiteLLM'stencent/deepseek-v4-pro(0.4350 / 0.0036 / 0.8700) to four significant figures, including the unusual0.003625. That looks like a Tencent reseller price that landed under apricing_urlspointing at DeepSeek's own page. This PR replaces it with DeepSeek's published rate either way, but you may want to check whether the importer that produced it is mixing resellers into first-party providers elsewhere.Tests
tests/dataset/usages.jsonpins a single UTC instant (2025-11-06 12:00Z, off-peak), so it cannot catch constraint-resolution drift between the two engines — which is exactly what the two-window shape introduces. So this adds a parametrized test on each side over the shipped data, covering both window edges and the gap between them:tests/test_price_calc.py::test_price_constraint_two_time_of_date_windowspackages/js/src/__tests__/comprehensive.test.ts→calcPrice - Deepseek V4 peak windowsBoth run the same eight hours against both repriced models — 00:00 off-peak, 01:00 peak (inclusive start), 04:00 off-peak (exclusive end), 05:00 off-peak (the gap), 06:00 peak, 09:00 peak, 10:00 off-peak, 23:00 off-peak — so a wrong rate on either flash ($22/$44 per 100M input tokens) or pro ($66/$132) fails CI on both sides.
A second pair pins the preserved history and the known residual gap at 2026-05-01:
tests/test_price_calc.py::test_price_deepseek_v4_before_repricingpackages/js/src/__tests__/comprehensive.test.ts→calcPrice - Deepseek V4 before the 2026-08-17 repricing00:00, 12:00 and 23:00 must return the old flat rate (flash $14, pro $43.50 per 100M input tokens); 02:00 is asserted to return the peak rate, so the 7h/day gap in §1 is a pinned known limitation rather than something that can drift silently. 24 cases per language.
usages.jsonis byte-identical tomain: its pinned instant is 2025-11-06 12:00Z, which now resolves to the preserved flat rate rather than the new one.Verification
Every generated file here was written by
make build— none hand-edited.Diff of the generated v2 payload, confirming nothing outside these two models moved: