Repository navigation
Description
Summary
The cost-optimized routing strategy treats any model that exists in the catalog as having a price, even when its catalog price is null (unknown / not per-token billed). Null-priced models compute to $0 and are therefore always selected as the "cheapest" route.
Root cause
models.Calculate collapses a nil price to 0 cost (models/calculator.go):
func perM(price *float64, n int) float64 {
if price == nil || n == 0 { return 0 } // nil price → 0 cost
return *price * float64(n) / 1_000_000
}internal/strategies/costoptimized.go:58 then sets hasPrice from ModelFound, not from whether a real price exists:
candidates = append(candidates, priced{
target: t,
costUSD: result.InputUSD, // 0 when price is null
hasPrice: result.ModelFound, // true whenever the model is in the catalog
})The cheapest-wins loop (lines 67-77) then picks the null-priced candidate because its costUSD is 0.
Impact
Any provider with null catalog pricing, when placed in a cost-optimized target group, is always chosen as cheapest regardless of actual cost.
- Today: affects ZAI GLM models (prices are
nullin the catalog). - Incoming: the NanoGPT aggregator PR (feat: add NanoGPT provider and missing ZAI GLM-5 models model-catalog#13) is credit-based and will carry
nullprices by design. Per the catalog convention,null= "not applicable / unknown," but the router currently reads it as "$0 = cheapest."
This is a silent misroute: cost-optimizing operators would have all matching traffic pulled onto whichever provider happens to have unpriced catalog entries.
Expected behavior
A model with unknown (null) per-token price should be treated as unknown cost, not zero cost. Options:
- Change
hasPriceto mean "has a non-null input/output price" (distinguish "in catalog" from "priced"). Exclude unknown-cost models from cheapest-selection, or rank them last, instead of treating them as $0. - Optionally surface a config knob for how to treat unknown-cost candidates (skip vs. fallback vs. allow).
Repro sketch
- Configure a cost-optimized target group with two providers that both support model
X: one with real catalog pricing, one withnullpricing (e.g. a ZAI GLM model). - Send a request for model
X. - Observe the null-priced provider is always selected as "cheapest."
Notes
- Catalog side needs no change.
nullis the correct encoding for credit-based / unpriced models. The fix belongs in the router (and possibly theCostResultcontract, e.g. aPriced boolseparate fromModelFound).
Activity
Rachit-Gandhi commentedon May 24, 2026
@MitulShah1 I would like to take this issue up, if you can assign it to me.
MitulShah1 commentedon May 24, 2026
Hey @Rachit-Gandhi thanks, assigned to you.
Rachit-Gandhi commentedon May 24, 2026
@MitulShah1 There is no develop branch, should I raise a pr to main or a particular branch?
MitulShah1 commentedon May 24, 2026
Hey Yeah i will map to release branch later.
fix(strategies): treat null-priced catalog models as unknown cost, no…
docs(changelog): add v1.1.1 stability hotfix entry
fix: release v1.1.1 — stability hotfix (C1/C2/C3/C4 + H5/H7 + #126) (#…