Skip to content

Commit b647ec7

Browse files
committed
fix(cost): bill the TTL split on the telemetry recompute and in tier blocks
What: - Pass `cacheWrite1h`/`cacheWrite12h` to the cost call inside `captureUsageData`. The bag already carried them and fed the shared accumulator vars, but the telemetry recompute passed only the aggregate write count, so a cancelled request's telemetry billed every write at the short-TTL rate. - Add `cacheWritesPrice1h`/`cacheWritesPrice12h` to the `tiers` schema and to the four Anthropic and four Bedrock 1M-context tier blocks, derived from each tier's own input price. - Carry the tier's TTL rates through all three tier-merge seams (anthropic, anthropic-vertex, bedrock) and derive them from the tier input price in the OpenAI service-tier seam, so a flex/priority or 1M-context write is not billed at the base tier's TTL rate. Why: Two under-reports survived the first pass. `captureUsageData` recomputes cost for telemetry when a provider yields no final cost chunk (the cancelled-stream path), and it dropped the split: 100k 1h tokens at claude-3-5-sonnet billed 0.375 instead of 0.6. Separately, the 1M-context tier merge overrode `inputPrice` but not the TTL rate, so the entry-level rate derived from the 200K input price survived: a 1M-context 1h write billed 6.0 instead of the tier's 12.0, a 50% under-report. Impact: Both paths now bill at the correct TTL rate. Three new tests fail on the pre-fix code (0.375 vs 0.6, and 6 vs 12); a fourth asserts the tier rate reaches the cost function. No request shape changes.
1 parent 0f52a75 commit b647ec7

12 files changed

Lines changed: 196 additions & 1 deletion

File tree

‎packages/types/src/model.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,6 +187,11 @@ export const modelInfoSchema = z.object({
187187
inputPrice: z.number().optional(),
188188
outputPrice: z.number().optional(),
189189
cacheWritesPrice: z.number().optional(),
190+
// Per-TTL write rates for this tier. A tier that overrides inputPrice
191+
// must also override these, otherwise a long-context or flex/priority
192+
// write bills at the base tier's TTL rate.
193+
cacheWritesPrice1h: z.number().optional(),
194+
cacheWritesPrice12h: z.number().optional(),
190195
cacheReadsPrice: z.number().optional(),
191196
}),
192197
)

‎packages/types/src/providers/anthropic.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ export const anthropicModels = {
2626
inputPrice: 6.0, // $6 per million input tokens (>200K context)
2727
outputPrice: 22.5, // $22.50 per million output tokens (>200K context)
2828
cacheWritesPrice: 7.5, // $7.50 per million tokens (>200K context)
29+
cacheWritesPrice1h: 12, // $12.00 per million tokens (2x tier input price, 1h cache writes)
2930
cacheReadsPrice: 0.6, // $0.60 per million tokens (>200K context)
3031
},
3132
],
@@ -68,6 +69,7 @@ export const anthropicModels = {
6869
inputPrice: 6.0, // $6 per million input tokens (>200K context)
6970
outputPrice: 22.5, // $22.50 per million output tokens (>200K context)
7071
cacheWritesPrice: 7.5, // $7.50 per million tokens (>200K context)
72+
cacheWritesPrice1h: 12, // $12.00 per million tokens (2x tier input price, 1h cache writes)
7173
cacheReadsPrice: 0.6, // $0.60 per million tokens (>200K context)
7274
},
7375
],
@@ -90,6 +92,7 @@ export const anthropicModels = {
9092
inputPrice: 6.0, // $6 per million input tokens (>200K context)
9193
outputPrice: 22.5, // $22.50 per million output tokens (>200K context)
9294
cacheWritesPrice: 7.5, // $7.50 per million tokens (>200K context)
95+
cacheWritesPrice1h: 12, // $12.00 per million tokens (2x tier input price, 1h cache writes)
9396
cacheReadsPrice: 0.6, // $0.60 per million tokens (>200K context)
9497
},
9598
],
@@ -112,6 +115,7 @@ export const anthropicModels = {
112115
inputPrice: 10.0, // $10 per million input tokens (>200K context)
113116
outputPrice: 37.5, // $37.50 per million output tokens (>200K context)
114117
cacheWritesPrice: 12.5, // $12.50 per million tokens (>200K context)
118+
cacheWritesPrice1h: 20, // $20.00 per million tokens (2x tier input price, 1h cache writes)
115119
cacheReadsPrice: 1.0, // $1.00 per million tokens (>200K context)
116120
},
117121
],

‎packages/types/src/providers/bedrock.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ export const bedrockModels = {
4949
inputPrice: 6.0, // $6 per million input tokens (>200K context)
5050
outputPrice: 22.5, // $22.50 per million output tokens (>200K context)
5151
cacheWritesPrice: 7.5, // $7.50 per million tokens (>200K context)
52+
cacheWritesPrice1h: 12, // $12.00 per million tokens (2x tier input price, 1h cache writes)
5253
cacheReadsPrice: 0.6, // $0.60 per million tokens (>200K context)
5354
},
5455
],
@@ -188,6 +189,7 @@ export const bedrockModels = {
188189
inputPrice: 10.0, // $10 per million input tokens (>200K context)
189190
outputPrice: 37.5, // $37.50 per million output tokens (>200K context)
190191
cacheWritesPrice: 12.5, // $12.50 per million tokens (>200K context)
192+
cacheWritesPrice1h: 20, // $20.00 per million tokens (2x tier input price, 1h cache writes)
191193
cacheReadsPrice: 1.0, // $1.00 per million tokens (>200K context)
192194
},
193195
],
@@ -213,6 +215,7 @@ export const bedrockModels = {
213215
inputPrice: 10.0, // $10 per million input tokens (>200K context)
214216
outputPrice: 37.5, // $37.50 per million output tokens (>200K context)
215217
cacheWritesPrice: 12.5, // $12.50 per million tokens (>200K context)
218+
cacheWritesPrice1h: 20, // $20.00 per million tokens (2x tier input price, 1h cache writes)
216219
cacheReadsPrice: 1.0, // $1.00 per million tokens (>200K context)
217220
},
218221
],
@@ -240,6 +243,7 @@ export const bedrockModels = {
240243
inputPrice: 10.0, // $10 per million input tokens (>200K context)
241244
outputPrice: 37.5, // $37.50 per million output tokens (>200K context)
242245
cacheWritesPrice: 12.5, // $12.50 per million tokens (>200K context)
246+
cacheWritesPrice1h: 20, // $20.00 per million tokens (2x tier input price, 1h cache writes)
243247
cacheReadsPrice: 1.0, // $1.00 per million tokens (>200K context)
244248
},
245249
],

‎src/api/providers/__tests__/anthropic-vertex.spec.ts‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import { AnthropicVertexHandler } from "../anthropic-vertex"
1010
import { MAX_TIMEOUT_MS } from "../utils/abort-signal"
1111
import { asyncStreamFrom, collectStream } from "../../../test-utils/stream"
1212
import { makeCreateMessageMetadata } from "../../../test-utils/api"
13+
import type { ApiStreamChunk } from "../../../api/transform/stream"
1314

1415
vitest.mock("../utils/timeout-config", () => ({
1516
getApiRequestTimeout: vitest.fn().mockReturnValue(300_000),
@@ -453,6 +454,41 @@ describe("VertexHandler", () => {
453454
)
454455
})
455456

457+
it("should surface the 1h cache-write split from cache_creation", async () => {
458+
const mockStream = [
459+
{
460+
type: "message_start",
461+
message: {
462+
usage: {
463+
input_tokens: 10,
464+
output_tokens: 0,
465+
cache_creation_input_tokens: 100_000,
466+
cache_creation: {
467+
ephemeral_5m_input_tokens: 40_000,
468+
ephemeral_1h_input_tokens: 60_000,
469+
},
470+
cache_read_input_tokens: 0,
471+
},
472+
},
473+
},
474+
]
475+
476+
const mockCreate = vitest.fn().mockResolvedValue(asyncStreamFrom(mockStream))
477+
// Typed structural cast rather than `as any`: the client is a private field and
478+
// this file already has 31 suppressed `any` uses, a budget this test must not grow.
479+
;(handler["client"].messages as unknown as { create: typeof mockCreate }).create = mockCreate
480+
481+
const stream = handler.createMessage(systemPrompt, [{ role: "user", content: "Hello" }])
482+
const chunks = await collectStream(stream)
483+
484+
const usageChunk = chunks.find(
485+
(chunk): chunk is Extract<ApiStreamChunk, { type: "usage" }> =>
486+
chunk.type === "usage" && chunk.cacheWriteTokens !== undefined,
487+
)
488+
expect(usageChunk?.cacheWriteTokens).toBe(100_000)
489+
expect(usageChunk?.cacheWrite1hTokens).toBe(60_000)
490+
})
491+
456492
it("should handle cache-related usage metrics for Claude", async () => {
457493
handler = new AnthropicVertexHandler({
458494
apiModelId: "claude-3-5-sonnet-v2@20241022",

‎src/api/providers/__tests__/anthropic.spec.ts‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ import { asyncStreamFrom, collectStream } from "../../../test-utils/stream"
77
import { makeCreateMessageMetadata } from "../../../test-utils/api"
88
import type { ApiStreamChunk, ApiStreamUsageChunk } from "../../../api/transform/stream"
99
import { clearAllMocks } from "../../../test-utils/reset"
10+
import { calculateApiCostAnthropic } from "../../../shared/cost"
1011

1112
// Mock TelemetryService
1213
vitest.mock("@roo-code/telemetry", () => ({
@@ -1538,6 +1539,38 @@ describe("AnthropicHandler", () => {
15381539
expect(model.info.outputPrice).toBe(22.5)
15391540
})
15401541

1542+
it("carries the tier's own cacheWritesPrice1h when the 1M context tier applies", () => {
1543+
// glm P2: the tier merge overrode inputPrice but not the TTL rate, so the
1544+
// entry-level 1h rate (derived from the 200K input price) survived and a
1545+
// 1M-context 1h write billed 6.0 instead of the tier's 12.0 (50% under).
1546+
const handler = new AnthropicHandler({
1547+
apiKey: "test-api-key",
1548+
apiModelId: "claude-sonnet-4-5",
1549+
anthropicBeta1MContext: true,
1550+
})
1551+
const model = handler.getModel()
1552+
1553+
// Tier input price 6.0 => 1h rate must be 2x that, not the 200K entry's 6.0.
1554+
expect(model.info.cacheWritesPrice1h).toBe(12.0)
1555+
expect(model.info.cacheWritesPrice1h).not.toBe(6.0)
1556+
})
1557+
1558+
it("bills a 1M-context 1h cache write at the tier rate, not the 200K rate", () => {
1559+
const handler = new AnthropicHandler({
1560+
apiKey: "test-api-key",
1561+
apiModelId: "claude-sonnet-4-5",
1562+
anthropicBeta1MContext: true,
1563+
})
1564+
const model = handler.getModel()
1565+
1566+
const { totalCost } = calculateApiCostAnthropic(model.info, 0, 0, 100_000, 0, 100_000)
1567+
1568+
// Tier 1h rate 12.0: (12.0 / 1_000_000) * 100_000 = 1.2
1569+
// The pre-fix behaviour billed the 200K 1h rate: 0.6 (50% under).
1570+
expect(totalCost).toBeCloseTo(1.2, 9)
1571+
expect(totalCost).not.toBeCloseTo(0.6, 9)
1572+
})
1573+
15411574
it("should honor a custom/unrecognized model ID instead of silently falling back to anthropicDefaultModelId", () => {
15421575
const handler = new AnthropicHandler({
15431576
apiKey: "test-api-key",

‎src/api/providers/__tests__/minimax.spec.ts‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -373,6 +373,38 @@ describe("MiniMaxHandler", () => {
373373
expect(firstChunk.value).toEqual({ type: "usage", inputTokens: 10, outputTokens: 20 })
374374
})
375375

376+
it("createMessage should surface the 1h cache-write split from cache_creation", async () => {
377+
mockCreate.mockResolvedValueOnce(
378+
asyncStreamFrom([
379+
{
380+
type: "message_start",
381+
message: {
382+
usage: {
383+
input_tokens: 10,
384+
output_tokens: 0,
385+
cache_creation_input_tokens: 100_000,
386+
cache_creation: {
387+
ephemeral_5m_input_tokens: 40_000,
388+
ephemeral_1h_input_tokens: 60_000,
389+
},
390+
cache_read_input_tokens: 0,
391+
},
392+
},
393+
},
394+
]),
395+
)
396+
397+
const stream = handler.createMessage("system prompt", [])
398+
const firstChunk = await stream.next()
399+
400+
expect(firstChunk.done).toBe(false)
401+
expect(firstChunk.value).toMatchObject({
402+
type: "usage",
403+
cacheWriteTokens: 100_000,
404+
cacheWrite1hTokens: 60_000,
405+
})
406+
})
407+
376408
it("createMessage should pass correct parameters to MiniMax client", async () => {
377409
const modelId: MinimaxModelId = "MiniMax-M2"
378410
const modelInfo = minimaxModels[modelId]

‎src/api/providers/anthropic-vertex.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,10 @@ export class AnthropicVertexHandler extends BaseProvider implements SingleComple
275275
inputPrice: tier.inputPrice,
276276
outputPrice: tier.outputPrice,
277277
cacheWritesPrice: tier.cacheWritesPrice,
278+
// Carry the tier's own TTL rate; the entry-level one was derived from
279+
// the 200K input price and would halve a 1M-context 1h write.
280+
cacheWritesPrice1h: tier.cacheWritesPrice1h ?? info.cacheWritesPrice1h,
281+
cacheWritesPrice12h: tier.cacheWritesPrice12h ?? info.cacheWritesPrice12h,
278282
cacheReadsPrice: tier.cacheReadsPrice,
279283
}
280284
}

‎src/api/providers/anthropic.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -442,6 +442,11 @@ export class AnthropicHandler extends BaseProvider implements SingleCompletionHa
442442
inputPrice: tier.inputPrice,
443443
outputPrice: tier.outputPrice,
444444
cacheWritesPrice: tier.cacheWritesPrice,
445+
// The tier's own TTL rate must come along: the entry-level rate was
446+
// derived from the 200K input price, so keeping it here would bill a
447+
// 1M-context 1h write at half its tier rate.
448+
cacheWritesPrice1h: tier.cacheWritesPrice1h ?? info.cacheWritesPrice1h,
449+
cacheWritesPrice12h: tier.cacheWritesPrice12h ?? info.cacheWritesPrice12h,
445450
cacheReadsPrice: tier.cacheReadsPrice,
446451
}
447452
}

‎src/api/providers/bedrock.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1295,6 +1295,10 @@ export class AwsBedrockHandler extends BaseProvider implements SingleCompletionH
12951295
inputPrice: tier?.inputPrice ?? modelConfig.info.inputPrice,
12961296
outputPrice: tier?.outputPrice ?? modelConfig.info.outputPrice,
12971297
cacheWritesPrice: tier?.cacheWritesPrice ?? modelConfig.info.cacheWritesPrice,
1298+
// Carry the tier's own TTL rate; the entry-level one was derived from the
1299+
// 200K input price and would halve a 1M-context 1h write.
1300+
cacheWritesPrice1h: tier?.cacheWritesPrice1h ?? modelConfig.info.cacheWritesPrice1h,
1301+
cacheWritesPrice12h: tier?.cacheWritesPrice12h ?? modelConfig.info.cacheWritesPrice12h,
12981302
cacheReadsPrice: tier?.cacheReadsPrice ?? modelConfig.info.cacheReadsPrice,
12991303
}
13001304
}

‎src/api/providers/openai-native.ts‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ import {
1616
SERVICE_TIER_KEY,
1717
type ServiceTier,
1818
ApiProviderError,
19+
CACHE_WRITE_MULTIPLIER_1H,
20+
CACHE_WRITE_MULTIPLIER_12H,
1921
} from "@roo-code/types"
2022
import { TelemetryService } from "@roo-code/telemetry"
2123

@@ -1490,6 +1492,17 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
14901492
outputPrice: tierInfo.outputPrice ?? info.outputPrice,
14911493
cacheReadsPrice: tierInfo.cacheReadsPrice ?? info.cacheReadsPrice,
14921494
cacheWritesPrice: tierInfo.cacheWritesPrice ?? info.cacheWritesPrice,
1495+
// A service tier that overrides the write rate must also override the TTL
1496+
// rate, otherwise a flex/priority 12h write bills at the standard-tier 12h
1497+
// rate. Derived from the tier's own input price so the 2.5x ratio holds.
1498+
cacheWritesPrice12h:
1499+
tierInfo.cacheWritesPrice12h ??
1500+
(tierInfo.inputPrice !== undefined ? tierInfo.inputPrice * CACHE_WRITE_MULTIPLIER_12H : undefined) ??
1501+
info.cacheWritesPrice12h,
1502+
cacheWritesPrice1h:
1503+
tierInfo.cacheWritesPrice1h ??
1504+
(tierInfo.inputPrice !== undefined ? tierInfo.inputPrice * CACHE_WRITE_MULTIPLIER_1H : undefined) ??
1505+
info.cacheWritesPrice1h,
14931506
}
14941507
}
14951508

0 commit comments

Comments
 (0)