Repository navigation
Commit 7a06eac
committed
feat(bedrock): emit the opt-in cache TTL only for 1h-capable Converse models
What:
- Add `promptCacheTtlSupport: ["5m", "1h"]` to the 13 Bedrock catalog rows whose
AWS "Supported TTL" table entry lists 1h. The remaining rows stay absent, which
is the fail-closed default.
- Thread one resolved TTL through `CacheStrategyConfig.cacheTtl` into
`CacheStrategy.createCachePoint`, and make that factory public so the provider's
message-level emitter reuses it instead of building its own literal.
- Resolve the TTL once in `createMessage` via the shared `resolveCacheTtl` helper
and pass it to `convertToBedrockConverseMessages`, so both cache-point emitters
(system/message blocks from the strategy, and the `messages[i].content` literal
that used to hardcode `{ cachePoint: { type: "default" } }`) carry the same value.
- Read the 1h share of `usage.cacheDetails` on the Converse stream and report it as
`cacheWrite1hTokens`, clamped to the reported aggregate.
Why:
ADR-0006 PR-C7, the last request-side step of the TTL program. C5a (#2260) made a
1h cache write bill at 2x instead of the 5m rate and C5b (#2264) added the knob plus
the capability field, but no Bedrock request could carry a TTL, so the knob was
inert on the one provider that hard-fails on a wrong one.
Impact:
No behavior change by default: an unset or "5m" knob resolves to `undefined` and
every cache point stays the byte-identical `{ type: "default" }`. A 1h request is
emitted only when the resolved base model id is exactly listed, so a custom ARN, an
inference profile naming an unlisted model, or any Nova row keeps today's bytes.
Bedrock rejects an explicit ttl for a 5m-only model with a ValidationException that
kills the turn, so the gate is capability-based and never error-driven: no
catch-400-retry path exists. One TTL per request also makes the AWS
"longer-before-shorter across tools -> system -> messages" ordering hazard
unrepresentable rather than merely avoided. Cache-point placement is untouched.1 parent 65ac862 commit 7a06eac
6 files changed
Lines changed: 691 additions & 7 deletions
File tree
- packages/types/src/providers
- src/api
- providers
- __tests__
- transform/cache-strategy
- __tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
15 | 23 | | |
16 | 24 | | |
17 | 25 | | |
18 | 26 | | |
19 | 27 | | |
20 | 28 | | |
| 29 | + | |
21 | 30 | | |
22 | 31 | | |
23 | 32 | | |
| |||
33 | 42 | | |
34 | 43 | | |
35 | 44 | | |
| 45 | + | |
36 | 46 | | |
37 | 47 | | |
38 | 48 | | |
| |||
60 | 70 | | |
61 | 71 | | |
62 | 72 | | |
| 73 | + | |
63 | 74 | | |
64 | 75 | | |
65 | 76 | | |
| |||
173 | 184 | | |
174 | 185 | | |
175 | 186 | | |
| 187 | + | |
176 | 188 | | |
177 | 189 | | |
178 | 190 | | |
| |||
199 | 211 | | |
200 | 212 | | |
201 | 213 | | |
| 214 | + | |
202 | 215 | | |
203 | 216 | | |
204 | 217 | | |
| |||
225 | 238 | | |
226 | 239 | | |
227 | 240 | | |
| 241 | + | |
228 | 242 | | |
229 | 243 | | |
230 | 244 | | |
| |||
254 | 268 | | |
255 | 269 | | |
256 | 270 | | |
| 271 | + | |
257 | 272 | | |
258 | 273 | | |
259 | 274 | | |
| |||
275 | 290 | | |
276 | 291 | | |
277 | 292 | | |
| 293 | + | |
278 | 294 | | |
279 | 295 | | |
280 | 296 | | |
| |||
300 | 316 | | |
301 | 317 | | |
302 | 318 | | |
| 319 | + | |
303 | 320 | | |
304 | 321 | | |
305 | 322 | | |
| |||
326 | 343 | | |
327 | 344 | | |
328 | 345 | | |
| 346 | + | |
329 | 347 | | |
330 | 348 | | |
331 | 349 | | |
| |||
345 | 363 | | |
346 | 364 | | |
347 | 365 | | |
| 366 | + | |
348 | 367 | | |
349 | 368 | | |
350 | 369 | | |
| |||
364 | 383 | | |
365 | 384 | | |
366 | 385 | | |
| 386 | + | |
367 | 387 | | |
368 | 388 | | |
369 | 389 | | |
| |||
437 | 457 | | |
438 | 458 | | |
439 | 459 | | |
| 460 | + | |
440 | 461 | | |
441 | 462 | | |
442 | 463 | | |
| |||
0 commit comments