Repository navigation
Commit 607042b
authored
feat(providers): add opt-in transient-5xx retry with a shared total-send budget (#2981)
* fix(upstream-retry): make attempts one total-send budget across both retry layers
fetchWithTransientRetry forwarded its whole opts object, attempts included,
into every fetchWithResetRetry call, so the two layers multiplied: attempts:3
allowed 3 transient rounds each independently retrying 3 connection resets,
for up to 9 upstream sends, and attempts:10 allowed up to 100.
The existing doc comment already flagged the hazard and noted it was inert
because 'no caller passes it today'. The provider-level transientRetryOn5xx
policy in #2643/#2655 is the first caller that does, which would have turned
a latent note into live behavior — and multiplying load against an
already-failing provider is worse than not retrying at all.
A counted fetch wrapper now increments a shared send count before each
await, and only the remaining budget is passed inward. Recovery labels,
evidence wrapping, backoff, Retry-After, cancellation, slow-attempt return,
and terminal-body preservation are unchanged.
* test(upstream-retry): pin the total-send budget with mixed resets and 503s
* test(upstream-retry): assert the shared budget with correct terminal shapes
* feat(providers): add opt-in transient-5xx retry for key-auth openai-chat
Closes #2643.
providers.<name>.transientRetryOn5xx opts a provider into retrying pre-stream
transient statuses (500/502/503/504/520/521/522) across all three send
paths: the initial Responses request, the terminal-guard continuation, and
native /v1/chat/completions.
Disabled unless present; a bare {} opts in with defaults. Scope is key-auth
openai-chat only — the resolver checks the adapter explicitly rather than
letting any generic key-auth provider opt in, and auth mode follows the same
fail-closed rule as rateLimitRetryPolicyFor. The legacy direct-Google
exception is preserved unchanged.
attempts is a TOTAL send budget (1..10, default 3) covering both retry
layers, so 3 means at most three real upstream requests.
Both call sites extend the existing key-failover import, so no new module
edge reaches responses/core.ts.
* test(providers): cover the transient-retry resolver scope gates
* docs: describe the transientRetryOn5xx provider option
* fix(responses): keep 429-recovery refetches on the shared transient-retry budget
Review finding on 2107f64: after an initial 429 the Responses recovery
path enters rebuildAndRefetch, which called fetchWithHeaderTimeout directly.
An opted-in provider's transient-5xx policy therefore applied to the initial
send and to native chat but was silently bypassed on Responses recovery — a
429 that recovered into a retryable 503 got no retry at all.
Every send now goes through the same selection. The budget is request-scoped
rather than per-leg: fetchWithTransientRetry reports its consumed sends via
onSendsConsumed (in a finally, since it returns from five places and throws
from one), and the refetch receives only what is left. A request that
recovers several times therefore cannot multiply upstream load.
* fix(responses): hoist the transient-send budget to the shared request scope
* test(upstream-retry): prove the shared budget survives across request legs
* docs: localize the transientRetryOn5xx provider contract
Review finding: the option was documented only in the English reference. All
seven translated provider references now carry the same contract, including
that attempts is one request-scoped total-send budget shared with
connection-reset recovery and now also covering 429/account-recovery
refetches.1 parent f636763 commit 607042b
16 files changed
Lines changed: 317 additions & 20 deletions
File tree
- docs-site/src/content/docs
- fr/reference/configuration
- ja/reference/configuration
- ko/reference/configuration
- reference/configuration
- ru/reference/configuration
- tr/reference/configuration
- zh-cn/reference/configuration
- zh-tw/reference/configuration
- src
- lib
- providers
- server
- responses
- types
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
115 | 115 | | |
116 | 116 | | |
117 | 117 | | |
| 118 | + | |
118 | 119 | | |
119 | 120 | | |
120 | 121 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
| 105 | + | |
105 | 106 | | |
106 | 107 | | |
107 | 108 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
| 105 | + | |
105 | 106 | | |
106 | 107 | | |
107 | 108 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
119 | 119 | | |
120 | 120 | | |
121 | 121 | | |
| 122 | + | |
122 | 123 | | |
123 | 124 | | |
124 | 125 | | |
| |||
0 commit comments