|
| 1 | +# 010 — design: `retryOn429` same-target wait-and-retry |
| 2 | + |
| 3 | +의존: `000_research.md` |
| 4 | + |
| 5 | +## Goal |
| 6 | + |
| 7 | +Provider-level opt-in knob: on HTTP 429, wait (upstream `Retry-After` or a fixed interval) |
| 8 | +and replay the identical request on the same key, up to `attempts` extra times, before the |
| 9 | +existing multi-key failover runs. Default off → zero behavior change for existing setups. |
| 10 | + |
| 11 | +## Surface |
| 12 | + |
| 13 | +```jsonc |
| 14 | +// ~/.opencodex/config.json → providers.<name> |
| 15 | +"retryOn429": { |
| 16 | + "enabled": true, // object presence also enables; false disables |
| 17 | + "attempts": 3, // extra replays after the first 429 (1..20) |
| 18 | + "intervalMs": 5000, // fixed wait when no usable Retry-After |
| 19 | + "maxIntervalMs": 60000, // cap for any single wait |
| 20 | + "respectRetryAfter": true // prefer the upstream Retry-After when parseable |
| 21 | +} |
| 22 | +``` |
| 23 | + |
| 24 | +## Implementation |
| 25 | + |
| 26 | +- `src/types.ts`: `RateLimitRetryPolicy` interface + `OcxProviderConfig.retryOn429`. |
| 27 | +- `src/config.ts`: zod schema entry (zod's default strip inside the object — an unknown key is |
| 28 | + dropped, never a config-rejecting error; the outer provider schema stays passthrough). |
| 29 | + Load-time degradation: one hand-edited invalid optional field (e.g. `attempts: 0`) is dropped |
| 30 | + with a warning instead of tripping the whole schema and hiding all providers behind a default |
| 31 | + config; misnamed keys (e.g. `attempt`) are warned about too; the management write boundary |
| 32 | + still rejects invalid policies. |
| 33 | +- `src/providers/key-failover.ts`: `rateLimitRetryPolicyFor` (normalize/default) and |
| 34 | + `rateLimitRetryDelayMs` (Retry-After seconds/HTTP-date → capped, else `intervalMs`), |
| 35 | + reusing the existing `parseRetryAfterMs` cooldown parser; the fixed fallback is capped at |
| 36 | + `maxIntervalMs` too, so a single wait never exceeds the cap. Fail closed: only `authMode: |
| 37 | + "key"` (or the documented omitted default for custom API-key providers) may use replays — |
| 38 | + OAuth/forward are never replayed on the same token, local runtimes have no remote key to |
| 39 | + preserve, and unknown values are rejected. `providerConfigSeed` preserves the registry auth |
| 40 | + kind (including `"local"`) so the gate survives the seed round-trip. |
| 41 | +- `src/usage/log.ts`: new `AttemptRecoveryKind` member `"rate-limit-429"`. |
| 42 | +- `src/server/responses/core.ts`: in the pre-stream recovery loop, BEFORE the multi-key |
| 43 | + failover `while`, wait then `rebuildAndRefetch("rate-limit-429")`. Abort during the wait |
| 44 | + cancels the client request (the unread 429 body is released first). The retry budget lives |
| 45 | + OUTSIDE the recovery loop, so a 413/401 replay that comes back 429 cannot re-arm a fresh |
| 46 | + budget — bounded to `attempts` per request. After attempts are exhausted the existing |
| 47 | + failover and error mapping run unchanged. The same wait-and-replay applies to the other |
| 48 | + key-auth surfaces that bypass that loop: |
| 49 | + - Responses passthrough wire (`openai-responses` key-auth gateways, e.g. the built-in |
| 50 | + DeepSeek preset) — pre-relay, before the forward-pool logic; |
| 51 | + - image/video bridge and web-search sidecar loops (`src/images/loop.ts`, |
| 52 | + `src/web-search/loop.ts`) — before their `on429` key rotation; |
| 53 | + - Anthropic terminal-guard continuations — before key/account failover. |
| 54 | + Bridge retries apply to HTTP adapters only: a custom transport that enters the |
| 55 | + `adapter.runTurn` branch (`src/images/loop.ts`) returns before the HTTP 429 retry loop and |
| 56 | + therefore does not receive the wait-and-replay policy. |
| 57 | + Every surface releases (and awaits the cancellation of) the unread 429 body BEFORE the |
| 58 | + backoff, records the `rate-limit-429` recovery kind on replay sends, and (bridges) clears the |
| 59 | + old response-header deadline before the wait and starts a fresh one afterward, re-checking |
| 60 | + client cancellation before telemetry and replay. |
| 61 | + Covers Responses, chat completions, and routed Claude messages (they all enter |
| 62 | + `handleResponses`). |
| 63 | + |
| 64 | +## Safety |
| 65 | + |
| 66 | +- Pre-stream only: a 429 arrives before any bytes are relayed, so replaying the string-body |
| 67 | + request is lossless (same invariant as the transient-5xx layer in `lib/upstream-retry.ts`). |
| 68 | +- Ordering: same-key retries run before failover, so "primary-first" users keep their key on |
| 69 | + rate-limit blips; failover still works after retries exhaust. |
| 70 | +- Retry-wait bound: the SLEEP component is at most `attempts × maxIntervalMs` (default |
| 71 | + 3 × 60 s = 180 s) when honoring upstream `Retry-After`; `attempts × intervalMs` |
| 72 | + (default 15 s) when `respectRetryAfter=false` or no header is present. Total request |
| 73 | + latency is higher: every attempt also consumes its own connect/response time (bounded by |
| 74 | + `connectTimeoutMs`), so the documented bound covers deliberate waits only. |
| 75 | +- Identical replay: rebuilds are deterministic for the same parsed request (same serialized |
| 76 | + body and auth headers); the passthrough/continuation/e2e tests assert byte-identical bodies |
| 77 | + and identical auth headers across replays, not just send counts. |
| 78 | +- Abort during the wait: the sleep is abort-aware — when the server observes the client |
| 79 | + disconnect (Bun propagates this asynchronously, observed 1–10 s), the wait is interrupted, |
| 80 | + the unread 429 body is released, and the request is cancelled with 499 before any replay. |
| 81 | + Because the propagation is async, a replay can still precede the cancel if the interval |
| 82 | + elapses first; that is bounded by the same `attempts` budget. Terminal continuations sleep |
| 83 | + on the upstream signal, so a body-cancel (SSE already streaming) aborts the wait too. |
| 84 | +- Concurrency: each request honors its own policy independently — no process-wide cooldown is |
| 85 | + shared between concurrent requests (unlike the Kiro 429 pattern). Upstream volume per |
| 86 | + request: same-key replays add at most `attempts` sends, then multi-key failover adds up to |
| 87 | + `poolKeys − 1` more (or Anthropic account rotations), so the combined bound is |
| 88 | + `attempts + poolKeys` sends (pool size = configured `apiKeyPool` length, fixed per request) — |
| 89 | + a storm multiplies by that factor per request, not by `attempts + 1`. |
| 90 | +- Header deadlines: the image/video and web-search bridge loops restart their response-header |
| 91 | + deadline after each deliberate wait, so backoffs never consume the connect budget and a |
| 92 | + rate-limit wait is never misattributed as a 504 header timeout. The old deadline is cleared |
| 93 | + BEFORE the sleep and client cancellation is re-checked after it, so 499 always wins over a |
| 94 | + stale-deadline edge. |
| 95 | +- Expired `Retry-After`: a valid HTTP-date already in the past retries immediately (same as |
| 96 | + numeric `Retry-After: 0`) instead of falling back to the fixed interval. |
| 97 | +- Recovery observability: every retry surface records the `rate-limit-429` recovery kind |
| 98 | + (normal loop, passthrough wire, image/video bridge, web-search sidecar, terminal |
| 99 | + continuations), so usage logs explain the extra sends. |
| 100 | +- Final 429 still carries `Retry-After` for clients that honor it (Claude Code). |
| 101 | + |
| 102 | +## Tests |
| 103 | + |
| 104 | +- `tests/rate-limit-retry.test.ts` — policy normalization (incl. OAuth/forward gating) + |
| 105 | + delay computation (seconds, HTTP-date, `0`, malformed, cap; deterministic) + abort during |
| 106 | + the wait: a directly-invoked `handleResponses` with a controlled abort signal returns 499, |
| 107 | + cancels the unread 429 body, and performs no further upstream sends (deterministic — no |
| 108 | + real-socket disconnect timing involved). |
| 109 | +- `tests/usage-log.test.ts` — the `rate-limit-429` recovery kind survives persisted usage logs. |
| 110 | +- `tests/server-rate-limit-retry-e2e.test.ts` — single-key replay to success, immediate |
| 111 | + passthrough without the knob, exhausted attempts surface 429, and retry-before-failover |
| 112 | + ordering with a 2-key pool, plus key-auth `openai-responses` passthrough replaying 429 on |
| 113 | + the same key. |
| 114 | +- `tests/terminal-guard-server.test.ts` — an Anthropic terminal-guard continuation that 429s |
| 115 | + is replayed on the same key before the error surfaces (3 upstream sends). |
| 116 | +- `tests/images/loop.test.ts` + `tests/web-search.test.ts` — the bridge loops replay 429 on |
| 117 | + the same key before `on429` rotation runs (same-key sends counted, rotations zero). |
0 commit comments