Skip to content

Commit eb2ceb2

Browse files
authored
Merge pull request #865 from harryzhou2000/feat/429-same-target-retry
feat(proxy): opt-in same-target 429 wait-and-retry before key failover (#487)
2 parents 48338a1 + 70c80b4 commit eb2ceb2

48 files changed

Lines changed: 3038 additions & 130 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# 000 — same-target 429 retry: why the client cannot do it
2+
3+
## 요약
4+
5+
Codex turns die instantly when an upstream provider (e.g. BLSC with a single API key)
6+
returns HTTP 429. The proxy forwards the error with `Retry-After` (#514), but Codex
7+
never acts on it.
8+
9+
## Evidence
10+
11+
- **Upstream [openai/codex#30471](https://github.com/openai/codex/issues/30471) (open):**
12+
`codex-rs/codex-api/src/provider.rs` has a retry policy with an explicit `retry_429` flag,
13+
and the endpoint configs set it to `false`; HTTP 429 is only mapped to `UsageLimitReached`
14+
when the body matches the Codex usage-limit shape, otherwise it falls through to a generic
15+
transport/API error and surfaces as the misleading "exceeded retry limit" message. The
16+
suggested fix is to reword the error, deliberately preserving `retry_429=false` — there is
17+
no user-facing knob to enable client-side 429 retry.
18+
- **opencodex issue #487 (auto-closed as `not_planned` for missing detail):** the author
19+
requested exactly this feature and noted "The Codex client itself does not retry 429
20+
(upstream: openai/codex#30471) — it only retries 5xx — so a proxy-side retry would fill a
21+
real gap."
22+
- **opencodex PR #514 (merged):** attaches a client `Retry-After` (upstream header → message
23+
hint → default 2) across Responses, chat completions, Claude messages, and passthrough
24+
paths. Its own test plan lists the Codex recovery check as *unverified*. Claude Code honors
25+
`Retry-After` and absorbs 429s (#507); Codex does not.
26+
27+
## Current proxy behavior (v2.8.0 / dev)
28+
29+
- `src/server/responses/core.ts` recovery loop: the only 429 retry is multi-key failover
30+
(`hasKeyPoolFailover` requires ≥2 keys in `apiKeyPool`). Single-key pools no-op, and the
31+
429 falls through to `rate_limit_error` with `Retry-After`.
32+
- `src/server/chat-completions.ts` and the routed Claude path reuse `handleResponses`, so one
33+
insertion point covers all three inbound surfaces.
34+
35+
## Conclusion
36+
37+
Client-side 429 retry does not exist and is not planned. The fix must live in the proxy:
38+
an opt-in wait-and-retry that replays the identical pre-stream request on the same key before
39+
any failover.
Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
# 010 — design: `retryOn429` same-target wait-and-retry
2+
3+
의존: `000_research.md`
4+
5+
## Goal
6+
7+
Provider-level opt-in knob: on HTTP 429, wait (upstream `Retry-After` or a fixed interval)
8+
and replay the identical request on the same key, up to `attempts` extra times, before the
9+
existing multi-key failover runs. Default off → zero behavior change for existing setups.
10+
11+
## Surface
12+
13+
```jsonc
14+
// ~/.opencodex/config.json → providers.<name>
15+
"retryOn429": {
16+
"enabled": true, // object presence also enables; false disables
17+
"attempts": 3, // extra replays after the first 429 (1..20)
18+
"intervalMs": 5000, // fixed wait when no usable Retry-After
19+
"maxIntervalMs": 60000, // cap for any single wait
20+
"respectRetryAfter": true // prefer the upstream Retry-After when parseable
21+
}
22+
```
23+
24+
## Implementation
25+
26+
- `src/types.ts`: `RateLimitRetryPolicy` interface + `OcxProviderConfig.retryOn429`.
27+
- `src/config.ts`: zod schema entry (zod's default strip inside the object — an unknown key is
28+
dropped, never a config-rejecting error; the outer provider schema stays passthrough).
29+
Load-time degradation: one hand-edited invalid optional field (e.g. `attempts: 0`) is dropped
30+
with a warning instead of tripping the whole schema and hiding all providers behind a default
31+
config; misnamed keys (e.g. `attempt`) are warned about too; the management write boundary
32+
still rejects invalid policies.
33+
- `src/providers/key-failover.ts`: `rateLimitRetryPolicyFor` (normalize/default) and
34+
`rateLimitRetryDelayMs` (Retry-After seconds/HTTP-date → capped, else `intervalMs`),
35+
reusing the existing `parseRetryAfterMs` cooldown parser; the fixed fallback is capped at
36+
`maxIntervalMs` too, so a single wait never exceeds the cap. Fail closed: only `authMode:
37+
"key"` (or the documented omitted default for custom API-key providers) may use replays —
38+
OAuth/forward are never replayed on the same token, local runtimes have no remote key to
39+
preserve, and unknown values are rejected. `providerConfigSeed` preserves the registry auth
40+
kind (including `"local"`) so the gate survives the seed round-trip.
41+
- `src/usage/log.ts`: new `AttemptRecoveryKind` member `"rate-limit-429"`.
42+
- `src/server/responses/core.ts`: in the pre-stream recovery loop, BEFORE the multi-key
43+
failover `while`, wait then `rebuildAndRefetch("rate-limit-429")`. Abort during the wait
44+
cancels the client request (the unread 429 body is released first). The retry budget lives
45+
OUTSIDE the recovery loop, so a 413/401 replay that comes back 429 cannot re-arm a fresh
46+
budget — bounded to `attempts` per request. After attempts are exhausted the existing
47+
failover and error mapping run unchanged. The same wait-and-replay applies to the other
48+
key-auth surfaces that bypass that loop:
49+
- Responses passthrough wire (`openai-responses` key-auth gateways, e.g. the built-in
50+
DeepSeek preset) — pre-relay, before the forward-pool logic;
51+
- image/video bridge and web-search sidecar loops (`src/images/loop.ts`,
52+
`src/web-search/loop.ts`) — before their `on429` key rotation;
53+
- Anthropic terminal-guard continuations — before key/account failover.
54+
Bridge retries apply to HTTP adapters only: a custom transport that enters the
55+
`adapter.runTurn` branch (`src/images/loop.ts`) returns before the HTTP 429 retry loop and
56+
therefore does not receive the wait-and-replay policy.
57+
Every surface releases (and awaits the cancellation of) the unread 429 body BEFORE the
58+
backoff, records the `rate-limit-429` recovery kind on replay sends, and (bridges) clears the
59+
old response-header deadline before the wait and starts a fresh one afterward, re-checking
60+
client cancellation before telemetry and replay.
61+
Covers Responses, chat completions, and routed Claude messages (they all enter
62+
`handleResponses`).
63+
64+
## Safety
65+
66+
- Pre-stream only: a 429 arrives before any bytes are relayed, so replaying the string-body
67+
request is lossless (same invariant as the transient-5xx layer in `lib/upstream-retry.ts`).
68+
- Ordering: same-key retries run before failover, so "primary-first" users keep their key on
69+
rate-limit blips; failover still works after retries exhaust.
70+
- Retry-wait bound: the SLEEP component is at most `attempts × maxIntervalMs` (default
71+
3 × 60 s = 180 s) when honoring upstream `Retry-After`; `attempts × intervalMs`
72+
(default 15 s) when `respectRetryAfter=false` or no header is present. Total request
73+
latency is higher: every attempt also consumes its own connect/response time (bounded by
74+
`connectTimeoutMs`), so the documented bound covers deliberate waits only.
75+
- Identical replay: rebuilds are deterministic for the same parsed request (same serialized
76+
body and auth headers); the passthrough/continuation/e2e tests assert byte-identical bodies
77+
and identical auth headers across replays, not just send counts.
78+
- Abort during the wait: the sleep is abort-aware — when the server observes the client
79+
disconnect (Bun propagates this asynchronously, observed 1–10 s), the wait is interrupted,
80+
the unread 429 body is released, and the request is cancelled with 499 before any replay.
81+
Because the propagation is async, a replay can still precede the cancel if the interval
82+
elapses first; that is bounded by the same `attempts` budget. Terminal continuations sleep
83+
on the upstream signal, so a body-cancel (SSE already streaming) aborts the wait too.
84+
- Concurrency: each request honors its own policy independently — no process-wide cooldown is
85+
shared between concurrent requests (unlike the Kiro 429 pattern). Upstream volume per
86+
request: same-key replays add at most `attempts` sends, then multi-key failover adds up to
87+
`poolKeys − 1` more (or Anthropic account rotations), so the combined bound is
88+
`attempts + poolKeys` sends (pool size = configured `apiKeyPool` length, fixed per request) —
89+
a storm multiplies by that factor per request, not by `attempts + 1`.
90+
- Header deadlines: the image/video and web-search bridge loops restart their response-header
91+
deadline after each deliberate wait, so backoffs never consume the connect budget and a
92+
rate-limit wait is never misattributed as a 504 header timeout. The old deadline is cleared
93+
BEFORE the sleep and client cancellation is re-checked after it, so 499 always wins over a
94+
stale-deadline edge.
95+
- Expired `Retry-After`: a valid HTTP-date already in the past retries immediately (same as
96+
numeric `Retry-After: 0`) instead of falling back to the fixed interval.
97+
- Recovery observability: every retry surface records the `rate-limit-429` recovery kind
98+
(normal loop, passthrough wire, image/video bridge, web-search sidecar, terminal
99+
continuations), so usage logs explain the extra sends.
100+
- Final 429 still carries `Retry-After` for clients that honor it (Claude Code).
101+
102+
## Tests
103+
104+
- `tests/rate-limit-retry.test.ts` — policy normalization (incl. OAuth/forward gating) +
105+
delay computation (seconds, HTTP-date, `0`, malformed, cap; deterministic) + abort during
106+
the wait: a directly-invoked `handleResponses` with a controlled abort signal returns 499,
107+
cancels the unread 429 body, and performs no further upstream sends (deterministic — no
108+
real-socket disconnect timing involved).
109+
- `tests/usage-log.test.ts` — the `rate-limit-429` recovery kind survives persisted usage logs.
110+
- `tests/server-rate-limit-retry-e2e.test.ts` — single-key replay to success, immediate
111+
passthrough without the knob, exhausted attempts surface 429, and retry-before-failover
112+
ordering with a 2-key pool, plus key-auth `openai-responses` passthrough replaying 429 on
113+
the same key.
114+
- `tests/terminal-guard-server.test.ts` — an Anthropic terminal-guard continuation that 429s
115+
is replayed on the same key before the error surfaces (3 upstream sends).
116+
- `tests/images/loop.test.ts` + `tests/web-search.test.ts` — the bridge loops replay 429 on
117+
the same key before `on429` rotation runs (same-key sends counted, rotations zero).

‎docs-site/src/content/docs/guides/providers.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,12 @@ labels local presets separately; those normally omit both `authMode` and `apiKey
5959
| `forward` | Relays **your incoming Codex auth headers** verbatim to the provider — no key stored. This is the ChatGPT-login passthrough. | OpenAI (`openai-responses` adapter). |
6060
| `oauth` | Resolves a stored OAuth access token (auto-refreshed before expiry) and uses it as the bearer key. | xAI, Anthropic, Kimi, Kiro, Google Antigravity, Cursor, GitHub Copilot. |
6161

62+
The [`retryOn429`](/reference/configuration/) same-key 429 replay applies only to API-key
63+
providers (`authMode: "key"`). OAuth, forward, and local presets are excluded — their
64+
credentials must never be replayed on the same token, and local runtimes have no remote key to
65+
preserve. It is opt-in: when the option is absent the feature is off; object presence enables
66+
it unless `enabled: false`.
67+
6268
## 1. ChatGPT login (forward / passthrough)
6369

6470
The `openai` provider needs **no API key**. Direct forwards credentials from your existing

‎docs-site/src/content/docs/ja/guides/providers.md‎

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ description: opencodex が LLM プロバイダーを認証し通信するすべ
99
## OpenAI アカウントモード
1010

1111
| プロバイダー ID | 用途 | 認証情報/アカウットルール |
12-
--- | --- | --- |
12+
| --- | --- | --- |
1313
| `openai` | Codex ログイン | Pool(デフォルト)はメイン + 追加アカウントを選び、Direct は現在の caller/メインログインのみを使います。 |
1414
| `openai-apikey` | OpenAI API | 設定された API キー/キープールのみを使い、Codex アカウントは読みません。 |
1515

@@ -49,11 +49,16 @@ Codex login を Pool モードで使うと、Providers の概要には任意の
4949
ローカルプリセットを別に分類します。ローカルプリセットでは通常 `authMode` と `apiKey` を両方使いません。
5050

5151
| `authMode` | 認証方式 | 用途 |
52-
--- | --- | --- |
52+
| --- | --- | --- |
5353
| `key` | API キーを送信します(`Authorization: Bearer …`、またはアダプターにより `x-api-key` / `api-key`)。キーはリテラルまたは `${ENV_VAR}` 参照です。 | 大半のプロバイダー。 |
5454
| `forward` | **受け取った Codex 認証ヘッダーを**プロバイダーにそのまま中継します — キーを保存しません。ChatGPT ログインのパススルーです。 | OpenAI(`openai-responses` アダプター)。 |
5555
| `oauth` | 保存された OAuth アクセストークンを読み込み bearer キーとして使い、期限切れ前に自動更新します。 | xAI、Anthropic、Kimi、Kiro、Google Antigravity、Cursor。 |
5656

57+
[`retryOn429`](/ja/reference/configuration/)(同一キーでの 429 リトライ)は API キー プロバイダー
58+
(`authMode: "key"`)のみに適用されます。OAuth・forward・ローカル プリセットは除外されます —
59+
同じトークンを再送すべきではなく、ローカルランタイムには保存すべきリモートキーがありません。
60+
オプトインです: オプションが無ければ無効、オブジェクトがあれば `enabled: false` でない限り有効です。
61+
5762
## 1. ChatGPT ログイン(forward / パススルー)
5863

5964
デフォルトプロバイダーは**API キー不要**です。既存の `codex login` の認証情報を OpenAI Responses バックエンドに
@@ -157,7 +162,7 @@ Cline IDE/CLI のみで API からは使えません。`minimax/minimax-m2.5`
157162
無料試用モデルとして文書化されています。
158163

159164
| プロバイダー | ベース URL |
160-
--- | --- |
165+
| --- | --- |
161166
| **OpenAI (API キー)** | `https://api.openai.com/v1` |
162167
| **Anthropic (API キー)** | `https://api.anthropic.com` |
163168
| **OpenRouter** | `https://openrouter.ai/api/v1` |
@@ -266,7 +271,7 @@ OAuth、API キープールを確認・切り替えできます。完全なコ
266271
Sol/Terra/Luna をフォールバックリストに入れています。
267272

268273
| Codex 経路 | 事前登録されたモデル ID | Codex に表示されるコンテキスト |
269-
--- | --- | --- |
274+
| --- | --- | --- |
270275
| Codex ログイン(Pool または Direct) | `gpt-5.6-*` | 372,000 |
271276
| OpenAI (API キー) | `openai-apikey/gpt-5.6-*` と `*-pro` | 1,050,000 (max input 922,000) |
272277
| OpenRouter | `openrouter/openai/gpt-5.6-sol`、`openrouter/openai/gpt-5.6-terra`、`openrouter/openai/gpt-5.6-luna` | 1,050,000 |
@@ -288,6 +293,15 @@ Amazon Bedrock ネイティブ API のような、これらの実装のいずれ
288293
**サブスクリプショントークン**(通常の API キーではない)で認証します。**Cloudflare AI
289294
Gateway** は URL にアカウント + ゲートウェイ ID を埋める必要があります。
290295

296+
Copilot は混在 wire カタログを提供します。GPT-5 系モデル(`gpt-5.3-codex`、`gpt-5.4`、
297+
`gpt-5.4-mini`、`gpt-5.5`、`gpt-5.6-luna`、`gpt-5.6-sol`、`gpt-5.6-terra`)はエージェント
298+
通信の `/chat/completions` を拒否するため、opencodex はこれらのモデルを組み込みデフォルトで
299+
Responses API 経由にルーティングし、他の Copilot モデルはすべて chat completions のままです。
300+
優先順位は次のとおりです: ハード wire ピン → 明示的な
301+
[`modelAdapters`](/ja/reference/configuration/providers/) エントリ → レジストリのデフォルト →
302+
プロバイダー全体の adapter。組み込みデフォルトのないモデル(例: `gpt-5.4-nano`)を Responses
303+
に移すには、`"modelAdapters": { "gpt-5.4-nano": "openai-responses" }` を設定してください。
304+
291305
Cursor は別の実験的アダプターとして追跡します。`adapter: "cursor"` は `ocx init` とダッシュボード Add
292306
Provider ピッカーに実験的 local config 項目として表示され、Cursor の静的フォールバックモデルカタログ
293307
メタデータを保存します。Cursor アクセストークンを設定すると opencodex は Cursor ライブ HTTP/2 トランスポートを
@@ -321,7 +335,7 @@ Ollama の `:size` タグに寛容なので `gpt-oss` は `gpt-oss:120b` と `gp
321335
opencodex をローカルの OpenAI 互換サーバーに向けてください — 通常は空キーで使います:
322336

323337
| プロバイダー | ベース URL |
324-
--- | --- |
338+
| --- | --- |
325339
| Ollama (local) | `http://localhost:11434/v1` |
326340
| vLLM | `http://localhost:8000/v1` |
327341
| LM Studio | `http://localhost:1234/v1` |

‎docs-site/src/content/docs/ja/reference/adapters.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,11 @@ interface ProviderAdapter {
4646
**対象:** OpenAI **Responses API**。**`passthrough: true`** — 元のリクエスト本文をそのまま渡し、レスポンスを **変換せずに** ストリーミングします。
4747
**認証:** `forward`(呼び出し元ヘッダー中継)または `key`。
4848

49+
`key` 認証では、[`retryOn429`](/ja/reference/configuration/) もここに適用されます: プリストリームの
50+
429 は、翻訳された `openai-chat` / Anthropic リクエスト経路と同様に、他の処理やフェイルオーバーに
51+
先立って、同じキーで同一リクエストを待機して再送します。カスタム `runTurn` トランスポートは
52+
HTTP リトライ ループの対象外です。
53+
4954
- `forward` URL → `{baseUrl}/responses`。`key` provider はデフォルトで従来の `{baseUrl}/v1/responses` 構築を使います。
5055
- `key` provider は検証済みの相対 `responsesPath` を設定できます。adapter は `baseUrl` 末尾の `/` を 1 つ除き、`{trimmedBaseUrl}{responsesPath}` に送信します。Ark Agent Plan では `baseUrl: "https://ark.cn-beijing.volces.com/api/plan/v3"` と `responsesPath: "/responses"` を使います。
5156
- `forward` モードでは安全なヘッダー許可リスト(`FORWARD_HEADERS`)だけを中継します。authorization、ChatGPT account id、OpenAI beta/originator/session ヘッダーが対象です。この ChatGPT ログイン経路は [サイドカー](/ja/guides/sidecars/) にも使われます。

0 commit comments

Comments
 (0)