Skip to content

Commit e247a0d

Browse files
committed
fix(streaming): prevent Darwin rewrite stalls
1 parent 6a7351b commit e247a0d

14 files changed

Lines changed: 115 additions & 56 deletions

File tree

‎docs-site/src/content/docs/ja/troubleshooting/windows-memory.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,6 @@ Windows では、#32111 クラッシュを回避するために、opencodex は
4343
未検証の領域 — 私たちがテストしていないランタイムで opencodex を実行しています。自己責任で。サービスのインストールにとって重要: オーバーライドは、サービスの開始時ではなく、**サービス アーティファクトの生成時に**読み込まれます。環境変数を設定し、同じシェルから `ocx service install` を再実行すると、パスが永続サービス定義に組み込まれます。 env を設定するだけでは、すでにインストールされているサービスには何も影響しません。
4444

4545
3. **`streamMode: "eager-relay"` を使用して有界リレーにオプトインします。** 2 つの方法:
46-
`config.json` を編集する (`"streamMode": "eager-relay"` を追加する) か、管理 API を呼び出します。`PUT /api/settings` と `{"streamMode":"eager-relay"}` は、再起動せずに新しいターンに適用されます。 **クラッシュのリスク警告:** Bun 1.3.14 では、#32111 の影響を受けるストリーム形状が使用されており、ストリームの途中でプロセスがクラッシュする可能性があります (Windows に限らず、どの OS でも)。サービス マネージャーはサービスを再起動しますが、実行中のリクエストは失敗します。 `"legacy-tee"` は現在のデフォルトを固定します。 Windows では、`"auto"` (デフォルト) によりランタイム ゲートが決定します。 macOS では、`"auto"` は常に T 上にあります。明示的な `"eager-relay"` はオプトインです。
46+
`config.json` を編集する (`"streamMode": "eager-relay"` を追加する) か、管理 API を呼び出します。`PUT /api/settings` と `{"streamMode":"eager-relay"}` は、再起動せずに新しいターンに適用されます。 **クラッシュのリスク警告:** Bun 1.3.14 では、#32111 の影響を受けるストリーム形状が使用されており、ストリームの途中でプロセスがクラッシュする可能性があります (Windows に限らず、どの OS でも)。サービス マネージャーはサービスを再起動しますが、実行中のリクエストは失敗します。通常のストリームでは `"legacy-tee"` が現在のデフォルトを固定します。ただし、クライアント向け payload rewrite が必要なリクエストは、tee/rewrite チェーンの停止を避けるため Windows と macOS で自動的に有界 single-reader relay を使い、この安全上の例外は `streamMode` では上書きできません。Windows の通常ストリームでは `"auto"` (デフォルト) がランタイムゲートに従います。macOS の通常の `"auto"` トラフィックは tee のままで、明示的な `"eager-relay"` がオプトインです。
4747

4848
これらのいずれかを実際の Windows ワークロードで試した場合は、[#314](https://github.com/lidge-jun/opencodex/issues/314) の `ocx doctor` メモリ セクションの前後を報告してください。これがまさにこの軽減策が待っている検証です。

‎docs-site/src/content/docs/ko/troubleshooting/windows-memory.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,6 @@ Windows에서는 opencodex가 #32111 충돌을 피하기 위해 스트리밍 응
3636

3737
2. **`OPENCODEX_BUN_PATH`로 신뢰하는 Bun 런타임을 사용합니다.** 이 경로는 검증되지 않은 영역입니다. opencodex를 아직 테스트하지 않은 런타임에서 실행하는 것이므로, 위험은 사용자에게 있습니다. 서비스 설치에서 특히 중요한 점은 이 override가 서비스 시작 시가 아니라 **서비스 아티팩트를 생성할 때** 읽힌다는 것입니다. 환경 변수를 설정한 뒤, 같은 셸에서 `ocx service install`을 다시 실행해야 경로가 영구적인 서비스 정의에 반영됩니다. 환경 변수만 설정하면 이미 설치된 서비스에는 아무 영향이 없습니다.
3838

39-
3. **`streamMode: "eager-relay"`로 bounded relay를 opt-in합니다.** 방법은 두 가지입니다. `config.json`을 수정해 `"streamMode": "eager-relay"`를 추가하거나, 관리 API에 `PUT /api/settings`와 `{"streamMode":"eager-relay"}`를 보내 새 턴에 재시작 없이 적용합니다. **충돌 위험 경고:** Bun 1.3.14에서는 이 방식이 #32111의 영향을 받는 스트림 형태를 사용하므로, 어떤 OS에서든 스트림 중간에 프로세스가 충돌할 수 있습니다. 서비스 관리자가 다시 시작하겠지만, 진행 중이던 요청은 실패합니다. `"legacy-tee"`는 현재 기본값을 고정합니다. Windows에서는 `"auto"`(기본값)가 런타임 게이트 판단을 따르게 합니다. macOS에서는 `"auto"`가 항상 tee를 유지하고, 명시적 `"eager-relay"`만 opt-in입니다.
39+
3. **`streamMode: "eager-relay"`로 bounded relay를 opt-in합니다.** 방법은 두 가지입니다. `config.json`을 수정해 `"streamMode": "eager-relay"`를 추가하거나, 관리 API에 `PUT /api/settings`와 `{"streamMode":"eager-relay"}`를 보내 새 턴에 재시작 없이 적용합니다. **충돌 위험 경고:** Bun 1.3.14에서는 이 방식이 #32111의 영향을 받는 스트림 형태를 사용하므로, 어떤 OS에서든 스트림 중간에 프로세스가 충돌할 수 있습니다. 서비스 관리자가 다시 시작하겠지만, 진행 중이던 요청은 실패합니다. 일반 스트림에서는 `"legacy-tee"`가 현재 기본 경로를 고정합니다. 다만 클라이언트용 payload rewrite가 필요한 요청은 tee/rewrite 체인이 멈출 수 있으므로 Windows와 macOS에서 자동으로 bounded single-reader relay를 사용하며, `streamMode`는 이 전송 안전 예외를 덮어쓰지 않습니다. Windows의 일반 스트림에서는 `"auto"`(기본값)가 런타임 게이트 판단을 따릅니다. macOS의 일반 `"auto"` 트래픽은 tee를 유지하고, 명시적 `"eager-relay"`만 opt-in입니다.
4040

4141
이 중 어떤 방법이든 실제 Windows 워크로드에 적용해 보셨다면, 변경 전후의 `ocx doctor` 메모리 섹션을 [#314](https://github.com/lidge-jun/opencodex/issues/314)에 남겨 주세요. 이것이 바로 이 완화책이 기다리고 있는 검증입니다.

‎docs-site/src/content/docs/ru/troubleshooting/windows-memory.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -83,9 +83,12 @@ opencodex поставляет рантайм Bun (сейчас это **1.3.14*
8383
перезапуска. **Предупреждение о риске crash:** на Bun 1.3.14 это использует форму streaming,
8484
затронутую #32111, и процесс может упасть прямо посреди потока (на любой ОС, не только на
8585
Windows). Service manager перезапустит его, но все запросы в полёте потерпят неудачу.
86-
`"legacy-tee"` жёстко фиксирует текущий путь по умолчанию. На Windows `"auto"` (по умолчанию)
87-
позволяет рантайму выбрать путь самому. На macOS `"auto"` всегда остаётся на tee; явный
88-
`"eager-relay"` — это opt-in.
86+
Для обычных потоков `"legacy-tee"` жёстко фиксирует текущий путь по умолчанию.
87+
Запросы, которым нужен клиентский payload rewrite, автоматически используют bounded
88+
single-reader relay на Windows и macOS, потому что цепочка tee/rewrite может зависнуть;
89+
`streamMode` не отменяет это исключение безопасности транспорта. На Windows `"auto"`
90+
(по умолчанию) передаёт выбор runtime-gate для обычных потоков, а на macOS обычный
91+
`"auto"` остаётся на tee; явный `"eager-relay"` — это opt-in.
8992

9093
Если вы попробуете любой из этих вариантов на реальной Windows-нагрузке, пожалуйста, пришлите
9194
разделы памяти из `ocx doctor` до и после в

‎docs-site/src/content/docs/troubleshooting/windows-memory.md‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,9 +91,13 @@ restart it.
9191
applies to new turns without a restart. **Crash risk warning:** on Bun
9292
1.3.14 this uses the stream shape affected by #32111, which can crash the
9393
process mid-stream (on any OS, not just Windows). The service manager will
94-
restart it, but in-flight requests fail. `"legacy-tee"` pins the current
95-
default. On Windows, `"auto"` (default) lets the runtime gate decide. On
96-
macOS, `"auto"` always stays on tee; explicit `"eager-relay"` is the opt-in.
94+
restart it, but in-flight requests fail. For ordinary streams,
95+
`"legacy-tee"` pins the current default. Requests that need a client-facing
96+
payload rewrite use the bounded single-reader relay automatically on Windows
97+
and macOS because the tee/rewrite chain can stall; `streamMode` does not
98+
override that transport-safety exception. On Windows, `"auto"` (default)
99+
lets the runtime gate decide for ordinary streams. On macOS, ordinary
100+
`"auto"` traffic stays on tee; explicit `"eager-relay"` is the opt-in.
97101

98102
If you try any of these on a real Windows workload, please report the before
99103
and after `ocx doctor` memory sections on

‎docs-site/src/content/docs/zh-cn/troubleshooting/windows-memory.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,6 @@ opencodex 打包了 Bun 运行时(当前为 **1.3.14**)。这类内存增长
3636

3737
2. **通过 `OPENCODEX_BUN_PATH` 运行你信任的 Bun 运行时。** 这属于未验证区域,你是在一个我们没有测试过的运行时上运行 opencodex,风险自负。对服务安装而言,这个覆盖值是在生成服务产物时读取的,而不是在服务启动时读取的。先设置环境变量,然后在同一个 shell 中重新运行 `ocx service install`,这样路径才会被写入持久化的服务定义。只设置环境变量对已经安装好的服务没有任何作用。
3838

39-
3. **通过 `streamMode: "eager-relay"` 显式启用有界中继。** 有两种方式:编辑 `config.json`(添加 `"streamMode": "eager-relay"`),或调用管理 API - `PUT /api/settings` 携带 `{"streamMode":"eager-relay"}`,即可对新轮次生效,无需重启。**崩溃风险警告:** 在 Bun 1.3.14 上,这会使用受 #32111 影响的流形态,可能在流中途使进程崩溃(任何操作系统都会受影响,不只是 Windows)。服务管理器会把它重启,但正在进行的请求会失败。`"legacy-tee"` 会固定在当前默认路径。Windows 上,`"auto"`(默认值)会交给运行时门控决定。macOS 上,`"auto"` 始终保持 tee;显式 `"eager-relay"` 才是显式启用选项。
39+
3. **通过 `streamMode: "eager-relay"` 显式启用有界中继。** 有两种方式:编辑 `config.json`(添加 `"streamMode": "eager-relay"`),或调用管理 API - `PUT /api/settings` 携带 `{"streamMode":"eager-relay"}`,即可对新轮次生效,无需重启。**崩溃风险警告:** 在 Bun 1.3.14 上,这会使用受 #32111 影响的流形态,可能在流中途使进程崩溃(任何操作系统都会受影响,不只是 Windows)。服务管理器会把它重启,但正在进行的请求会失败。对于普通流,`"legacy-tee"` 会固定当前默认路径;需要面向客户端做 payload rewrite 的请求会在 Windows 和 macOS 上自动使用有界单读者中继,因为 tee/rewrite 链可能卡住,`streamMode` 不会覆盖这个传输安全例外。Windows 上,普通流的 `"auto"`(默认值)会交给运行时门控决定。macOS 上,普通 `"auto"` 流量仍保持 tee;显式 `"eager-relay"` 才会启用有界中继。
4040

4141
如果你在真实的 Windows 工作负载上尝试这些方案,请把变更前后 `ocx doctor` 的内存部分发到 [#314](https://github.com/lidge-jun/opencodex/issues/314)——这正是这个缓解措施在等待的验证。

‎src/lib/bun-stream-caps.ts‎

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,10 @@
88
* "known-bad" until a bundle-bump commit sets it. Windows no-rewrite traffic
99
* follows this runtime/config decision. Darwin no-rewrite traffic stays on tee
1010
* for `auto` regardless of runtime capability and reaches eager relay only via
11-
* explicit `streamMode: "eager-relay"` opt-in (see
12-
* devlog/_plan/260731_macos_rss_retention/100_darwin_eager_optin.md).
11+
* explicit `streamMode: "eager-relay"` opt-in. On Windows and Darwin, traffic
12+
* that needs a client payload rewrite always uses the eager single-reader path:
13+
* the tee()+JS-pull rewrite chain can stall before delivering any client frames
14+
* (see #864 and devlog/_plan/260731_macos_rss_retention/100_darwin_eager_optin.md).
1315
*
1416
* Prerelease conservatism: a version carrying a prerelease suffix (e.g.
1517
* `1.4.0-canary.3`) is NEVER treated as fixed even when its numeric triple
@@ -93,7 +95,8 @@ export function decideEagerRelay(
9395
* Windows preserves the decision for no-rewrite traffic. Darwin permits only
9496
* explicit config opt-in; `auto` remains tee even on a future fixed runtime.
9597
* Returns the normalized effective decision, or null when platform policy,
96-
* rewrite needs, or a Darwin non-config-eager mode selects tee.
98+
* rewrite needs, or a Darwin non-config-eager mode selects tee. Rewrite traffic
99+
* is handled separately by `requiresEagerRewriteRelay`.
97100
*/
98101
export function selectEagerPath(
99102
platform: NodeJS.Platform,
@@ -112,15 +115,16 @@ export function selectEagerPath(
112115
}
113116

114117
/**
115-
* #864 transport gate: win32 traffic that needs a client payload rewrite must
116-
* use the eager single reader with the rewrite applied inline, because the
117-
* alternative tee()+JS-pull chain is the Bun#32111-unsafe path that loses the
118-
* terminal SSE block on Windows. Independent of the version-based eager
119-
* policy: the pull chain is unsafe on the AFFECTED runtimes by definition.
118+
* Transport safety gate: Windows and Darwin traffic that needs a client payload
119+
* rewrite must use the eager single reader with the rewrite applied inline.
120+
* The alternative tee()+JS-pull chain can lose the terminal block on Windows
121+
* (#864) and can stall before the first client frame on Darwin when a Desktop-
122+
* shaped request produces large SSE events. This overrides `streamMode`, since
123+
* pinning the known-bad chain would reintroduce a turn that never completes.
120124
*/
121-
export function isWin32EagerRewrite(
125+
export function requiresEagerRewriteRelay(
122126
platform: NodeJS.Platform,
123127
needsClientRewrite: boolean,
124128
): boolean {
125-
return platform === "win32" && needsClientRewrite;
129+
return (platform === "win32" || platform === "darwin") && needsClientRewrite;
126130
}

‎src/server/index.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -251,8 +251,8 @@ function attachLiveSidebandUpstream(ws: ServerWebSocket<WsData>): void {
251251
// const repairConfig = route.provider.responsesItemIdRepair;
252252
// const needsClientRewrite = imageGenCallAliases.size > 0
253253
// #314 gated shape: win32 no-rewrite traffic follows runtime/config policy; darwin no-rewrite
254-
// traffic requires explicit config-eager opt-in (`auto` always stays tee on darwin). Default OFF
255-
// on the bundled known-bad runtime; policy lives in 260731_macos_rss_retention phase 100.
254+
// traffic requires explicit config-eager opt-in (`auto` always stays tee on darwin). Rewrite
255+
// traffic on either platform takes the eager single-reader safety path regardless of streamMode.
256256
// selectEagerPath(process.platform, needsClientRewrite, config.streamMode ?? "auto")
257257
// relaySseEagerBounded(upstreamResponse.body, turnAc,
258258
// new Response(eagerBody,

‎src/server/relay-eager.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,8 +38,8 @@ export type EagerRelayHooks = {
3838
inspectChunk: (chunk: Uint8Array) => void;
3939
/**
4040
* Optional inline client-facing payload rewrite, framed to complete SSE
41-
* blocks inside the single reader. This is what lets win32 rewrite traffic
42-
* (image_gen restore, item-id repair) use this relay instead of the
41+
* blocks inside the single reader. This is what lets Windows/Darwin rewrite
42+
* traffic (image_gen restore, item-id repair) use this relay instead of the
4343
* Bun#32111-unsafe tee()+JS-pull chain (#864).
4444
*/
4545
rewritePayload?: SsePayloadRewrite;

‎src/server/responses/core.ts‎

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ import {
144144
sanitizePassthroughHeaders,
145145
} from "../relay";
146146
import { relaySseEagerBounded } from "../relay-eager";
147-
import { isWin32EagerRewrite, selectEagerPath } from "../../lib/bun-stream-caps";
147+
import { requiresEagerRewriteRelay, selectEagerPath } from "../../lib/bun-stream-caps";
148148
import { cancelBodyOnAbort } from "../../lib/abort";
149149
import {
150150
createResponsesItemIdPayloadRewrite,
@@ -1825,16 +1825,17 @@ async function handleResponsesInner(
18251825
? createResponsesItemIdPayloadRewrite(repairConfig!, translatorBudget)
18261826
: undefined,
18271827
].filter((rewrite): rewrite is NonNullable<typeof rewrite> => rewrite !== undefined);
1828-
// #864: win32 rewrite traffic must never enter the tee()+JS-pull chain
1829-
// (Bun#32111 JS-sink segfault — text frames pass, the terminal block is
1830-
// lost). The eager single reader applies the same rewrites inline.
1831-
const win32EagerRewrite = isWin32EagerRewrite(process.platform, needsClientRewrite);
1828+
// Rewrite traffic on Windows or Darwin must never enter the tee()+JS-pull
1829+
// chain: Windows can lose the terminal block (#864), while Darwin can stall
1830+
// before delivering the first large Desktop-shaped SSE frame. The eager
1831+
// single reader applies the same bounded rewrites inline on both platforms.
1832+
const eagerRewrite = requiresEagerRewriteRelay(process.platform, needsClientRewrite);
18321833
const eagerPath = selectEagerPath(
18331834
process.platform,
18341835
needsClientRewrite,
18351836
config.streamMode ?? "auto",
18361837
);
1837-
if (eagerPath?.useEagerRelay || win32EagerRewrite) {
1838+
if (eagerPath?.useEagerRelay || eagerRewrite) {
18381839
const turnAc = new AbortController();
18391840
linkAbortSignal(upstream, turnAc.signal);
18401841
registerTurn(turnAc, options.turnAdmissionLease);
@@ -1871,7 +1872,7 @@ async function handleResponsesInner(
18711872
finishInspection: () => inspector.finish(),
18721873
disposeInspection: () => inspector.dispose(),
18731874
sawTerminal: () => inspector.reported(),
1874-
...(win32EagerRewrite
1875+
...(eagerRewrite
18751876
? { rewritePayload: composeSsePayloadRewrites(...payloadRewrites) }
18761877
: {}),
18771878
onSynthetic: kind => {
@@ -1887,10 +1888,10 @@ async function handleResponsesInner(
18871888
},
18881889
onClientCancel: () => options.onNativePassthroughCancel?.(),
18891890
onDone: () => unregisterTurn(turnAc),
1890-
}, win32EagerRewrite ? { rewriteBudget: translatorBudget } : undefined);
1891+
}, eagerRewrite ? { rewriteBudget: translatorBudget } : undefined);
18911892
// selectEagerPath admits only no-rewrite traffic on both eligible platforms;
1892-
// win32 rewrite traffic reaches this relay too, but with the payload rewrite
1893-
// applied inline — never via an image/item-id JS pull wrapper (#32111, #864).
1893+
// Windows/Darwin rewrite traffic reaches this relay through the safety gate,
1894+
// with the payload rewrite inline rather than an image/item-id JS pull wrapper.
18941895
if (!headers.has("content-type")) headers.set("content-type", "text/event-stream");
18951896
return markEagerRelaySseResponse(
18961897
markNativePassthroughSseResponse(new Response(eagerBody, {

0 commit comments

Comments
 (0)