Skip to content

Commit d219dbc

Browse files
committed
fix: harden responses snapshot repair
1 parent 2670f24 commit d219dbc

17 files changed

Lines changed: 741 additions & 61 deletions

File tree

‎devlog/_plan/260731_macos_rss_retention/060_impl_roadmap.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ the user for this unit's commits.
3434
| `070_harness_warm_fix.md` | wp2 | Apply 050's harness diff (monotonic pause clamp, child-exit/duration split) + post-patch remote smoke calibration | — |
3535
| `080_inspection_bounds.md` | wp3 | relay.ts inspector caps + clear points + disconnect cancel (both consumers) + parse-once + observability counters on `/api/system/memory` | — |
3636
| `090_eager_failed_tail.md` | wp4 | relay-eager.ts synthetic `response.failed` tail on mid-stream reset | 080 (dispose hook touches the eager producer finally) |
37-
| `100_darwin_eager_optin.md` | wp5 | core.ts gate: darwin joins win32 for explicit `eager-relay` no-rewrite traffic; `auto` stays tee; local darwin abort-stress gate | 090 (tail must exist before opt-in is reachable) |
37+
| `100_darwin_eager_optin.md` | wp5 | core.ts gate: darwin joins win32 for explicit `eager-relay`, including inline client rewrites; `auto` stays tee; local darwin abort-stress gate | 090 (tail must exist before opt-in is reachable) |
3838
| `110_verify_and_push.md` | wp6 | full-suite/typecheck/privacy gates, devlog closeout, push | 070–100 |
3939

4040
Ordering rationale: 070 is independent and unblocks any future macmini re-run.

‎devlog/_plan/260731_macos_rss_retention/100_darwin_eager_optin.md‎

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -139,8 +139,9 @@ gate execution record is authoritative.
139139

140140
## Not changed
141141

142-
- Rewrite traffic (image-gen aliases / item-id repair) stays on tee on both
143-
platforms (`needsClientRewrite` guard intact).
142+
- Darwin `auto` rewrite traffic stays on tee. An explicit
143+
`streamMode: "eager-relay"` may compose a client-facing rewrite inline on
144+
the single reader; raw inspection still receives the original bytes.
144145
- `decideEagerRelay` itself unchanged — the darwin restriction lives at the
145146
call site because it is platform policy, not runtime capability.
146147
- Linux unchanged (no opt-in until asked; smallest honest scope).
@@ -159,7 +160,7 @@ extraction is preferred for clarity, not necessity. Test matrix:
159160
2. win32 + no-rewrite + auto/known-bad → tee (unchanged).
160161
3. darwin + no-rewrite + config-eager → eager (NEW).
161162
4. darwin + no-rewrite + auto (even with minFixed satisfied) → tee.
162-
5. darwin + rewrite + config-eager → tee.
163+
5. darwin + rewrite + config-eager → eager single-reader inline rewrite.
163164
6. linux + anything → tee.
164165

165166
Plus one DIRECT `handleResponses` integration test gated to darwin only
@@ -251,3 +252,21 @@ Confirmed by the same round: C1-1 ack chain sound, C1-2 stall detection and
251252
all-or-nothing unreachable classification internally sound, C1-3 single
252253
normalized selector contract with darwin-auto never eager and win32
253254
unchanged.
255+
256+
## 2026-08-02 client-rewrite amendment
257+
258+
An opt-in Responses snapshot repair exposed a contradiction in the original
259+
wp5 scope: keeping Darwin rewrite traffic on tee would reintroduce the second
260+
reader and its retention cost precisely when a provider needs client-facing
261+
compatibility repair. Explicit Darwin eager mode now permits inline payload
262+
rewrites after raw inspection; Darwin `auto` and every Linux mode remain
263+
unchanged.
264+
265+
The handler-level regression test uses `streamMode: "eager-relay"` together
266+
with an active snapshot rewrite and asserts the eager-path marker plus the
267+
repaired terminal. Relay tests separately prove fragmented SSE framing,
268+
client-only failed-tail behavior, upstream abort/cancel on rewrite failure,
269+
and raw-terminal accounting. The 2026-08-01 abort-stress record above did not
270+
exercise payload rewriting, so it remains evidence for the shared
271+
single-reader transport rather than a separate rewrite-stress claim; the
272+
feature remains explicit opt-in with `legacy-tee` as rollback.

‎docs-site/src/content/docs/ja/reference/configuration/providers.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ description: プロバイダー エントリ、認証、エンドポイント、
7474
| `noPenaltyModels?` | `string[]` |存在/周波数ペナルティを拒否するモデル。 |
7575
| `parallelToolCalls?` | `boolean` |並列ツール呼び出しを切り替えます。 OpenAI Chat はデフォルトでオンになっています。非チャット アダプターは明示的な `true` でのみアドバタイズします。 |
7676
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` |正確なプレースホルダー ID および欠落している端末 ID に対するダウンストリーム SSE 修復はデフォルトで無効になっています。関数呼び出し ID は決して書き換えられません。 |
77+
| `responsesSnapshotRepair?` | `boolean` | デフォルトで無効のクライアント向け修復です。SSE と JSON の Responses ライフサイクルで欠落した status、output、ツールメタデータを補完し、raw 検査と永続化は変更しません。 |
7778
| `autoToolChoiceOnlyModels?` | `string[]` | `tool_choice` が `auto` または `none` のみを受け入れるモデル。強制的な選択は格下げされます。 |
7879
| `preserveReasoningContentModels?` | `string[]` |チャット履歴に以前のアシスタント `reasoning_content` が必要なモデル。 |
7980
| `thinkingToggleModels?` | `string[]` |エフォート ラダーではなく `thinking.enabled` を使用してモデルをチャットします。 |
@@ -186,7 +187,8 @@ Anthropic アカウント ポリシーのリスクを理解していない限り
186187
"reasoning": ["rs_0"],
187188
"message": ["msg_0"],
188189
"repairMissingTerminalIds": true
189-
}
190+
},
191+
"responsesSnapshotRepair": true
190192
}
191193
}
192194
}

‎docs-site/src/content/docs/ko/reference/configuration/providers.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ description: 공급자 항목, 인증, 엔드포인트, 모델 카탈로그, 할
7474
| `noPenaltyModels?` | `string[]` | presence/frequency penalty를 허용하지 않는 모델입니다. |
7575
| `parallelToolCalls?` | `boolean` | 병렬 도구 호출을 켜거나 끕니다. OpenAI Chat은 기본으로 켜져 있고, 비-chat 어댑터는 명시적으로 `true`일 때만 이를 노출합니다. |
7676
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` | 기본값이 꺼진 downstream SSE 복구입니다. 정확한 자리표시자 id와 누락된 종료 id를 복구합니다. function-call id는 다시 쓰지 않습니다. |
77+
| `responsesSnapshotRepair?` | `boolean` | 기본값이 꺼진 클라이언트용 복구입니다. SSE와 JSON의 Responses 수명 주기에서 누락된 status, output, 도구 메타데이터를 채우며 raw 검사와 영속화는 변경하지 않습니다. |
7778
| `autoToolChoiceOnlyModels?` | `string[]` | `tool_choice`가 `auto` 또는 `none`만 받는 모델입니다. 강제 선택은 낮은 수준으로 바뀝니다. |
7879
| `preserveReasoningContentModels?` | `string[]` | chat 기록에서 이전 assistant `reasoning_content`가 필요한 모델입니다. |
7980
| `thinkingToggleModels?` | `string[]` | effort 계층 대신 `thinking.enabled`를 쓰는 chat 모델입니다. |
@@ -189,7 +190,8 @@ Anthropic 계정 정책 위험을 이해하지 못한다면 이 기능은 꺼두
189190
"reasoning": ["rs_0"],
190191
"message": ["msg_0"],
191192
"repairMissingTerminalIds": true
192-
}
193+
},
194+
"responsesSnapshotRepair": true
193195
}
194196
}
195197
}

‎docs-site/src/content/docs/reference/configuration/providers.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids.
8585
| `noPenaltyModels?` | `string[]` | Models that reject presence/frequency penalties. |
8686
| `parallelToolCalls?` | `boolean` | Toggle parallel tool calls. OpenAI Chat defaults on; non-chat adapters advertise only on explicit `true`. |
8787
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` | Disabled-by-default downstream SSE repair for exact placeholder ids and missing terminal ids. Function-call ids are never rewritten. |
88+
| `responsesSnapshotRepair?` | `boolean` | Disabled-by-default client-facing repair for sparse Responses lifecycle snapshots in SSE and JSON. Fills missing canonical status, output, and tool metadata while raw inspection and persistence remain unchanged. |
8889
| `autoToolChoiceOnlyModels?` | `string[]` | Models whose `tool_choice` accepts only `auto` or `none`; forced choices are downgraded. |
8990
| `preserveReasoningContentModels?` | `string[]` | Models requiring prior assistant `reasoning_content` in chat history. |
9091
| `thinkingToggleModels?` | `string[]` | Chat models using `thinking.enabled` rather than an effort ladder. |
@@ -230,7 +231,8 @@ For a broken `openai-responses` gateway, repair belongs on the provider object:
230231
"reasoning": ["rs_0"],
231232
"message": ["msg_0"],
232233
"repairMissingTerminalIds": true
233-
}
234+
},
235+
"responsesSnapshotRepair": true
234236
}
235237
}
236238
}

‎docs-site/src/content/docs/ru/reference/configuration/providers.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,7 @@ cross-route credential fallback не существует. Строки API GPT-
9090
| `noPenaltyModels?` | `string[]` | Модели, отвергающие penalty presence/frequency. |
9191
| `parallelToolCalls?` | `boolean` | Переключатель parallel tool call'ов. Для OpenAI Chat по умолчанию включено; не-chat adapter'ы рекламируют это только при явном `true`. |
9292
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` | По умолчанию выключенная downstream SSE-repair для exact placeholder-id и отсутствующих terminal-id. Function-call id никогда не переписываются. |
93+
| `responsesSnapshotRepair?` | `boolean` | По умолчанию выключенная клиентская repair для неполных lifecycle snapshot'ов Responses в SSE и JSON. Добавляет отсутствующие status, output и tool metadata, не меняя raw inspection и persistence. |
9394
| `autoToolChoiceOnlyModels?` | `string[]` | Модели, у которых `tool_choice` принимает только `auto` или `none`; forced choice понижается. |
9495
| `preserveReasoningContentModels?` | `string[]` | Модели, которым нужен предыдущий assistant `reasoning_content` в chat history. |
9596
| `thinkingToggleModels?` | `string[]` | Chat-модели, использующие `thinking.enabled` вместо effort-ladder. |
@@ -240,7 +241,8 @@ Beijing, а `alibaba-token-plan-intl` обслуживает междунаро
240241
"reasoning": ["rs_0"],
241242
"message": ["msg_0"],
242243
"repairMissingTerminalIds": true
243-
}
244+
},
245+
"responsesSnapshotRepair": true
244246
}
245247
}
246248
}

‎docs-site/src/content/docs/zh-cn/reference/configuration/providers.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ description: 提供者条目、身份验证、端点、模型目录、配额、
7474
| `noPenaltyModels?` | `string[]` | 会拒绝 presence/frequency penalty 的模型。 |
7575
| `parallelToolCalls?` | `boolean` | 切换并行工具调用。OpenAI Chat 默认开启;非 chat 适配器只有显式 `true` 时才会声明支持。 |
7676
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` | 默认关闭的下游 SSE 修复,用于精确占位 id 和缺失的终止 id。function-call id 永远不会被重写。 |
77+
| `responsesSnapshotRepair?` | `boolean` | 默认关闭的客户端修复,用于补全 SSE 与 JSON 中稀疏 Responses 生命周期快照缺失的 status、output 和工具元数据;原始检查与持久化保持不变。 |
7778
| `autoToolChoiceOnlyModels?` | `string[]` | `tool_choice` 只接受 `auto` 或 `none` 的模型;强制选择会被降级。 |
7879
| `preserveReasoningContentModels?` | `string[]` | 需要在聊天历史中保留先前 assistant `reasoning_content` 的模型。 |
7980
| `thinkingToggleModels?` | `string[]` | 使用 `thinking.enabled` 而不是 effort 阶梯的 chat 模型。 |
@@ -183,7 +184,8 @@ affinity。这些策略不能规避 provider enforcement。
183184
"reasoning": ["rs_0"],
184185
"message": ["msg_0"],
185186
"repairMissingTerminalIds": true
186-
}
187+
},
188+
"responsesSnapshotRepair": true
187189
}
188190
}
189191
}

‎src/lib/bun-stream-caps.ts‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -92,10 +92,10 @@ export function decideEagerRelay(
9292
* Apply the two-platform eager-relay policy to the runtime/config capability.
9393
* Windows preserves the decision for no-rewrite traffic. Darwin permits only
9494
* explicit config opt-in; `auto` remains tee even on a future fixed runtime.
95-
* Returns the normalized effective decision, or null when platform policy or a Darwin
96-
* non-config-eager mode selects tee. Windows rewrites remain on tee because the extra JS pull
97-
* wrapper is part of the Bun#32111 crash shape; Darwin's explicit eager mode may compose a
98-
* client-facing rewrite after the single-reader relay.
95+
* Returns the normalized effective decision, or null when platform policy, a Windows rewrite,
96+
* or a Darwin non-config-eager mode excludes this path. Windows rewrites are forced onto the
97+
* eager single-reader path separately by `isWin32EagerRewrite`; Darwin's explicit eager mode may
98+
* compose a client-facing rewrite after the single-reader relay.
9999
*/
100100
export function selectEagerPath(
101101
platform: NodeJS.Platform,

‎src/server/relay-eager.ts‎

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,10 @@ export function relaySseEagerBounded(
161161
let queuedBytes = 0;
162162
let cancelled = false;
163163
let done = false;
164+
// Raw inspection can observe a terminal before an inline rewrite succeeds.
165+
// Track client delivery separately so a rewrite failure cannot leave the
166+
// client with an empty/truncated stream while persistence records success.
167+
let clientTerminalEnqueued = false;
164168
// Pause gate: resolved by client pull, client cancel, or upstream abort so a
165169
// paused producer ALWAYS resumes (audit blocker 2 — no deadlock; onDone and
166170
// turn unregistration stay reachable, drainAndShutdown never hangs).
@@ -192,6 +196,7 @@ export function relaySseEagerBounded(
192196

193197
const producer = async () => {
194198
let syntheticKind: "incomplete" | "failed" | null = null;
199+
let clientRelayFailed = false;
195200
// reader.read() is not intrinsically tied to the upstream AbortController
196201
// (a fetch body usually rejects on abort, but that coupling is the fetch
197202
// implementation's, not the stream's). Race every read against the abort
@@ -212,7 +217,10 @@ export function relaySseEagerBounded(
212217
const tail = flushRewriteTail();
213218
if (tail.byteLength > 0 && !cancelled) {
214219
queuedBytes += tail.byteLength;
215-
try { controllerRef?.enqueue(tail); } catch { /* client already gone */ }
220+
try {
221+
controllerRef?.enqueue(tail);
222+
if (hooks.sawTerminal()) clientTerminalEnqueued = true;
223+
} catch { /* client already gone */ }
216224
}
217225
}
218226
if (!hooks.sawTerminal() && !cancelled && !upstream.signal.aborted) {
@@ -235,6 +243,7 @@ export function relaySseEagerBounded(
235243
queuedBytes += outbound.byteLength;
236244
try {
237245
controllerRef?.enqueue(outbound);
246+
if (hooks.sawTerminal()) clientTerminalEnqueued = true;
238247
} catch {
239248
// Controller already torn down (client went away without cancel()).
240249
cancelled = true;
@@ -249,16 +258,19 @@ export function relaySseEagerBounded(
249258
} catch (err) {
250259
// Upstream read failure. Distinguish genuine mid-stream reset from
251260
// abort-driven teardown (shutdown/cancel-expiry) — audit M3.
252-
if (!hooks.sawTerminal() && !cancelled && !upstream.signal.aborted) {
261+
if (!clientTerminalEnqueued && !cancelled && !upstream.signal.aborted) {
253262
// Serializing `err` can run user-defined accessors (Error.message
254263
// getters, toString) that re-entrantly cancel the client or abort the
255264
// upstream. Build the tail FIRST, then re-check eligibility before
256265
// committing to the synthetic terminal (adversarial review blocker).
257266
const tail = new TextEncoder().encode(
258267
`\n\nevent: response.failed\ndata: ${buildFailedTailPayload(err)}\n\ndata: [DONE]\n\n`,
259268
);
260-
if (!hooks.sawTerminal() && !cancelled && !upstream.signal.aborted) {
261-
syntheticKind = "failed";
269+
if (!clientTerminalEnqueued && !cancelled && !upstream.signal.aborted) {
270+
// If raw inspection already recorded a real terminal, the failed
271+
// tail is client-only. Do not overwrite raw terminal accounting.
272+
if (!hooks.sawTerminal()) syntheticKind = "failed";
273+
clientRelayFailed = true;
262274
queuedBytes += tail.byteLength;
263275
try { controllerRef?.enqueue(tail); } catch { /* client already torn down */ }
264276
try { controllerRef?.close(); } catch { /* client already torn down */ }
@@ -276,14 +288,15 @@ export function relaySseEagerBounded(
276288
if (cancelled && !hooks.sawTerminal()) {
277289
hooks.onClientCancel();
278290
}
279-
if (cancelled || upstream.signal.aborted || syntheticKind === "failed") {
291+
if (cancelled || upstream.signal.aborted || syntheticKind === "failed" || clientRelayFailed) {
280292
upstream.abort();
281293
reader.cancel().catch(() => {});
282294
}
283295
if (!cancelled) {
284296
try { controllerRef?.close(); } catch { /* already closed/errored */ }
285297
}
286298
try { hooks.disposeInspection?.(); } catch { /* inspection teardown must not block lifecycle cleanup */ }
299+
try { rewrite?.dispose?.(); } catch { /* rewrite teardown must not block lifecycle cleanup */ }
287300
fireDone();
288301
}
289302
};

0 commit comments

Comments
 (0)