You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 3a8554b
Browse filesBrowse the repository at this point in the historyBrowse files
fix(responses): record the serving route only after it serves
`updateReasoningReplayServingIdentity` compared and committed in one call, and
`bindRouteReasoningReplayScope` calls it before the request goes out — so the
candidate destination was recorded whether or not that request ever completed.
turn 1 -> A succeeds record = A
turn 2 -> B: A != B, strip A blobs record = B (committed too early)
... this request then fails (rate limit, transport, 5xx)
turn 3 -> retry B: B == B, no strip
but the transcript still carries A-minted blobs -> rejected
The opaque-blob recovery rescues turn 3, so this degraded rather than broke:
one wasted round trip and one turn of degraded reasoning on a path meant to be
deterministic. The record's meaning was the defect — it should mean "this
destination served this thread", and a request that never completed served
nothing.
Split the call in two. `reasoningReplayServingIdentityChanged` compares without
writing; `commitReasoningReplayServingIdentity` records, and runs only at a
successful terminal response. Bounded discipline is unchanged: same LRU/TTL and
byte accounting, same refusal to record without a durable identity dimension,
same fail-soft direction where no record still means keep the blobs.
For bridged transports a terminal means `completed` or `incomplete`. For
streamed passthrough it means a non-error upstream status before relay starts:
waiting for SSE completion would retain request state for the stream's
lifetime, and a later body failure does not undo that the destination accepted
and served the turn. That boundary is stated in the code rather than implied.
The two post-recovery re-records are gone — a successful recovery now reaches
the same terminal commit as any other success.
Regression test: A succeeds, an A->B turn strips and then fails, and the next B
request for the same thread still strips. Verified it fails against the old
code.
Co-Authored-By: Claude Fable 5 <[email protected]>
Copy file name to clipboardExpand all lines: structure/04_transports-and-sidecars.md
+7-3Lines changed: 7 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -563,8 +563,12 @@ its summary survive; `status` has already been removed on every path. Missing, e
563
563
identity state is unknown. The comparison uses the durable destination and credential identities
564
564
with the provider, adapter, and model, so OAuth token-generation refreshes do not look like backend
565
565
changes; when either durable dimension is unavailable it refuses to record rather than falling back
566
-
to a volatile identity. This deterministic pre-flight is the primary path and covers threads the
567
-
process has served while their record remains inside the TTL/LRU bounds. Missing, expired, evicted, and
566
+
to a volatile identity. Route binding only compares: it does not replace the recorded identity until
567
+
the destination successfully serves the turn. Bridged streams commit on a completed or incomplete
568
+
terminal; native passthrough streams use the non-error upstream status before relay as their success
569
+
boundary so the proxy does not retain request state across the whole stream. This deterministic
570
+
pre-flight is the primary path and covers threads the process has served while their record remains
571
+
inside the TTL/LRU bounds. Missing, expired, evicted, and
568
572
pre-process history stays fail-soft on the first send. If a Responses upstream then returns its own
569
573
self-identifying opaque-blob 4xx (`invalid_encrypted_content`, or xAI's two `invalid-argument`
570
574
decoder errors), the proxy rebuilds once through the same sanitation path: reasoning
@@ -591,7 +595,7 @@ conversation's last-serving identity and cause a later main-model turn to strip
591
595
- 목적과 의도: Keep same-backend opaque reasoning replay while preventing backend-private blobs and output-only fields from breaking the first turn after a route change.
592
596
- 기존 구현 및 제약 조건: Reasoning-input sanitation already handled raw content and `ocxr1:` envelopes; the replay cache already supplied a bounded, thread-scoped physical-route identity, but no record connected that identity to native `encrypted_content` provenance.
593
597
- 검토한 주요 대안: Strip every opaque blob, persist provenance across restarts, trust generic 4xx prose, rely only on a retry, or combine deterministic route comparison with a narrowly identified recovery.
594
-
- 선택한 방식: Remove output-only `status` from every reasoning input item without changing the pre-existing raw-`content` blanking rule; compare and update a 64-entry/256 KiB/one-hour in-process serving-identity record using durable destination and credential dimensions at request time, pass a proven change into the Responses adapter before the first send, and use one self-identified opaque-blob recovery only when provenance was unknown.
598
+
- 선택한 방식: Remove output-only `status` from every reasoning input item without changing the pre-existing raw-`content` blanking rule; compare a 64-entry/256 KiB/one-hour in-process serving-identity record using durable destination and credential dimensions before sending, commit it only after the selected destination succeeds, pass a proven change into the Responses adapter before the first send, and use one self-identified opaque-blob recovery only when provenance was unknown.
595
599
- 다른 대안 대신 이 방식을 선택한 이유: The former blob/status coupling was defensive rather than observed, and live backends showed that removing `status` preserves same-backend Grok replay while allowing cold cross-backend requests to reach blob validation. Unknown provenance can still be valid after restart, durable storage is unnecessary for this bounded compatibility hint, and deterministic pre-flight avoids the extra paid or stateful upstream attempt whenever the process has evidence. The upstream's narrow error identity supplies authoritative evidence only for histories the process could not observe.
596
600
- 장점, 단점 및 영향: Same-route and unknown replay retain cached reasoning on the first send without replaying an output-only field, known cross-route replay keeps the reasoning item without its undecodable blob, and a cold cross-route replay can reach opaque-blob recovery instead of failing early on `status`. A repeated blob rejection is surfaced unchanged after exactly one recovery attempt.
0 commit comments