Skip to content

Commit 4f9f728

Browse files
olddonkeyclaude
andcommitted
fix(responses): compare the serving identity on rotation-safe dimensions
The serving-identity record compared `credentialIdentity`, which for OAuth is `accountId + generation` and therefore changes on every token refresh. Six of the eight `bindRouteReasoningReplayScope` call sites are key-rotation or OAuth-refresh rebinds, so an ordinary refresh registered as "the backend changed" and the next turn on that thread dropped a valid blob. Key-pool providers would have paid that repeatedly, and silently — nothing errors, the model just loses cached reasoning. The module already distinguishes the durable dimensions for exactly this reason (#1926: the rotating generation deliberately does not participate). The serving record now compares `providerDestinationDurableIdentity` and `credentialDurableIdentity`, and refuses to record at all when those are missing rather than falling back to the volatile pair: a missed strip costs one degraded turn, a spurious strip is a permanent quality regression. The proxy-owned replay cache keeps its stricter key, which is deliberate. Also documents two behaviours that would otherwise read as bugs: a combo that rotates targets between turns legitimately drops blobs while the SSE model-name rewrite hides the switch from the client, and the image/web-search loops consume the replay scope without rebinding, which is what stops an internal small-model call from poisoning the record for the main conversation. Co-Authored-By: Claude Fable 5 <[email protected]>
1 parent c3d62b0 commit 4f9f728

3 files changed

Lines changed: 87 additions & 9 deletions

File tree

‎src/responses/reasoning-replay-cache.ts‎

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,25 @@ function tupleForIdentity(
9191
];
9292
}
9393

94+
function tupleForServingIdentity(
95+
identity: Readonly<OcxReasoningReplayIdentity> | undefined,
96+
): ReasoningReplayIdentityTuple | undefined {
97+
if (
98+
!nonEmpty(identity?.providerName)
99+
|| !nonEmpty(identity?.providerDestinationDurableIdentity)
100+
|| !nonEmpty(identity?.adapterName)
101+
|| !nonEmpty(identity?.modelId)
102+
|| !nonEmpty(identity?.credentialDurableIdentity)
103+
) return undefined;
104+
return [
105+
identity.providerName,
106+
identity.providerDestinationDurableIdentity,
107+
identity.adapterName,
108+
identity.modelId,
109+
identity.credentialDurableIdentity,
110+
];
111+
}
112+
94113
function keyFor(callId: string, scope: OcxReasoningReplayScopeRef | undefined): string | undefined {
95114
const identity = tupleForIdentity(scope?.current);
96115
if (!nonEmpty(callId) || !nonEmpty(scope?.clientThreadId) || !identity) return undefined;
@@ -119,14 +138,16 @@ function sweepExpiredServingIdentities(at: number): void {
119138
* record the current route. A live mismatch means replayed opaque reasoning was minted by
120139
* another backend and must not be forwarded to this one.
121140
*
122-
* Missing, expired, or evicted state is deliberately unknown rather than a mismatch. This
123-
* store is process-local, so a backend switch spanning a proxy restart is not detected.
141+
* Serving provenance uses restart-stable destination and credential dimensions so token
142+
* generations and other volatile credential material cannot create false route changes. Missing
143+
* durable identity, expired, or evicted state is deliberately unknown rather than a mismatch.
144+
* This store is process-local, so a backend switch spanning a proxy restart is not detected.
124145
*/
125146
export function updateReasoningReplayServingIdentity(
126147
scope: OcxReasoningReplayScopeRef | undefined,
127148
): boolean {
128149
const threadId = scope?.clientThreadId;
129-
const identityTuple = tupleForIdentity(scope?.current);
150+
const identityTuple = tupleForServingIdentity(scope?.current);
130151
if (!nonEmpty(threadId) || !identityTuple) return false;
131152
const identity = JSON.stringify(identityTuple);
132153

‎structure/04_transports-and-sidecars.md‎

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -488,14 +488,29 @@ blob is kept unless the in-process thread record proves that the current provide
488488
adapter, model, or credential differs from the route recorded for the prior request on that client
489489
thread. On a proven change the blob and `status` are removed while the reasoning item and its summary
490490
survive; `status` is also removed from blobless reasoning items. Missing, expired, or evicted identity
491-
state is unknown. The record is deliberately process-local, so a backend switch spanning a proxy
491+
state is unknown. The comparison uses the durable destination and credential identities with the
492+
provider, adapter, and model, so OAuth token-generation refreshes do not look like backend changes;
493+
when either durable dimension is unavailable it refuses to record rather than falling back to a
494+
volatile identity. The record is deliberately process-local, so a backend switch spanning a proxy
492495
restart is not detected and may still be rejected upstream.
493496

497+
A combo target rotation between turns legitimately changes that serving identity, so the following
498+
turn drops blobs minted by the prior target. This is correct because the new target cannot decode
499+
them, but it is intentionally unobvious to the client: `pickComboTarget` keys selection state only by
500+
combo id, without a conversation dimension, and the SSE model-name rewrite preserves the requested
501+
combo name instead of exposing the concrete target switch. A user can therefore observe a reasoning
502+
cache drop with no visible model change.
503+
504+
The image and web-search auxiliary loops consume `_reasoningReplayScope` for bridge-level replay but
505+
never call `bindRouteReasoningReplayScope`, so their internal small-model requests do not update the
506+
serving-identity record. That omission is intentional: binding those routes would poison the main
507+
conversation's last-serving identity and cause a later main-model turn to strip valid blobs.
508+
494509
[Decision Log]
495510
- 목적과 의도: Keep same-backend opaque reasoning replay while preventing backend-private blobs and output-only fields from breaking the first turn after a route change.
496511
- 기존 구현 및 제약 조건: Reasoning-input sanitation already handled raw content and `ocxr1:` envelopes; the replay cache already supplied a bounded, thread-scoped physical-route identity, but no record connected that identity to native `encrypted_content` provenance.
497512
- 검토한 주요 대안: Strip every opaque blob, persist provenance across restarts, retry after an upstream 4xx, or compare and strip before the first outbound request only when an in-process record proves a route change.
498-
- 선택한 방식: Preserve `status` whenever its blob is forwarded without changing the pre-existing raw-`content` blanking rule; otherwise remove output-only `status`, compare and update a 64-entry/256 KiB/one-hour in-process serving-identity record at request time, and pass the proven-change decision into the Responses adapter to remove foreign `encrypted_content`.
513+
- 선택한 방식: Preserve `status` whenever its blob is forwarded without changing the pre-existing raw-`content` blanking rule; otherwise remove output-only `status`, compare and update a 64-entry/256 KiB/one-hour in-process serving-identity record using durable destination and credential dimensions at request time, and pass the proven-change decision into the Responses adapter to remove foreign `encrypted_content`.
499514
- 다른 대안 대신 이 방식을 선택한 이유: Unknown provenance can still be valid after restart, durable storage is unnecessary for this bounded compatibility hint, and a deterministic pre-flight decision avoids a second paid or stateful upstream attempt.
500515
- 장점, 단점 및 영향: Same-route and unknown replay retain cached reasoning, known cross-route replay keeps the reasoning item without its undecodable blob, and switches spanning a proxy restart remain an explicit coverage gap.
501516

‎tests/reasoning-replay-identity.test.ts‎

Lines changed: 46 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,11 @@ function scope(
2626
current: {
2727
providerName: "provider-a",
2828
providerDestinationIdentity: "destination:provider-a",
29+
providerDestinationDurableIdentity: "destination:durable-provider-a",
2930
adapterName: "openai-chat",
3031
modelId: "deepseek-v4-flash",
3132
credentialIdentity: "key:physical-a",
33+
credentialDurableIdentity: "credential:durable-slot-a",
3234
...overrides,
3335
},
3436
};
@@ -70,17 +72,35 @@ describe("reasoning replay provider and credential identity", () => {
7072
}
7173
});
7274

73-
test("serving identity comparison reports only known model or destination changes", () => {
74-
expect(updateReasoningReplayServingIdentity(scope())).toBe(false);
75-
expect(updateReasoningReplayServingIdentity(scope())).toBe(false);
75+
test("serving identity ignores credential generation but reports durable route changes", () => {
76+
expect(updateReasoningReplayServingIdentity(scope({
77+
credentialIdentity: "oauth:slot-a-generation-a",
78+
}))).toBe(false);
79+
expect(updateReasoningReplayServingIdentity(scope({
80+
credentialIdentity: "oauth:slot-a-generation-b",
81+
}))).toBe(false);
7682

77-
const changedModel = scope({ modelId: "deepseek-v4" });
83+
const changedModel = scope({
84+
modelId: "deepseek-v4",
85+
credentialIdentity: "oauth:slot-a-generation-b",
86+
});
7887
expect(updateReasoningReplayServingIdentity(changedModel)).toBe(true);
7988
expect(updateReasoningReplayServingIdentity(changedModel)).toBe(false);
8089

90+
const changedCredential = scope({
91+
modelId: "deepseek-v4",
92+
credentialIdentity: "oauth:slot-b-generation-a",
93+
credentialDurableIdentity: "credential:durable-slot-b",
94+
});
95+
expect(updateReasoningReplayServingIdentity(changedCredential)).toBe(true);
96+
expect(updateReasoningReplayServingIdentity(changedCredential)).toBe(false);
97+
8198
const changedDestination = scope({
8299
modelId: "deepseek-v4",
83100
providerDestinationIdentity: "destination:provider-b",
101+
providerDestinationDurableIdentity: "destination:durable-provider-b",
102+
credentialIdentity: "oauth:slot-b-generation-a",
103+
credentialDurableIdentity: "credential:durable-slot-b",
84104
});
85105
expect(updateReasoningReplayServingIdentity(changedDestination)).toBe(true);
86106
expect(updateReasoningReplayServingIdentity(changedDestination)).toBe(false);
@@ -89,6 +109,28 @@ describe("reasoning replay provider and credential identity", () => {
89109
expect(updateReasoningReplayServingIdentity({ clientThreadId: "thread-unknown" })).toBe(false);
90110
});
91111

112+
test("serving identity refuses to record when durable dimensions are unavailable", () => {
113+
const clientThreadId = "thread-without-durable-identity";
114+
expect(updateReasoningReplayServingIdentity({
115+
...scope({ credentialDurableIdentity: undefined }),
116+
clientThreadId,
117+
})).toBe(false);
118+
expect(updateReasoningReplayServingIdentity({
119+
...scope({ modelId: "different-model" }),
120+
clientThreadId,
121+
})).toBe(false);
122+
123+
const destinationThreadId = "thread-without-durable-destination";
124+
expect(updateReasoningReplayServingIdentity({
125+
...scope({ providerDestinationDurableIdentity: undefined }),
126+
clientThreadId: destinationThreadId,
127+
})).toBe(false);
128+
expect(updateReasoningReplayServingIdentity({
129+
...scope({ modelId: "different-model" }),
130+
clientThreadId: destinationThreadId,
131+
})).toBe(false);
132+
});
133+
92134
test("expired serving identity is unknown rather than a backend change", () => {
93135
let clock = 1_000;
94136
clearReasoningReplayCacheForTests(() => clock);

0 commit comments

Comments
 (0)