Skip to content

Commit ecd9eaa

Browse files
wtfsayoclaude
andcommitted
fix(devin): replay Claude signatures and retry a refused turn without them
Withholding every Anthropic signature stopped the invalid_argument refusals but also stopped Claude recalling its earlier reasoning: in the live benchmark claude-opus-5-5 matched 2 of 6 on dev and 0 of 6 with the signature withheld. The signature is sent again, and a turn Cognition refuses with invalid_argument before any output is retried once with the Anthropic signatures withheld and the thinking text kept. Other signature types and refusals after output are not retried. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
1 parent e0b81e5 commit ecd9eaa

7 files changed

Lines changed: 193 additions & 28 deletions

File tree

‎scripts/test-layout/layout.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -920,6 +920,7 @@
920920
"devin-provider-merge-migration.test.ts": "providers",
921921
"devin-stated-reset-hardening.test.ts": "providers",
922922
"devin-stated-reset-retry.test.ts": "providers",
923+
"devin-anthropic-signature-fallback.test.ts": "providers",
923924
"devin-reasoning-continuation.test.ts": "providers",
924925
"devin-stream-deadline.test.ts": "providers",
925926
"digitalocean-scaleway-provider.test.ts": "providers",

‎src/adapters/devin.ts‎

Lines changed: 36 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ import { getCachedCatalog, type CacheEntry } from "./devin/cloud-direct/catalog"
1515
import { collapseDevinModelUid } from "./devin/live-models";
1616
import { buildNonOpenAIToolCatalogNudgeForTools } from "./tool-catalog-nudge";
1717
import { DEVIN_DEFAULT_API_SERVER, resolveDevinApiServer } from "../oauth/devin";
18-
import { devinAssistantReasoning, encodeDevinSignature } from "./devin/reasoning-signature";
18+
import { devinAssistantReasoning, encodeDevinSignature, hasAnthropicSignature } from "./devin/reasoning-signature";
1919
import { SendBudgetExhaustedError } from "../lib/upstream-retry";
2020

2121
/**
@@ -370,7 +370,10 @@ function assistantText(message: OcxAssistantMessage): string {
370370
.join("\n");
371371
}
372372

373-
export function mapOcxMessagesToDevin(parsed: OcxParsedRequest): ChatHistoryItem[] {
373+
export function mapOcxMessagesToDevin(
374+
parsed: OcxParsedRequest,
375+
options: { withholdAnthropicSignatures?: boolean } = {},
376+
): ChatHistoryItem[] {
374377
const items: ChatHistoryItem[] = [];
375378
// Cognition is not an OpenAI host, and this adapter does advertise a real
376379
// client tool catalog (proto #10 via `mapOcxToolsToDevin`), so the same
@@ -389,13 +392,17 @@ export function mapOcxMessagesToDevin(parsed: OcxParsedRequest): ChatHistoryItem
389392
if (system) items.push({ role: "system", content: system });
390393

391394
for (const message of parsed.context.messages) {
392-
const mapped = mapOneMessage(message, parsed.modelId);
395+
const mapped = mapOneMessage(message, parsed.modelId, options);
393396
if (mapped) items.push(mapped);
394397
}
395398
return items;
396399
}
397400

398-
function mapOneMessage(message: OcxMessage, modelId: string): ChatHistoryItem | undefined {
401+
function mapOneMessage(
402+
message: OcxMessage,
403+
modelId: string,
404+
options: { withholdAnthropicSignatures?: boolean },
405+
): ChatHistoryItem | undefined {
399406
if (message.role === "user" || message.role === "developer") {
400407
const content = mapOcxContentToWire(message.content);
401408
// An image with no caption text is a complete user message on its own.
@@ -407,7 +414,7 @@ function mapOneMessage(message: OcxMessage, modelId: string): ChatHistoryItem |
407414
if (message.role === "assistant") {
408415
const toolCalls = assistantToolCalls(message);
409416
const text = assistantText(message);
410-
const reasoning = devinAssistantReasoning(message, modelId);
417+
const reasoning = devinAssistantReasoning(message, modelId, options.withholdAnthropicSignatures === true);
411418
// A turn that produced only reasoning is still worth replaying: dropping it
412419
// is what makes the next turn re-derive the same chain.
413420
if (!text && toolCalls.length === 0 && !reasoning.thinking && !reasoning.signature) return undefined;
@@ -616,12 +623,20 @@ export function createDevinAdapter(
616623
// An admitted HTTP turn owns globally shared capacity until this call
617624
// emits. Without an explicit wait allowance, preserve the typed reset
618625
// delay in generated diagnostic wording and return immediately.
619-
for await (const event of streamChatEventsWithResetRetry({
626+
const signedMessages = mapOcxMessagesToDevin(parsed);
627+
// A Claude signature is replayed because it is what carries the reasoning into this
628+
// turn, but Cognition streams Claude's thinking as a summary the signature does not
629+
// cover, and some replays are refused with invalid_argument before any output. That
630+
// refusal is retried once with the Anthropic signatures withheld and the text kept.
631+
const unsignedMessages = hasAnthropicSignature(signedMessages, parsed.modelId)
632+
? mapOcxMessagesToDevin(parsed, { withholdAnthropicSignatures: true })
633+
: undefined;
634+
const request = (messages: ChatHistoryItem[]) => streamChatEventsWithResetRetry({
620635
apiKey,
621636
apiServerUrl: host,
622637
modelUid,
623638
catalog,
624-
messages: mapOcxMessagesToDevin(parsed),
639+
messages,
625640
tools: mapOcxToolsToDevin(parsed.context.tools),
626641
cascadeId,
627642
// Input and output ceilings are separate wire fields. Omitting the
@@ -643,7 +658,20 @@ export function createDevinAdapter(
643658
onPhysicalSend: incoming.onPhysicalSend,
644659
onRecoveryWithheld: incoming.onRecoveryWithheld,
645660
},
646-
})) {
661+
});
662+
async function* withSignatureFallback() {
663+
let produced = false;
664+
try {
665+
for await (const event of request(signedMessages)) {
666+
produced ||= event.kind === "text" || event.kind === "reasoning" || event.kind === "tool_call_start" || event.kind === "tool_call_args";
667+
yield event;
668+
}
669+
} catch (error) {
670+
if (!unsignedMessages || produced || !(error instanceof CloudChatError && error.code === "invalid_argument")) throw error;
671+
yield* request(unsignedMessages);
672+
}
673+
}
674+
for await (const event of withSignatureFallback()) {
647675
if (incoming.abortSignal?.aborted) {
648676
// Emitting nothing here left the bridge to synthesize adapter_eof.
649677
// Say what happened instead, the way the other runTurn-only adapter

‎src/adapters/devin/reasoning-signature.ts‎

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -46,16 +46,13 @@ export function decodeDevinSignature(stored: string): { signature: string; signa
4646
* Any other mix (two signed blocks, a signed block beside unsigned text) has no
4747
* single attestation for the joined text, so the turn is replayed unsigned.
4848
*
49-
* An Anthropic signature is never replayed. Cognition streams Claude's thinking
50-
* as a summary while the signature covers the original, so the pair fails
51-
* validation: live on claude-opus-5-5 the next turn was refused with
52-
* `invalid_argument` in 5 of 6 signed replays and 0 of 3 text-only ones. The
53-
* text still goes back. A stored signature from before the type was recorded
54-
* falls back to the model being called.
49+
* `withholdAnthropic` drops an Anthropic signature and keeps the text: the
50+
* fallback for a Claude turn Cognition refused (see hasAnthropicSignature).
5551
*/
5652
export function devinAssistantReasoning(
5753
message: OcxAssistantMessage,
5854
modelId = "",
55+
withholdAnthropic = false,
5956
): { thinking?: string; signature?: string; signature_type?: string } {
6057
const blocks = message.content.filter(
6158
(part): part is Extract<typeof part, { type: "thinking" }> => part.type === "thinking",
@@ -70,10 +67,28 @@ export function devinAssistantReasoning(
7067
stored = signatureOnly[0]!.signature;
7168
}
7269
let decoded = stored ? decodeDevinSignature(stored) : undefined;
73-
if (decoded && (decoded.signatureType ?? (/claude/i.test(modelId) ? "anthropic" : undefined)) === "anthropic") decoded = undefined;
70+
if (decoded && withholdAnthropic && signatureTypeFor(decoded, modelId) === "anthropic") decoded = undefined;
7471
return {
7572
...(text ? { thinking: text } : {}),
7673
...(decoded ? { signature: decoded.signature } : {}),
7774
...(decoded?.signatureType ? { signature_type: decoded.signatureType } : {}),
7875
};
7976
}
77+
78+
/** A stored signature from before its type was recorded falls back to the model being called. */
79+
function signatureTypeFor(decoded: { signatureType?: string }, modelId: string): string | undefined {
80+
return decoded.signatureType ?? (/claude/i.test(modelId) ? "anthropic" : undefined);
81+
}
82+
83+
/**
84+
* True when the mapped history replays a Claude signature. Cognition streams
85+
* Claude's thinking as a summary while the signature covers the original, so
86+
* the pair can fail validation: live on claude-opus-5-5 a signed replay of a
87+
* visible-thinking turn was refused with `invalid_argument` in 5 of 6 tries and
88+
* a text-only one in none, while a signed replay that is accepted is what lets
89+
* the model recall its earlier reasoning. The adapter therefore sends the
90+
* signature and retries a refusal once without it.
91+
*/
92+
export function hasAnthropicSignature(items: ReadonlyArray<{ signature?: string; signature_type?: string }>, modelId: string): boolean {
93+
return items.some(item => Boolean(item.signature) && signatureTypeFor({ signatureType: item.signature_type }, modelId) === "anthropic");
94+
}

‎structure/providers-and-adapters.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ rewrite rules and the routed-id settlement.
127127
| `src/adapters/declaration-carrier.ts`, `src/adapters/input-media-guard.ts` | Default-deny allowlists for constraints the normalized request carries but a wire may not be able to express: `tools[*].allowed_callers`, which fences a tool off from callers, and inline document bytes. Both are refused with a 400 at the single guard every registered adapter passes through, rather than left to each adapter, because an adapter that never learned about the carrier rebuilds without it and answers normally. `allowed_callers` reaches the `anthropic` wire; document bytes reach `anthropic`, `openai-chat` and `google`; the `openai-responses` wire is exempt from the whole guard because it forwards the original body. Adding an `AdapterWire` member makes the omission visible in these lists instead of at a customer's upstream. The unrestricted `["direct"]` caller default is not a restriction. |
128128
| `src/adapters/azure.ts` | Azure OpenAI bridge. |
129129
| `src/adapters/cursor.ts`, `src/adapters/cursor/` | Cursor protobuf transport: discovery, request builder, event decoding, MCP, thread continuity, native-exec policy. |
130-
| `src/adapters/devin.ts`, `src/adapters/devin/cloud-direct/` | Devin runTurn transport over Cognition Connect-RPC. Assistant reasoning replays as ChatMessagePrompt #11 thinking, #12 signature and #18 signature type (`src/adapters/devin/reasoning-signature.ts`): the #10/#21 pair arrives after the visible answer and becomes its own signature-only reasoning item, so a single unsigned thinking block plus exactly one signature-only block is replayed as one signed prompt, and a signature-only turn (GPT, Gemini) is replayed rather than dropped. An Anthropic signature is withheld and only the thinking text replayed, because the streamed thinking is a summary the signature does not cover and the pair is refused with `invalid_argument`. `GetChatMessage` uses the Responses provider executor and shared physical-send budget; catalog, JWT, and `src/web-search/devin-executor.ts` native search support RPCs remain outside inference-send accounting. Provider-stated pre-output 429 reset delays are surfaced immediately by default, releasing shared active-turn capacity. A positive `OPENCODEX_DEVIN_STATED_RESET_WAIT_MS` explicitly enables bounded waiting and up to two replays on standalone turns, which hold that capacity until completion or cancellation. Combo children bypass that wait and surface a pre-output 429 so the next target can run. During an opted-in standalone wait, safe heartbeats commit the response preflight and keep the stream's stall watchdog fed. Invalid values fail closed to the immediate-refusal behavior. A recorded tenant host is used only for the stored account whose credential owns the transmitted key, searched in the configured provider id and then its deprecated alias; a configured, forwarded, or unmatched key uses the configured base URL or the US default. Native search previews the current route by effective adapter without mutating combo selection state, pins one admitted active-account snapshot for the request, and calls `GetWebSearchResults`, so it starts no CLI or second model. |
130+
| `src/adapters/devin.ts`, `src/adapters/devin/cloud-direct/` | Devin runTurn transport over Cognition Connect-RPC. Assistant reasoning replays as ChatMessagePrompt #11 thinking, #12 signature and #18 signature type (`src/adapters/devin/reasoning-signature.ts`): the #10/#21 pair arrives after the visible answer and becomes its own signature-only reasoning item, so a single unsigned thinking block plus exactly one signature-only block is replayed as one signed prompt, and a signature-only turn (GPT, Gemini) is replayed rather than dropped. An Anthropic signature is replayed, but because the streamed thinking is a summary the signature may not cover, a turn Cognition refuses with `invalid_argument` before any output is retried once with Anthropic signatures withheld and the thinking text kept. `GetChatMessage` uses the Responses provider executor and shared physical-send budget; catalog, JWT, and `src/web-search/devin-executor.ts` native search support RPCs remain outside inference-send accounting. Provider-stated pre-output 429 reset delays are surfaced immediately by default, releasing shared active-turn capacity. A positive `OPENCODEX_DEVIN_STATED_RESET_WAIT_MS` explicitly enables bounded waiting and up to two replays on standalone turns, which hold that capacity until completion or cancellation. Combo children bypass that wait and surface a pre-output 429 so the next target can run. During an opted-in standalone wait, safe heartbeats commit the response preflight and keep the stream's stall watchdog fed. Invalid values fail closed to the immediate-refusal behavior. A recorded tenant host is used only for the stored account whose credential owns the transmitted key, searched in the configured provider id and then its deprecated alias; a configured, forwarded, or unmatched key uses the configured base URL or the US default. Native search previews the current route by effective adapter without mutating combo selection state, pins one admitted active-account snapshot for the request, and calls `GetWebSearchResults`, so it starts no CLI or second model. |
131131
| `src/adapters/kiro.ts` and `src/adapters/kiro/` | Kiro event/tool/thinking/truncation/retry handling, including an egress-aware completion fallback, fixed public HTTP 5xx text, and closed-set status/code diagnostics. The original path is a facade over leaves for wire identity, reasoning, conversation state, token estimation, payload assembly, streaming, and the adapter. |
132132
| `src/adapters/mimo-free.ts` | Mimo Free transport (client identity + JWT). Concurrent requests share one JWT bootstrap bound only to its timeout; each request stops waiting on its own abort without cancelling the others. |
133133
| `src/adapters/command-code.ts`, `src/adapters/command-code-tool-text.ts`, `src/adapters/command-code-restored-schema.ts` | Command Code OAuth NDJSON translation. For every `xiaomi/mimo-` model, text, native calls, reasoning, and terminal decisions share one byte-bounded queue with linear queue visits. Markup is deduplicated against matching native calls; text-only restoration requires one contiguous text run, a clean finish, a declared tool, and arguments validated against supported schema constraints. A parameter-free (freeform) block may omit `</function>` but must end with `</tool_call>`; parameter blocks keep the canonical close. Markup appended after prose in the same delta is split off at the marker and held like a block that opens with `<tool_call>`; a marker split across deltas after prose is still released as text. Native, reasoning, and other intervening events interrupt a still-probing block but leave a held block held in arrival order, and the queued byte bound still flushes an unresolved envelope as text. An envelope the strict parser rejects but that opens with `<tool_call>`, closes with `</tool_call>`, and names a declared function is dropped when a native call for that same function arrives and on a clean finish; markup that parses but fits no supported schema is still released as text. Regex patterns, other unsupported constraints, and abnormal finishes fail closed. `tests/providers/command-code-tool-text-prose-split.test.ts` covers the split, the interleaved-event hold, and both drop paths. |

‎tests/fixtures/test-layout-expected.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -759,6 +759,7 @@
759759
"devin-provider-merge-migration.test.ts": "providers",
760760
"devin-stated-reset-hardening.test.ts": "providers",
761761
"devin-stated-reset-retry.test.ts": "providers",
762+
"devin-anthropic-signature-fallback.test.ts": "providers",
762763
"devin-reasoning-continuation.test.ts": "providers",
763764
"devin-stream-deadline.test.ts": "providers",
764765
"digitalocean-scaleway-provider.test.ts": "providers",

0 commit comments

Comments
 (0)