Skip to content

Commit bf49de2

Browse files
wtfsayoclaude
authored andcommitted
fix(devin): carry reasoning signatures across turns
SWE-2 streams its #10 delta_signature and #21 delta_signature_type after the visible answer, so the Responses layer stores them as a signature-only reasoning item behind the thinking item. The replay mapping kept only thinking blocks with text, so the signature never went back, and GPT and Gemini rows, whose reasoning is signature-only, replayed nothing at all. - Decode #21 with #10 and carry the type inside the stored signature. - Replay one unsigned thinking block plus exactly one signature-only block as a single signed prompt (#11, #12, #18), and replay signature-only turns instead of dropping them. Existing rules stay: a signed block wins over a stray signature-only block, and ambiguous mixes go unsigned. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> (cherry picked from commit c483502)
1 parent 6331e56 commit bf49de2

7 files changed

Lines changed: 178 additions & 40 deletions

File tree

‎scripts/test-layout/layout.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -759,6 +759,7 @@
759759
"devin-provider-merge-migration.test.ts": "providers",
760760
"devin-stated-reset-hardening.test.ts": "providers",
761761
"devin-stated-reset-retry.test.ts": "providers",
762+
"devin-reasoning-continuation.test.ts": "providers",
762763
"devin-stream-deadline.test.ts": "providers",
763764
"digitalocean-scaleway-provider.test.ts": "providers",
764765
"docs-429-failover-claims.test.ts": "ci-workflows",

‎src/adapters/devin.ts‎

Lines changed: 5 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ import { getCachedCatalog, type CacheEntry, type ModelCatalogEntry } from "./dev
1515
import { collapseDevinModelUid, devinFamiliesOf, devinFamilyBaseId, selectDevinFamilyMember, type DevinVariantRequest } from "./devin/live-models";
1616
import { buildNonOpenAIToolCatalogNudgeForTools } from "./tool-catalog-nudge";
1717
import { DEVIN_DEFAULT_API_SERVER, resolveDevinApiServer } from "../oauth/devin";
18-
import { isProviderIssuedThinkingSignature } from "../responses/reasoning-envelope";
18+
import { devinAssistantReasoning, encodeDevinSignature } from "./devin/reasoning-signature";
1919
import { SendBudgetExhaustedError } from "../lib/upstream-retry";
2020
import { devinContextOverflowEvent, isDevinHistoryOverflow } from "./devin/context-overflow";
2121

@@ -452,44 +452,12 @@ function assistantText(message: OcxAssistantMessage): string {
452452
// Thinking stays out of the replayed TEXT: folding chain-of-thought into
453453
// assistant text sends it back as visible prior output, which the model
454454
// then treats as something it said to the user. It is replayed in its own
455-
// field instead — see assistantThinking below.
455+
// field instead — see devinAssistantReasoning.
456456
.map((part) => (part.type === "text" ? part.text : ""))
457457
.filter(Boolean)
458458
.join("\n");
459459
}
460460

461-
/**
462-
* The assistant turn's own reasoning, for replay in ChatMessagePrompt #11.
463-
*
464-
* This adapter previously asserted that Cognition has no reasoning-replay
465-
* field and dropped the thinking outright, so a reasoning model restarted its
466-
* chain on every turn of a tool loop. The field exists: two independent
467-
* clients of the same service write #11 thinking with #12 signature and #18
468-
* signature_type on the assistant prompt.
469-
*
470-
* Field #12 attests the exact text at #11, and the wire has room for one pair.
471-
* Every block that carries text is replayed, so the chain stays intact; the
472-
* signature rides along only when the text being replayed IS the text it
473-
* attests, which is exactly the single-block case. Several independently signed
474-
* blocks send an unsigned prompt rather than pairing one block's attestation
475-
* with another block's words. A signature-only block attests encrypted thinking
476-
* that is not being replayed at all, so it is not one of these blocks and
477-
* cannot contribute the pair.
478-
*/
479-
function assistantThinking(
480-
message: OcxAssistantMessage,
481-
): { thinking?: string; signature?: string } {
482-
const blocks = message.content.filter(
483-
(part): part is Extract<typeof part, { type: "thinking" }> => part.type === "thinking",
484-
).filter(part => Boolean(part.thinking));
485-
if (blocks.length === 0) return {};
486-
const signature = blocks.length === 1 ? blocks[0]!.signature : undefined;
487-
return {
488-
thinking: blocks.map(part => part.thinking).join("\n"),
489-
...(isProviderIssuedThinkingSignature(signature) ? { signature } : {}),
490-
};
491-
}
492-
493461
export function mapOcxMessagesToDevin(parsed: OcxParsedRequest): ChatHistoryItem[] {
494462
const items: ChatHistoryItem[] = [];
495463
// Cognition is not an OpenAI host, and this adapter does advertise a real
@@ -527,10 +495,10 @@ function mapOneMessage(message: OcxMessage): ChatHistoryItem | undefined {
527495
if (message.role === "assistant") {
528496
const toolCalls = assistantToolCalls(message);
529497
const text = assistantText(message);
530-
const reasoning = assistantThinking(message);
498+
const reasoning = devinAssistantReasoning(message);
531499
// A turn that produced only reasoning is still worth replaying: dropping it
532500
// is what makes the next turn re-derive the same chain.
533-
if (!text && toolCalls.length === 0 && !reasoning.thinking) return undefined;
501+
if (!text && toolCalls.length === 0 && !reasoning.thinking && !reasoning.signature) return undefined;
534502
return {
535503
role: "assistant",
536504
content: text || "",
@@ -792,7 +760,7 @@ export function createDevinAdapter(
792760
if (event.kind === "reasoning_signature") {
793761
// Carried back out so the next turn can replay it in the prompt's
794762
// signature field; an unsigned replay is what the service ignores.
795-
emit({ type: "thinking_signature", signature: event.signature });
763+
emit({ type: "thinking_signature", signature: encodeDevinSignature(event.signature, event.signatureType) });
796764
continue;
797765
}
798766
if (event.kind === "tool_call_start") {

‎src/adapters/devin/cloud-direct/chat.ts‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -494,7 +494,7 @@ export type CloudChatEvent =
494494
* turn produced. Without decoding it there is nothing to put in the prompt's
495495
* #12 on the next turn, so the replay would always be unsigned.
496496
*/
497-
| { kind: 'reasoning_signature'; signature: string }
497+
| { kind: 'reasoning_signature'; signature: string; signatureType?: string }
498498
| { kind: 'tool_call_start'; id: string; name: string }
499499
| {
500500
kind: 'tool_call_args';
@@ -817,6 +817,10 @@ export function* decodeChatFrame(proto: Buffer): Generator<CloudChatEvent> {
817817
}
818818
}
819819
if (authoritativeUsage) yield authoritativeUsage;
820+
let signatureType: string | undefined;
821+
for (const f of iterFields(proto)) {
822+
if (f.num === 21 && f.wire === 2 && Buffer.isBuffer(f.value)) signatureType = (f.value as Buffer).toString('utf8') || undefined;
823+
}
820824
for (const f of iterFields(proto)) {
821825
if (f.num === 3 && f.wire === 2 && Buffer.isBuffer(f.value)) {
822826
// Visible delta_text — what the user should SEE in the chat.
@@ -842,7 +846,8 @@ export function* decodeChatFrame(proto: Buffer): Generator<CloudChatEvent> {
842846
if (s) yield { kind: 'reasoning', text: s };
843847
} else if (f.num === 10 && f.wire === 2 && Buffer.isBuffer(f.value)) {
844848
const s = (f.value as Buffer).toString('utf8');
845-
if (s) yield { kind: 'reasoning_signature', signature: s };
849+
// #21 delta_signature_type arrives in the same frame; the prompt replays it as #18.
850+
if (s) yield { kind: 'reasoning_signature', signature: s, ...(signatureType ? { signatureType } : {}) };
846851
} else if (f.num === 6 && f.wire === 2 && Buffer.isBuffer(f.value)) {
847852
let id: string | undefined;
848853
let name: string | undefined;
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
/**
2+
* Devin reasoning signatures across turns.
3+
*
4+
* GetChatMessage returns the turn's reasoning attestation as `delta_signature`
5+
* (#10) together with `delta_signature_type` (#21) in the same frame, and the
6+
* native client replays both on the assistant prompt as #12 and #18. Measured
7+
* live on swe-2-high the pair arrives AFTER the visible answer (reasoning, text,
8+
* then signature), so the Responses layer stores it as its own signature-only
9+
* reasoning item behind the thinking-text item. GPT and Gemini rows stream no
10+
* thinking text at all, only the signature.
11+
*
12+
* The type is carried inside the stored signature because the reasoning
13+
* envelope that round-trips through the client keeps a single signature string.
14+
* A signature stored before this prefix existed replays without a type.
15+
*/
16+
import type { OcxAssistantMessage } from "../../types";
17+
import { isProviderIssuedThinkingSignature } from "../../responses/reasoning-envelope";
18+
19+
const TYPED_SIGNATURE_PREFIX = "devin-sig1:";
20+
21+
export function encodeDevinSignature(signature: string, signatureType: string | undefined): string {
22+
return signatureType && !signatureType.includes(":")
23+
? `${TYPED_SIGNATURE_PREFIX}${signatureType}:${signature}`
24+
: signature;
25+
}
26+
27+
export function decodeDevinSignature(stored: string): { signature: string; signatureType?: string } {
28+
if (!stored.startsWith(TYPED_SIGNATURE_PREFIX)) return { signature: stored };
29+
const rest = stored.slice(TYPED_SIGNATURE_PREFIX.length);
30+
const colon = rest.indexOf(":");
31+
if (colon <= 0) return { signature: stored };
32+
return { signature: rest.slice(colon + 1), signatureType: rest.slice(0, colon) };
33+
}
34+
35+
/**
36+
* The assistant turn's reasoning for ChatMessagePrompt #11/#12/#18.
37+
*
38+
* All of the turn's thinking text is replayed. A signature rides along only
39+
* when it covers exactly that text:
40+
* - one thinking block carrying its own issued signature (a stray
41+
* signature-only block does not displace it);
42+
* - at most one unsigned thinking block plus exactly one signature-only block,
43+
* which is how a single Devin turn arrives once its late #10 frame has been
44+
* split into its own reasoning item. With no thinking block at all this is a
45+
* GPT or Gemini row, where the signature is the only reasoning there is.
46+
* Any other mix (two signed blocks, a signed block beside unsigned text) has no
47+
* single attestation for the joined text, so the turn is replayed unsigned.
48+
*/
49+
export function devinAssistantReasoning(
50+
message: OcxAssistantMessage,
51+
): { thinking?: string; signature?: string; signature_type?: string } {
52+
const blocks = message.content.filter(
53+
(part): part is Extract<typeof part, { type: "thinking" }> => part.type === "thinking",
54+
);
55+
const textBlocks = blocks.filter(part => Boolean(part.thinking));
56+
const signatureOnly = blocks.filter(part => !part.thinking && isProviderIssuedThinkingSignature(part.signature));
57+
const text = textBlocks.map(part => part.thinking).join("\n");
58+
let stored: string | undefined;
59+
if (textBlocks.length === 1 && isProviderIssuedThinkingSignature(textBlocks[0]!.signature)) {
60+
stored = textBlocks[0]!.signature;
61+
} else if (textBlocks.length <= 1 && signatureOnly.length === 1) {
62+
stored = signatureOnly[0]!.signature;
63+
}
64+
const decoded = stored ? decodeDevinSignature(stored) : undefined;
65+
return {
66+
...(text ? { thinking: text } : {}),
67+
...(decoded ? { signature: decoded.signature } : {}),
68+
...(decoded?.signatureType ? { signature_type: decoded.signatureType } : {}),
69+
};
70+
}

‎structure/providers-and-adapters.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ rewrite rules and the routed-id settlement.
138138
| `src/adapters/declaration-carrier.ts`, `src/adapters/input-media-guard.ts` | Default-deny allowlists for constraints the normalized request carries but a wire may not be able to express: `tools[*].allowed_callers`, which fences a tool off from callers, and inline document bytes. Both are refused with a 400 at the single guard every registered adapter passes through, rather than left to each adapter, because an adapter that never learned about the carrier rebuilds without it and answers normally. `allowed_callers` reaches the `anthropic` wire; document bytes reach `anthropic`, `openai-chat` and `google`; the `openai-responses` wire is exempt from the whole guard because it forwards the original body. Adding an `AdapterWire` member makes the omission visible in these lists instead of at a customer's upstream. The unrestricted `["direct"]` caller default is not a restriction. |
139139
| `src/adapters/azure.ts` | Azure OpenAI bridge. |
140140
| `src/adapters/cursor.ts`, `src/adapters/cursor/` | Cursor protobuf transport: discovery, request builder, event decoding, MCP, thread continuity, native-exec policy. |
141-
| `src/adapters/devin.ts`, `src/adapters/devin/cloud-direct/` | Devin runTurn transport over Cognition Connect-RPC. `GetChatMessage` uses the Responses provider executor and shared physical-send budget; catalog, JWT, and `src/web-search/devin-executor.ts` native search support RPCs remain outside inference-send accounting. Provider-stated pre-output 429 reset delays are surfaced immediately by default, releasing shared active-turn capacity. A positive `OPENCODEX_DEVIN_STATED_RESET_WAIT_MS` explicitly enables bounded waiting and up to two replays on standalone turns, which hold that capacity until completion or cancellation. Combo children bypass that wait and surface a pre-output 429 so the next target can run. During an opted-in standalone wait, safe heartbeats commit the response preflight and keep the stream's stall watchdog fed. Invalid values fail closed to the immediate-refusal behavior. A recorded tenant host is used only for the stored account whose credential owns the transmitted key, searched in the configured provider id and then its deprecated alias; a configured, forwarded, or unmatched key uses the configured base URL or the US default. Native search previews the current route by effective adapter without mutating combo selection state, pins one admitted active-account snapshot for the request, and calls `GetWebSearchResults`, so it starts no CLI or second model. The wire model UID comes from the catalog's family metadata (`ClientModelConfig` #23/#30/#31): a family id with no effort anchors on the family's default member and selects the nearest enabled rung, rounding up first; an effort moves only the effort axis, to the lowest rung at or above it (else the highest below) while Fast Mode, 1M Context and the other axes stay at the anchor's values unless the caller asked for `fast` or a `1m` value; not lowering a requested effort outranks keeping those axes (an unranked member counts as lower), a disabled row the caller named is kept when the request still selects it so the preflight names that refusal, and resolution never leaves the family. Rows without family metadata fall back to suffix resolution, whose variant scan matches the collapsed base rather than a string prefix. With no caller or configured output cap, the selected row's catalog `maxOutputTokens` fills CompletionConfiguration #2; #3 is `max_newlines` and is sent at a fixed value, never a context window. The leading system text is sent as `GetChatMessage` #2, a failed tool result sets ChatMessagePrompt #9 and keeps an in-band `ERROR:` marker, and Gemini uids have JSON-Schema type arrays split into per-type `anyOf` branches in tool parameters while preserving outer enum/const, boolean constraints (including `not`, `oneOf`, and `allOf`) on null, and existing null-branch restrictions. A pre-output `invalid_argument` on a history whose word-piece estimate, using the sanitized and truncated tool descriptions actually sent on the wire, reaches 95% of the selected UID's catalog input window, capped by configured provider and model limits (512 KiB of text only when no window is known) is surfaced as `context_length_exceeded` so Codex compacts; a small request with the same code stays a plain 400. Known limit: a malformed schema on a history already at or above that 95% threshold is also reported as overflow because the upstream returns the same `invalid_argument`. |
141+
| `src/adapters/devin.ts`, `src/adapters/devin/cloud-direct/` | Devin runTurn transport over Cognition Connect-RPC. Assistant reasoning replays as ChatMessagePrompt #11 thinking, #12 signature and #18 signature type (`src/adapters/devin/reasoning-signature.ts`): the #10/#21 pair arrives after the visible answer and becomes its own signature-only reasoning item, so a single unsigned thinking block plus exactly one signature-only block is replayed as one signed prompt, and a signature-only turn (GPT, Gemini) is replayed rather than dropped. `GetChatMessage` uses the Responses provider executor and shared physical-send budget; catalog, JWT, and `src/web-search/devin-executor.ts` native search support RPCs remain outside inference-send accounting. Provider-stated pre-output 429 reset delays are surfaced immediately by default, releasing shared active-turn capacity. A positive `OPENCODEX_DEVIN_STATED_RESET_WAIT_MS` explicitly enables bounded waiting and up to two replays on standalone turns, which hold that capacity until completion or cancellation. Combo children bypass that wait and surface a pre-output 429 so the next target can run. During an opted-in standalone wait, safe heartbeats commit the response preflight and keep the stream's stall watchdog fed. Invalid values fail closed to the immediate-refusal behavior. A recorded tenant host is used only for the stored account whose credential owns the transmitted key, searched in the configured provider id and then its deprecated alias; a configured, forwarded, or unmatched key uses the configured base URL or the US default. Native search previews the current route by effective adapter without mutating combo selection state, pins one admitted active-account snapshot for the request, and calls `GetWebSearchResults`, so it starts no CLI or second model. The wire model UID comes from the catalog's family metadata (`ClientModelConfig` #23/#30/#31): a family id with no effort selects the family's default member, an effort moves only the effort axis, to the lowest rung at or above it (else the highest below) while Fast Mode, 1M Context and the other axes stay at the anchor's values unless the caller asked for `fast` or a `1m` value; not lowering a requested effort outranks keeping those axes (an unranked member counts as lower), a disabled row the caller named is kept when the request still selects it so the preflight names that refusal, and resolution never leaves the family. Rows without family metadata fall back to suffix resolution, whose variant scan matches the collapsed base rather than a string prefix. With no caller or configured output cap, the selected row's catalog `maxOutputTokens` fills CompletionConfiguration #2; #3 is `max_newlines` and is sent at a fixed value, never a context window. The leading system text is sent as `GetChatMessage` #2, a failed tool result sets ChatMessagePrompt #9 and keeps an in-band `ERROR:` marker, and Gemini uids have JSON-Schema type arrays split into per-type `anyOf` branches in tool parameters while preserving outer enum/const and existing null-branch restrictions. A pre-output `invalid_argument` on a history whose word-piece estimate reaches 95% of the selected UID's catalog input window, capped by configured provider and model limits (512 KiB of text only when no window is known) is surfaced as `context_length_exceeded` so Codex compacts; a small request with the same code stays a plain 400. Known limit: a malformed schema on a history already at or above that 95% threshold is also reported as overflow because the upstream returns the same `invalid_argument`. |
142142
| `src/adapters/kiro.ts` and `src/adapters/kiro/` | Kiro event/tool/thinking/truncation/retry handling, including an egress-aware completion fallback, fixed public HTTP 5xx text, and closed-set status/code diagnostics. The original path is a facade over leaves for wire identity, reasoning, conversation state, token estimation, payload assembly, streaming, and the adapter. |
143143
| `src/adapters/mimo-free.ts` | Mimo Free transport (client identity + JWT). Concurrent requests share one JWT bootstrap bound only to its timeout; each request stops waiting on its own abort without cancelling the others. |
144144
| `src/adapters/command-code.ts`, `src/adapters/command-code-tool-text.ts`, `src/adapters/command-code-restored-schema.ts` | Command Code OAuth NDJSON translation. For every `xiaomi/mimo-` model, text, native calls, reasoning, and terminal decisions share one byte-bounded queue with linear queue visits. Markup is deduplicated against matching native calls; text-only restoration requires one contiguous text run, a clean finish, a declared tool, and arguments validated against supported schema constraints. A parameter-free (freeform) block may omit `</function>` but must end with `</tool_call>`; parameter blocks keep the canonical close. Markup appended after prose in the same delta is split off at the marker and held like a block that opens with `<tool_call>`; a marker split across deltas after prose is still released as text. Native, reasoning, and other intervening events interrupt a still-probing block but leave a held block held in arrival order, and the queued byte bound still flushes an unresolved envelope as text. An envelope the strict parser rejects but that opens with `<tool_call>`, closes with `</tool_call>`, and names a declared function is dropped when a native call for that same function arrives and on a clean finish; markup that parses but fits no supported schema is still released as text. Regex patterns, other unsupported constraints, and abnormal finishes fail closed. `tests/providers/command-code-tool-text-prose-split.test.ts` covers the split, the interleaved-event hold, and both drop paths. |

0 commit comments

Comments
 (0)