Skip to content

Commit 447ac22

Browse files
lidge-junmdwsk88luvs01
authored
fix(adapters,server): output budgets, adapter queue memory, and per-key model policy (#5265)
* fix(devin): send the configured output budget instead of the encoder default Codex never sends max_output_tokens, and the devin adapter only forwarded a caller-supplied value, so every devin turn was capped at the cloud-direct encoder's 8192 fallback however the provider was configured. A turn that legitimately needed more ended as an upstream incomplete/max_output_tokens the client then retried into the same deterministic wall. Resolve the cap the way the other adapters do, highest authority first: an explicit caller value forwarded unchanged, then the configured per-model cap, then the provider default, then the encoder fallback. The lookup is the same UID-aware hint resolution the input ceiling already uses, extracted so both sides read a per-model number identically. The output cap and the history ceiling stay separate. CompletionConfiguration #2 is the output cap and #3 is the context window, so the resolver reads neither contextWindow nor modelContextWindows -- collapsing them would ask Cognition to generate a whole context window of output. Also stop OAuth startup reconciliation from deleting an operator's output budget. No OAuth preset declares defaultMaxOutputTokens or modelMaxOutputTokens, so the delete-when-preset-undefined branch was the only branch either field ever took and a hand-edited value was wiped before the next startup finished. A preset that does declare one still refreshes the row. Closes #5190 * feat(coding-agent): derive the projected-history ceiling from the model context window The coding-agent CLIs replay the whole conversation each turn as one projected user message over stream-json stdin, and the projection had a flat 200k-character history ceiling: roughly 50k tokens of English or code, a fraction of what even a 128k-token model holds and far below the 1M-token families. Long sessions lost their earliest context at a bound unrelated to the model. Derive the ceiling from the declared model context window on the routed provider row -- modelContextWindows by model, then the provider-wide contextWindow -- at three characters per token, floored at the legacy 200k cap so small windows and missing metadata behave exactly as before, and capped by a 4M hard ceiling so runaway metadata cannot unbound stdin. The resolution lives once in the shared runCodingAgentTurn driver, so both CodeBuddy and Qoder turns get it and the family adapters are unchanged. This ceiling is a runaway-memory bound on replayed history, measured in characters. It is deliberately not the output budget: caller-side compaction remains the token authority, and nothing here decides how long a reply may run. Co-authored-by: mdwsk88 <[email protected]> * fix(adapters): bound the event queue by retained payload, not by event count alone The adapter event queue capped how many events it buffered but never how much those events held. Coalescing merges adjacent deltas up to 64 KiB an item, so the old 1024-event cap admitted about 64 MiB of retained text before it said anything, and a single oversized event was unbounded on its own. Charge what the queue actually retains, against two separate budgets. The aggregate budget bounds everything held at one moment; the per-event budget bounds one event and applies however empty the queue is. They describe different failures -- a consumer that is not keeping up versus an event that is malformed -- so they report different terminal messages and an operator can tell which happened. The accounting is exact on every path. Each queued item records what it was charged, so a merge pays only for the text it appends, a dequeue gives back precisely what it took, a refused event is priced before anything is retained and never charged, and the terminal record that explains a refusal is admitted past the budget it reports but still charged and released. Draining therefore returns the counter to zero after a normal turn, after an overflow abort and after a consumer walks away mid-stream; retainedCodeUnits() exposes that so a regression can assert it rather than infer it from an abort that happened to fire. A long healthy stream is still not capped by its total length: every dequeue releases its charge, so only an undrained backlog accumulates. The aggregate default is sized for the other legitimate case -- a synchronous producer that fills the queue before its consumer is scheduled, as the image loop does with over a million one-character deltas -- which is roughly 1.2 MB of retained text and must not abort. Retention is measured by walking own enumerable properties rather than by naming each variant's string fields, because a hand-written per-variant table is exhaustive over the AdapterEvent union and would silently stop counting a member added on another branch. The walk carries depth and node ceilings so one push stays cheap against the open provider-shaped payloads two members carry. Co-authored-by: luvs01 <[email protected]> * feat(server): scope an admission key to the models and providers it may reach A hub serving several clients with their own data-plane keys had no way to stop a mail or cron key spending a coding key's Grok or Claude quota. Hub-level model selection cannot express it: it hides a model from everyone or from no one. An admission key may now declare allowedProviders and allowedModels. Absent or empty means unrestricted, which is every existing key, so nothing changes until an operator sets one. Two rules decide what this is. A scope names destinations, not selectors: it is evaluated against the resolved provider and model a turn will actually bill, never against the string the client sent. Alias resolution, a policy or combo selection, a subagent fallback and a compaction override all rewrite that string, so a scope checked at the front door would authorize one destination and reach another. And a scope is never management authority -- it narrows which models an inference key may call and grants nothing else. Enforcement sits where each route becomes concrete. On the Responses path that is the single point every produced route passes through, which covers the direct name, an alias, a policy or combo child, a shadow-intercept target, both subagent-fallback re-routes and, through translation, the Chat and Messages surfaces. The native Chat lane and the compaction route send without re-entering that path, so each applies the same predicate itself. A refusal is 403 with a stable model_not_allowed_for_key type naming the caller's own selector; the resolved destination stays in the server log, because a key that may not reach a provider has no business learning that its alias points there. /v1/models filters by the same predicate, so what a key can see and what it can call cannot diverge. That filter is a convenience and not the boundary: hiding a row only stops a client that reads the catalog first. A malformed scope drops the key rather than degrading to undefined, unlike every other field on the record. Degrading a damaged permission field reads as "allowed everything", which is the one direction it must never fail. Management exposes the lists on GET /api/keys and accepts them on PATCH, where rename and scope are independent edits, and ocx access key get/set reads and writes them without printing or rotating the secret. Closes #5049 * docs(devlog): record lane E of the phase 2 consolidation batch * fix(server,adapters): close the virtual-model scope gap and harden queue retention Three findings from an adversarial review of this branch. The scope check ran before applyOpenAiVirtualModel, which rewrites route.modelId to the wire id that is actually billed. A key allowing only the public selector was therefore authorized on one model and sent on another. The settled route is now re-checked after normalization, so the id that is billed is the id that was authorized. The account-qualified branch of /v1/alpha/search resolves a model through the router and bills the account it names, so it applies the same rule. The endpoints that spend quota without routing a model -- images, audio, realtime, and the non-account-qualified search branch -- are recorded in the lane document as uncovered rather than left to read as covered. The queue's per-event budget comment claimed it bound any single event. It bounds a RETAINED one: an event handed straight to a waiting consumer is never held, so refusing it would abort a turn over memory this queue does not own. The comment now says what the code does. retainedEventCodeUnits also guards a non-object, so a malformed adapter emission becomes a terminal event rather than a TypeError thrown out of push with the queue half-updated. The scope regression reached the config schema through config/schema/leaf-validators directly, which enters that module cycle from the wrong end and threw a TDZ ReferenceError on CI. It now loads a hand-written config.json through src/config the way a startup does, which also proves the stronger property: a damaged permission field drops that key alone and its valid neighbour survives. --------- Co-authored-by: mdwsk88 <[email protected]> Co-authored-by: luvs01 <[email protected]>
1 parent 8e1fdea commit 447ac22

25 files changed

Lines changed: 1495 additions & 68 deletions
Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# Lane E — output budgets, queue memory, per-key policy
2+
3+
Status: OPEN. Branch `codex/260920-lane-e-budgets-key-policy` against `dev`, one pull request.
4+
Covers phase 2 bundles 10, 11 and 12 from [010_phase2.md](010_phase2.md).
5+
6+
## What each bundle turned out to be
7+
8+
### 10 — Devin output budget and the history ceiling
9+
10+
Two defects, not one. The adapter forwarded only a caller-supplied
11+
`max_output_tokens`, and Codex never sends one, so every `devin/*` turn was capped at
12+
the cloud-direct encoder's 8192 fallback however the provider was configured. The
13+
escape hatch was closed too: no OAuth preset declares `defaultMaxOutputTokens` or
14+
`modelMaxOutputTokens`, so the delete-when-preset-undefined branch in
15+
`applyOAuthPresetCatalog` was the only branch either field ever took and a
16+
hand-edited value was gone before the next startup finished. Both had to move, or
17+
wiring the adapter alone would have been unreachable in practice.
18+
19+
The resolver reads the caller's explicit value, then the configured per-model cap,
20+
then the provider default, then nothing — leaving the encoder fallback. It never
21+
reads `contextWindow` or `modelContextWindows`: CompletionConfiguration #2 is the
22+
output cap and #3 is the context window, and collapsing them would ask Cognition to
23+
generate a whole window of output.
24+
25+
The history ceiling is the other half and stays a separate quantity. #5189 is
26+
carried with attribution: it derives the coding-agent projected-history bound from
27+
the declared context window in characters. That bounds replayed history memory;
28+
nothing there decides how long a reply may run.
29+
30+
No retry change was needed. Source review of `stated-reset-retry.ts` and
31+
`upstream-retry.ts` confirms an upstream `incomplete / max_output_tokens` is a
32+
successful streaming response that has already emitted events, so it matches none
33+
of the replay conditions. The repeated identical attempts in #5190 are the client's.
34+
35+
### 11 — adapter event queue memory
36+
37+
PR #5182 had the right idea and the wrong number. Its 1 MiB aggregate default
38+
aborts a legitimate turn: a synchronous producer fills the queue before its
39+
consumer is scheduled, and the image loop does exactly that with over a million
40+
one-character deltas that coalesce into roughly 1.2 MB of retained text. Its own CI
41+
proved it, which is why it sits at `CHANGES_REQUESTED`.
42+
43+
The work is carried with attribution and reshaped around two budgets rather than
44+
one, because a stalled consumer and a malformed event are different failures and an
45+
operator reading the terminal error should learn which happened. Accounting is now
46+
exact by construction: each queued item records what it was charged, so a merge
47+
pays only for appended text, a refused event is priced before anything is retained
48+
and never charged, and the terminal record explaining a refusal is admitted past
49+
the budget it reports but still charged and released. `retainedCodeUnits()` exposes
50+
the counter so the regressions assert it reaches zero rather than inferring it from
51+
an abort that happened to fire.
52+
53+
Retention is measured by a bounded walk of own enumerable properties rather than a
54+
per-variant table. A table would be exhaustive over `AdapterEvent`, which is the
55+
union class `AGENTS.md` records: a member added on another branch would silently
56+
stop being counted.
57+
58+
### 12 — per-admission-key model and provider scope
59+
60+
The security question is where the check goes, not what it compares. A scope
61+
evaluated against the client's string authorizes one destination and reaches
62+
another, because alias resolution, policy and combo selection, subagent fallback
63+
and compaction override all rewrite that string. So the scope names destinations
64+
and is applied to the resolved route.
65+
66+
On the Responses path every route produced by the request — direct name, alias,
67+
policy, combo child, shadow-intercept target and both subagent-fallback re-routes —
68+
passes through one capture point, which is where the check sits. Chat and Messages
69+
translate into that path; their native lanes and the compaction route send without
70+
re-entering it, so each applies the same predicate itself. `/v1/models` filters by
71+
the same predicate, and that filter is explicitly not the boundary.
72+
73+
A malformed scope drops the key rather than degrading to `undefined` like every
74+
other field on the record, because degrading a permission field reads as "allowed
75+
everything".
76+
77+
Out of scope and deliberately not started: Redis, a full multi-tenant conversion,
78+
and any budget or RPM/TPM system.
79+
80+
#### What the scope does not cover, stated rather than implied
81+
82+
An adversarial review of the branch found authenticated data-plane endpoints that
83+
spend provider quota without resolving a model through the router, so the scope
84+
does not reach them:
85+
86+
- `/v1/images/generations` and `/v1/images/edits`,
87+
- `/v1/audio/transcriptions` and its streaming form,
88+
- `/v1/live`, `/v1/realtime/calls` and the standalone realtime sockets,
89+
- the non-account-qualified branch of `/v1/alpha/search`, which forwards the caller's
90+
model to a search sidecar without routing it.
91+
92+
The account-qualified search branch does route a model and is checked. The rest
93+
need a destination definition this lane does not own — an image or audio endpoint
94+
has a fixed-purpose model rather than a routed one — and inventing one here would
95+
be the multi-tenant expansion this batch rules out. They are recorded so the
96+
contract is not read as broader than it is.
97+
98+
The review also found that resolving an OpenAI virtual model rewrites
99+
`route.modelId` to the wire id after the initial check. That one was a real hole in
100+
the stated contract and is fixed: the settled route is re-checked after
101+
normalization, so the id that is billed is the id that was authorized.
102+
103+
## Verification
104+
105+
Static source review plus exact-head hosted CI. No local suite, individual test,
106+
typecheck, build, install or live `ocx` execution was run — those are NOT RUN, not
107+
passing.
108+
109+
Union-defect classes checked before pushing. No file in the touched set carries a
110+
`file-size-baseline.json` cap; the largest, `src/oauth/index.ts` and
111+
`src/server/index/serve-options.ts`, stay under the 2000-line new-file threshold.
112+
The two new test files are registered in both `scripts/test-layout/layout.json` and
113+
`tests/fixtures/test-layout-expected.json`. Nothing here restates a count or
114+
enumerates a union.
115+
116+
## Ownership
117+
118+
Lane E owns the adapter event queue budget and the Devin and coding-agent limits.
119+
Retry classification — `sendCount`, the send budget, the stage and cause vocabulary —
120+
is lane C's and is untouched.
121+
122+
## Carried work
123+
124+
- #5182 (luvs01) — adapter event queue backlog budget.
125+
- #5189 (mdwsk88) — coding-agent projected-history ceiling.
126+
127+
Both carry a `Co-authored-by` trailer in the branch commit. Neither original pull
128+
request is closed here; the coordinator handles that after this lane lands.

‎scripts/test-layout/layout.json‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1566,7 +1566,9 @@
15661566
"web-search-sidecar-429.test.ts": "web-search",
15671567
"management-google-tool-schema-policy.test.ts": "server",
15681568
"codex-shim-destroyed-probe.test.ts": "codex-integration",
1569-
"client-runtime.test.ts": "clients"
1569+
"client-runtime.test.ts": "clients",
1570+
"devin-output-budget.test.ts": "providers",
1571+
"api-key-model-scope.test.ts": "server"
15701572
},
15711573
"migrated": [
15721574
"adapters",

‎src/adapters/coding-agent/protocol.ts‎

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,30 @@ export const MAX_STREAM_LINE_BYTES = 8 * 1024 * 1024;
2121
export const MAX_STREAM_TOTAL_BYTES = 64 * 1024 * 1024;
2222
/** Hard ceiling on projected conversation history text (characters) to prevent runaway memory. */
2323
export const MAX_PROJECTED_HISTORY_CHARS = 200_000;
24+
/**
25+
* Chars-per-token ratio for deriving the projected-history ceiling from the model context
26+
* window. Sits between the English/code (~4 chars per token) and CJK (~1.5) extremes: the
27+
* ceiling is a runaway-memory bound and a coarse guard against cutting history the window can
28+
* hold, not a token accounting - the caller-side compaction line stays the token authority.
29+
*/
30+
const PROJECTED_HISTORY_CHARS_PER_TOKEN = 3;
31+
/** Absolute ceiling on a window-derived history cap, so runaway metadata cannot unbound stdin. */
32+
const MAX_PROJECTED_HISTORY_DERIVED_CHARS = 4_000_000;
33+
34+
/**
35+
* Projected-history character ceiling for a turn, derived from the declared model context
36+
* window. A missing or non-finite window keeps the legacy flat cap, and the derivation never
37+
* lowers the cap below it: small windows change nothing, while large windows scale (a 1M-token
38+
* model keeps 3M characters) until the hard ceiling. The flat 200k cap predates window
39+
* metadata and cut long replays to roughly 50k-130k tokens of content regardless of the model.
40+
*/
41+
export function projectedHistoryCharLimit(contextWindowTokens: number | undefined): number {
42+
if (typeof contextWindowTokens !== "number" || !Number.isFinite(contextWindowTokens) || contextWindowTokens <= 0) {
43+
return MAX_PROJECTED_HISTORY_CHARS;
44+
}
45+
const derived = contextWindowTokens * PROJECTED_HISTORY_CHARS_PER_TOKEN;
46+
return Math.min(Math.max(derived, MAX_PROJECTED_HISTORY_CHARS), MAX_PROJECTED_HISTORY_DERIVED_CHARS);
47+
}
2448

2549
export class CodingAgentStreamLimitError extends Error {
2650
constructor(message: string) {
@@ -395,7 +419,7 @@ export function buildSystemPrompt(parsed: OcxParsedRequest): string | undefined
395419
* prior conversation turns are structured as bounded context text with tool results as text,
396420
* clearly demarcated from the current user request. Codex retains tool control; vendor tools are never invoked.
397421
*/
398-
export function buildConversationInput(parsed: OcxParsedRequest): string[] {
422+
export function buildConversationInput(parsed: OcxParsedRequest, options: { maxHistoryChars?: number } = {}): string[] {
399423
const nonDev = parsed.context.messages.filter(m => m.role !== "developer");
400424
if (nonDev.length === 0) {
401425
return [JSON.stringify({ type: "user", message: { role: "user", content: [{ type: "text", text: "" }] } })];
@@ -475,10 +499,11 @@ export function buildConversationInput(parsed: OcxParsedRequest): string[] {
475499

476500
const imageBlocks: WireContentPart[] = [...historyImageBlocks, ...currentImageBlocks];
477501

502+
const maxHistoryChars = options.maxHistoryChars ?? MAX_PROJECTED_HISTORY_CHARS;
478503
let historyText = historyMessages.map(formatMessageForHistory).filter(Boolean).join("\n\n");
479-
if (historyText.length > MAX_PROJECTED_HISTORY_CHARS) {
504+
if (historyText.length > maxHistoryChars) {
480505
historyText = `[Earlier conversation history truncated for length...]\n\n` +
481-
historyText.slice(historyText.length - MAX_PROJECTED_HISTORY_CHARS);
506+
historyText.slice(historyText.length - maxHistoryChars);
482507
}
483508

484509
const combinedText = `Prior conversation context:\n\n${historyText}\n\nCurrent user request:\n\n${currentRequestText}`;

‎src/adapters/coding-agent/turn.ts‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
import { execFileSync, spawn as nodeSpawn, type ChildProcess, type SpawnOptions } from "node:child_process";
22
import type { AdapterEvent, OcxParsedRequest, OcxProviderConfig } from "../../types";
33
import { commandInvocation } from "../../lib/win-exec";
4+
import { modelRecordValue } from "../../reasoning-effort";
45
import type { IncomingMeta } from "../base";
5-
import { buildConversationInput, CodingAgentProtocolError, mapStreamMessageToEvents, readJsonLines, type StreamParseState } from "./protocol";
6+
import { buildConversationInput, CodingAgentProtocolError, mapStreamMessageToEvents, projectedHistoryCharLimit, readJsonLines, type StreamParseState } from "./protocol";
67
import { resolveCodingAgentBinary, resolveProfileByBaseUrl, type CodingAgentProviderProfile, type WhichFn } from "./profile";
78

89
/** Injectable spawn for tests; production uses node:child_process. */
@@ -264,7 +265,14 @@ export async function runCodingAgentTurn(input: CodingAgentTurnInput): Promise<v
264265
const stdin = child.stdin;
265266
if (stdin) {
266267
stdin.on("error", () => { /* EPIPE if the CLI exits early; surfaced via close/stderr */ });
267-
for (const line of buildConversationInput(parsed)) stdin.write(`${line}\n`);
268+
// The projected history scales with the model context window on the routed provider row
269+
// (catalog and config metadata merged): a 1M-token model keeps 3M characters of replay
270+
// where the flat cap cut it near 50k-130k tokens of content. Absent metadata keeps the
271+
// flat cap.
272+
const historyCharLimit = projectedHistoryCharLimit(
273+
modelRecordValue(provider.modelContextWindows, parsed.modelId) ?? provider.contextWindow,
274+
);
275+
for (const line of buildConversationInput(parsed, { maxHistoryChars: historyCharLimit })) stdin.write(`${line}\n`);
268276
stdin.end();
269277
}
270278
const stdout = child.stdout;

‎src/adapters/devin.ts‎

Lines changed: 72 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,35 @@ async function resolveWireModelUid(
192192
*/
193193
export const resolveWireModelUidForTests = resolveWireModelUid;
194194

195+
const positiveTokenCount = (value: unknown): number | undefined =>
196+
typeof value === "number" && Number.isSafeInteger(value) && value > 0 ? value : undefined;
197+
198+
/**
199+
* Read a per-model token count for the exact UID selected for this turn.
200+
*
201+
* Tries the selected UID and then its collapsed base id, preferring the
202+
* canonical spelling and accepting dotted or case-folded saved hints — the same
203+
* normalization the inference request applies to the model id. Where several
204+
* spellings match one id, the smallest wins: a ceiling stated twice is
205+
* satisfied by the lower statement.
206+
*/
207+
function devinModelTokenHint(
208+
record: Record<string, number> | undefined,
209+
modelUid: string,
210+
): number | undefined {
211+
if (!record) return undefined;
212+
for (const id of [modelUid, collapseDevinModelUid(modelUid)]) {
213+
const exact = Object.hasOwn(record, id) ? positiveTokenCount(record[id]) : undefined;
214+
if (exact !== undefined) return exact;
215+
const matches = Object.entries(record)
216+
.filter(([key]) => normalizeDevinModelId(key).toLowerCase() === id.toLowerCase())
217+
.map(([, value]) => positiveTokenCount(value))
218+
.filter((value): value is number => value !== undefined);
219+
if (matches.length > 0) return Math.min(...matches);
220+
}
221+
return undefined;
222+
}
223+
195224
/**
196225
* Resolve the INPUT ceiling for the exact UID selected for this turn. Catalog
197226
* ClientModelConfig #18 and CompletionConfiguration #3 both carry input tokens;
@@ -204,33 +233,50 @@ function resolveDevinMaxInputTokens(
204233
modelUid: string,
205234
liveWindow?: number,
206235
): number | undefined {
207-
const positive = (value: unknown): number | undefined =>
208-
typeof value === "number" && Number.isSafeInteger(value) && value > 0 ? value : undefined;
209-
const baseId = collapseDevinModelUid(modelUid);
210-
const configured = (record: Record<string, number> | undefined): number | undefined => {
211-
if (!record) return undefined;
212-
for (const id of [modelUid, baseId]) {
213-
// Prefer the canonical spelling; retain dotted/case-folded saved hints,
214-
// matching the model-id normalization used for the inference request.
215-
const exact = Object.hasOwn(record, id) ? positive(record[id]) : undefined;
216-
if (exact !== undefined) return exact;
217-
const matches = Object.entries(record)
218-
.filter(([key]) => normalizeDevinModelId(key).toLowerCase() === id.toLowerCase())
219-
.map(([, value]) => positive(value))
220-
.filter((value): value is number => value !== undefined);
221-
if (matches.length > 0) return Math.min(...matches);
222-
}
223-
return undefined;
224-
};
225-
const contextHint = configured(provider.modelContextWindows) ?? positive(provider.contextWindow);
226-
const inputHint = configured(provider.modelMaxInputTokens);
227-
const ceilings = [positive(liveWindow), contextHint, inputHint]
236+
const contextHint = devinModelTokenHint(provider.modelContextWindows, modelUid)
237+
?? positiveTokenCount(provider.contextWindow);
238+
const inputHint = devinModelTokenHint(provider.modelMaxInputTokens, modelUid);
239+
const ceilings = [positiveTokenCount(liveWindow), contextHint, inputHint]
228240
.filter((value): value is number => value !== undefined);
229241
return ceilings.length > 0 ? Math.min(...ceilings) : undefined;
230242
}
231243

232-
/** Pure test seam; runtime uses the same resolver immediately before dispatch. */
244+
/**
245+
* Resolve the OUTPUT ceiling for this turn, highest authority first:
246+
*
247+
* 1. the caller's explicit `max_output_tokens`, forwarded unchanged — an
248+
* explicit cap is a request, so a small one is never widened into a
249+
* configured larger one;
250+
* 2. the configured per-model cap (`modelMaxOutputTokens`), read through the
251+
* same UID-aware hint lookup the input ceiling uses;
252+
* 3. the provider-wide `defaultMaxOutputTokens`;
253+
* 4. undefined, which leaves the cloud-direct encoder's own 8192 fallback in
254+
* place for a provider that configured nothing.
255+
*
256+
* This is NOT the history ceiling, and the two must not collapse into one
257+
* number. CompletionConfiguration #2 is the output cap and #3 is the context
258+
* window, so feeding a context window into this resolver would ask Cognition to
259+
* generate a whole window's worth of output. Nothing here reads
260+
* `contextWindow` or `modelContextWindows` for that reason.
261+
*
262+
* Step 1 keeps the caller's raw value rather than `positiveTokenCount`: the
263+
* inbound parser owns what a caller may send, and re-filtering here would
264+
* silently promote a rejected value to a configured cap the caller never asked
265+
* for.
266+
*/
267+
function resolveDevinMaxOutputTokens(
268+
provider: OcxProviderConfig,
269+
modelUid: string,
270+
requested: number | undefined,
271+
): number | undefined {
272+
if (typeof requested === "number") return requested;
273+
return devinModelTokenHint(provider.modelMaxOutputTokens, modelUid)
274+
?? positiveTokenCount(provider.defaultMaxOutputTokens);
275+
}
276+
277+
/** Pure test seams; runtime uses the same resolvers immediately before dispatch. */
233278
export const resolveDevinMaxInputTokensForTests = resolveDevinMaxInputTokens;
279+
export const resolveDevinMaxOutputTokensForTests = resolveDevinMaxOutputTokens;
234280

235281
export class DevinMissingCredentialError extends Error {
236282
constructor() {
@@ -590,6 +636,9 @@ export function createDevinAdapter(
590636
const maxInputTokens = resolveDevinMaxInputTokens(
591637
provider, modelUid, catalog?.byUid.get(modelUid)?.contextWindow,
592638
);
639+
const maxOutputTokens = resolveDevinMaxOutputTokens(
640+
provider, modelUid, parsed.options.maxOutputTokens,
641+
);
593642
// The reset-retry wrapper waits out a 429 that states its own recovery
594643
// delay ("limit will reset in 35 seconds") and replays the identical
595644
// request — but only while zero events have been yielded, so a
@@ -606,7 +655,7 @@ export function createDevinAdapter(
606655
// input hint used to force every model through the 128k default.
607656
completionOpts: {
608657
...(maxInputTokens !== undefined ? { maxInputTokens } : {}),
609-
...(typeof parsed.options.maxOutputTokens === "number" ? { maxOutputTokens: parsed.options.maxOutputTokens } : {}),
658+
...(maxOutputTokens !== undefined ? { maxOutputTokens } : {}),
610659
...(typeof parsed.options.temperature === "number" ? { temperature: parsed.options.temperature } : {}),
611660
...(typeof parsed.options.topP === "number" ? { topP: parsed.options.topP } : {}),
612661
},

0 commit comments

Comments
 (0)