Skip to content

Commit 43a77cb

Browse files
committed
feat(codex): add per-model auto-compaction budgets
1 parent 8b67220 commit 43a77cb

22 files changed

Lines changed: 1154 additions & 65 deletions
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Per-model ChatGPT auto-compaction budgets
2+
3+
## Problem
4+
5+
The catalog currently derives `auto_compact_token_limit` from one fixed rule: 90% of the
6+
advertised context window, clamped by any measured max-input ceiling. That is a sound fallback,
7+
but it cannot express that Sol, Terra, Luna, and Daybreak may need different working reserves.
8+
Changing the hard context window is the wrong control: it also changes admission and model
9+
capability reporting.
10+
11+
OpenCodex also cannot honestly choose a new threshold after inspecting each request. Codex reads
12+
the catalog before the request, decides when to issue `/responses/compact`, and owns the history
13+
that replaces the pre-compaction conversation. The proxy sees the ordinary request only after that
14+
decision. Rewriting a shared catalog per request would race unrelated tasks, while silently issuing
15+
a second model call would not replace the client's history and could duplicate or lose context.
16+
17+
## Contract
18+
19+
Add a provider setting:
20+
21+
```http
22+
PATCH /api/providers?name=openai
23+
Content-Type: application/json
24+
25+
{
26+
"modelAutoCompactTokenLimits": {
27+
"gpt-5.6-sol": 800000,
28+
"gpt-5.6-terra": 810000,
29+
"gpt-5.6-luna": 760000,
30+
"gpt-daybreak-blue-latest": 700000
31+
}
32+
}
33+
```
34+
35+
The same map may be stored alongside the canonical provider fields in
36+
`providers.openai.modelAutoCompactTokenLimits`.
37+
38+
The map is model-specific and soft-policy only:
39+
40+
- each value may lower, never raise, the model's derived/default budget;
41+
- hard `context_window`, `max_context_window`, and measured max-input admission stay authoritative;
42+
- the provider-wide `providerContextCaps.openai` remains the final ceiling;
43+
- bare, account-qualified, pinned/fallback, custom native-alias, and combo rows use the same value;
44+
- a combo chooses the earliest effective budget among all possible targets;
45+
- malformed, unsafe, unknown, or selector-qualified canonical OpenAI keys are rejected atomically.
46+
47+
The native family defaults to a 272,000-token advertised window, so its default effective budget is
48+
244,800 tokens (90%). When the existing native-window control opts a model into the measured
49+
922,000-token ceiling, its derived budget becomes 829,800 tokens. All four models can then be tuned
50+
independently below their own effective window; a soft budget never opts a model into a wider hard
51+
window and we do not invent different product defaults without measurements.
52+
53+
## Future adaptive profiles
54+
55+
True task-aware compaction belongs behind an explicit Codex-client capability. A future client can
56+
select a deterministic profile (for example balanced, tool/output-heavy, or multi-agent), calculate
57+
a request/session budget, and clamp it below this model-specific ceiling. Structural signals such as
58+
tool-schema size, images, output reservation, and collaboration mode are reproducible; asking the
59+
model to judge its own threshold is not suitable for enforcement because it adds a prompt-injectable,
60+
nondeterministic call exactly when context is scarce.
61+
62+
Until such a capability exists, this per-model setting is the safe static authority and fallback.

‎src/codex/catalog/aggregation.ts‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ import { CODEX_REASONING_LEVELS, codexEffortRank, configuredReasoningEfforts, mo
1212
import { getModelMetadata, getModelMetadataCaseInsensitive, listModelMetadata, resolveMetadataProvider } from "../../generated/model-metadata";
1313
import { enrichProviderFromRegistry, shouldCaseFoldMetadataModelId } from "../../providers/derive";
1414
import { getProviderRegistryEntry } from "../../providers/registry";
15-
import { applyProviderContextCap, providerContextCap } from "../../providers/context-cap";
15+
import { applyProviderContextCap, clampAutoCompactTokenLimit, providerContextCap } from "../../providers/context-cap";
1616
import { routedSlug, slugEquals, slugEquivalenceKey, slugsEquivalent } from "../../providers/slug-codec";
1717
import { CODEX_GPT5_IDENTITY_LINE } from "../../adapters/identity";
1818
import { filterCursorConfiguredModelsByLiveDiscovery } from "../../adapters/cursor/discovery";
@@ -156,6 +156,15 @@ export function deriveComboCatalogModel(
156156
const maxInputTokens = Math.min(
157157
...members.map(member => member.maxInputTokens ?? member.contextWindow!),
158158
);
159+
// A failover combo must compact early enough for every possible target. Missing
160+
// per-model metadata means "use that member's normal derived budget", not "ignore it".
161+
const autoCompactTokenLimit = Math.min(
162+
...members.map(member => clampAutoCompactTokenLimit(
163+
member.contextWindow!,
164+
member.maxInputTokens,
165+
member.autoCompactTokenLimit,
166+
)),
167+
);
159168
const defaultReasoningEffort = effectiveComboDefault(
160169
combo.defaultEffort,
161170
reasoningEfforts,
@@ -167,6 +176,7 @@ export function deriveComboCatalogModel(
167176
owned_by: COMBO_NAMESPACE,
168177
contextWindow,
169178
maxInputTokens,
179+
autoCompactTokenLimit,
170180
...(hasLimitingContextCapMetadata ? { contextCapped } : {}),
171181
inputModalities,
172182
reasoningEfforts,
@@ -207,6 +217,7 @@ export function comboCatalogWarningSignature(
207217
key,
208218
contextWindow: member?.contextWindow ?? null,
209219
maxInputTokens: member?.maxInputTokens ?? null,
220+
autoCompactTokenLimit: member?.autoCompactTokenLimit ?? null,
210221
inputModalities: [...new Set(member?.inputModalities ?? [])].sort(),
211222
reasoningEfforts: [...new Set(member?.reasoningEfforts ?? [])].sort(),
212223
parallelToolCalls: member?.parallelToolCalls === true,
@@ -296,6 +307,7 @@ export function normalizedOpenAiApiSignature(model: CatalogModel): string {
296307
id: model.id,
297308
contextWindow: model.contextWindow ?? null,
298309
maxInputTokens: model.maxInputTokens ?? null,
310+
autoCompactTokenLimit: model.autoCompactTokenLimit ?? null,
299311
inputModalities: [...new Set(model.inputModalities ?? [])].sort(),
300312
reasoningEfforts: [...new Set(model.reasoningEfforts ?? [])].sort(),
301313
ownedBy: model.owned_by ?? null,

‎src/codex/catalog/effort.ts‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ import { CODEX_REASONING_LEVELS, codexEffortRank, configuredReasoningEfforts, mo
1212
import { getModelMetadata, getModelMetadataCaseInsensitive, listModelMetadata, resolveMetadataProvider } from "../../generated/model-metadata";
1313
import { enrichProviderFromRegistry, shouldCaseFoldMetadataModelId } from "../../providers/derive";
1414
import { getProviderRegistryEntry } from "../../providers/registry";
15-
import { applyProviderContextCap, providerContextCap } from "../../providers/context-cap";
15+
import { applyProviderContextCap, clampAutoCompactTokenLimit, providerContextCap } from "../../providers/context-cap";
1616
import { routedSlug, slugEquals, slugsEquivalent } from "../../providers/slug-codec";
1717
import { CODEX_GPT5_IDENTITY_LINE } from "../../adapters/identity";
1818
import { filterCursorConfiguredModelsByLiveDiscovery } from "../../adapters/cursor/discovery";
@@ -125,9 +125,21 @@ export function applyCatalogModelMetadata(entry: RawEntry, model?: CatalogModel)
125125
if (typeof model.contextWindow === "number" && model.contextWindow > 0) {
126126
entry.context_window = model.contextWindow;
127127
entry.max_context_window = model.contextWindow;
128-
entry.auto_compact_token_limit = Math.min(
129-
Math.floor(model.contextWindow * 0.9),
130-
model.maxInputTokens ?? Number.POSITIVE_INFINITY,
128+
}
129+
const effectiveContextWindow = typeof model.contextWindow === "number" && model.contextWindow > 0
130+
? model.contextWindow
131+
: typeof entry.context_window === "number" && entry.context_window > 0
132+
? entry.context_window
133+
: 128_000;
134+
if (
135+
(typeof model.contextWindow === "number" && model.contextWindow > 0)
136+
|| (typeof model.autoCompactTokenLimit === "number" && model.autoCompactTokenLimit > 0)
137+
|| (typeof model.maxInputTokens === "number" && model.maxInputTokens > 0)
138+
) {
139+
entry.auto_compact_token_limit = clampAutoCompactTokenLimit(
140+
effectiveContextWindow,
141+
model.maxInputTokens,
142+
model.autoCompactTokenLimit,
131143
);
132144
}
133145
if (Array.isArray(model.inputModalities) && model.inputModalities.length > 0) {

‎src/codex/catalog/metadata.ts‎

Lines changed: 47 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ import { CODEX_REASONING_LEVELS, codexEffortRank, configuredReasoningEfforts, mo
1313
import { getModelMetadata, getModelMetadataCaseInsensitive, listModelMetadata, resolveMetadataProvider } from "../../generated/model-metadata";
1414
import { enrichProviderFromRegistry, shouldCaseFoldMetadataModelId } from "../../providers/derive";
1515
import { getProviderRegistryEntry } from "../../providers/registry";
16-
import { applyProviderContextCap, providerContextCap } from "../../providers/context-cap";
16+
import { applyProviderContextCap, clampAutoCompactTokenLimit, providerContextCap } from "../../providers/context-cap";
1717
import { routedSlug, slugEquals, slugsEquivalent } from "../../providers/slug-codec";
1818
import { identifyRoutedModel } from "../../adapters/identity";
1919
import { filterCursorConfiguredModelsByLiveDiscovery } from "../../adapters/cursor/discovery";
@@ -144,10 +144,22 @@ const NATIVE_GPT56_FAMILY = new Set<string>([
144144
NATIVE_DAYBREAK_BLUE_MODEL,
145145
]);
146146

147-
export const NATIVE_OPENAI_CONTEXT_OVERRIDES: Record<string, { contextWindow?: number; maxContextWindow?: number; maxInputTokens?: number }> = {
147+
export interface NativeOpenAiContextOverride {
148+
contextWindow?: number;
149+
maxContextWindow?: number;
150+
/** Measured hard input ceiling. This is admission authority, not compaction policy. */
151+
maxInputTokens?: number;
152+
/** Model-specific soft budget exposed to Codex as auto_compact_token_limit. */
153+
autoCompactTokenLimit?: number;
154+
}
155+
156+
export const NATIVE_OPENAI_CONTEXT_OVERRIDES: Record<string, NativeOpenAiContextOverride> = {
148157
"gpt-5.5": { contextWindow: 272_000, maxContextWindow: 272_000 },
149158
"gpt-5.4": { contextWindow: 1_000_000, maxContextWindow: 1_000_000 },
150159
"gpt-5.3-codex-spark": { contextWindow: 100_000, maxContextWindow: 100_000 },
160+
// The shared clamp derives 90% of the effective 272k default (or 922k opt-in) for every
161+
// GPT-5.6 member. Per-model soft budgets may lower that value; no ineffective Sol-only
162+
// upper bound is stored here.
151163
"gpt-5.6-sol": { contextWindow: NATIVE_GPT56_CONTEXT_WINDOW, maxContextWindow: NATIVE_GPT56_MAX_INPUT_TOKENS, maxInputTokens: NATIVE_GPT56_MAX_INPUT_TOKENS },
152164
"gpt-5.6-terra": { contextWindow: NATIVE_GPT56_CONTEXT_WINDOW, maxContextWindow: NATIVE_GPT56_MAX_INPUT_TOKENS, maxInputTokens: NATIVE_GPT56_MAX_INPUT_TOKENS },
153165
"gpt-5.6-luna": { contextWindow: NATIVE_GPT56_CONTEXT_WINDOW, maxContextWindow: NATIVE_GPT56_MAX_INPUT_TOKENS, maxInputTokens: NATIVE_GPT56_MAX_INPUT_TOKENS },
@@ -274,6 +286,30 @@ export function nativeOpenAiMaxInputTokens(slug: string, limits?: NativeContextL
274286
return window === undefined ? narrowed : Math.min(narrowed, window);
275287
}
276288

289+
/** Effective native soft budget. User configuration may lower, but never raise, the model default. */
290+
export function nativeOpenAiAutoCompactTokenLimit(
291+
slug: string,
292+
limits?: NativeContextLimitsInput,
293+
configuredLimit?: number,
294+
): number | undefined {
295+
const contextWindow = nativeOpenAiContextWindow(slug, limits);
296+
if (contextWindow === undefined) return undefined;
297+
const nativeDefault = NATIVE_OPENAI_CONTEXT_OVERRIDES[slug]?.autoCompactTokenLimit;
298+
const positiveConfigured = typeof configuredLimit === "number" && configuredLimit > 0
299+
? configuredLimit
300+
: undefined;
301+
const softLimit = nativeDefault === undefined
302+
? positiveConfigured
303+
: positiveConfigured === undefined
304+
? nativeDefault
305+
: Math.min(nativeDefault, positiveConfigured);
306+
return clampAutoCompactTokenLimit(
307+
contextWindow,
308+
nativeOpenAiMaxInputTokens(slug, limits),
309+
softLimit,
310+
);
311+
}
312+
277313
export function nativeInputModalities(slug: string): string[] {
278314
const upstream = PINNED_NATIVE_CAPABILITY_ENTRIES.get(slug);
279315
if (Array.isArray(upstream?.input_modalities) && upstream!.input_modalities!.length > 0) {
@@ -384,7 +420,9 @@ export function desktopVisibleNativeSlugs(
384420
]);
385421
}
386422

387-
export function nativeModelRows(config: Pick<OcxConfig, "disabledModels" | "combos" | "providerContextCaps" | "providers">): Array<{ slug: string; disabled: boolean; contextWindow?: number; maxInputTokens?: number }> {
423+
export function nativeModelRows(
424+
config: Pick<OcxConfig, "disabledModels" | "combos" | "providerContextCaps" | "providers">,
425+
): Array<{ slug: string; disabled: boolean; contextWindow?: number; maxInputTokens?: number; autoCompactTokenLimit?: number }> {
388426
const disabled = disabledNativeSlugs(config);
389427
const shadowed = configuredNativeAliasSlugs(config);
390428
// Both user levers, not just the cap: a per-model window set from the dashboard has to show
@@ -393,11 +431,17 @@ export function nativeModelRows(config: Pick<OcxConfig, "disabledModels" | "comb
393431
return NATIVE_OPENAI_MODELS.filter(slug => !shadowed.has(slug)).map(slug => {
394432
const contextWindow = nativeOpenAiContextWindow(slug, limits);
395433
const maxInputTokens = nativeOpenAiMaxInputTokens(slug, limits);
434+
const autoCompactTokenLimit = nativeOpenAiAutoCompactTokenLimit(
435+
slug,
436+
limits,
437+
config.providers?.[OPENAI_CODEX_PROVIDER_ID]?.modelAutoCompactTokenLimits?.[slug],
438+
);
396439
return {
397440
slug,
398441
disabled: disabled.has(slug),
399442
...(contextWindow !== undefined ? { contextWindow } : {}),
400443
...(maxInputTokens !== undefined ? { maxInputTokens } : {}),
444+
...(autoCompactTokenLimit !== undefined ? { autoCompactTokenLimit } : {}),
401445
};
402446
});
403447
}

‎src/codex/catalog/parsing.ts‎

Lines changed: 37 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ import { CODEX_REASONING_LEVELS, codexEffortRank, configuredReasoningEfforts, mo
1212
import { getModelMetadata, getModelMetadataCaseInsensitive, listModelMetadata, resolveMetadataProvider } from "../../generated/model-metadata";
1313
import { enrichProviderFromRegistry, shouldCaseFoldMetadataModelId } from "../../providers/derive";
1414
import { getProviderRegistryEntry } from "../../providers/registry";
15-
import { applyProviderContextCap, providerContextCap } from "../../providers/context-cap";
15+
import { applyProviderContextCap, clampAutoCompactTokenLimit, providerContextCap } from "../../providers/context-cap";
1616
import { encodeRoutedModelId, routedSlug, slugEquals, slugsEquivalent } from "../../providers/slug-codec";
1717
import { CODEX_GPT5_IDENTITY_LINE } from "../../adapters/identity";
1818
import { filterCursorConfiguredModelsByLiveDiscovery } from "../../adapters/cursor/discovery";
@@ -111,6 +111,8 @@ export interface CatalogModel {
111111
defaultReasoningEffort?: string;
112112
contextWindow?: number;
113113
maxInputTokens?: number;
114+
/** Soft compaction budget; always clamped below the model's hard context and input ceilings. */
115+
autoCompactTokenLimit?: number;
114116
contextCap?: number;
115117
contextCapped?: boolean;
116118
inputModalities?: string[];
@@ -275,11 +277,21 @@ export function isNativeOpenAiEntry(entry: RawEntry): boolean {
275277
* longer trips this (922,000 window, 829,800 at 90%), but the routed and API-key rows carry
276278
* the same family at a 1,050,000 window where 90% would be 945,000 — past the ceiling.
277279
*/
278-
function nativeAutoCompactLimit(contextWindow: number, maxInputTokens: number | undefined, contextCap?: number): number {
279-
const ninety = Math.floor(contextWindow * 0.9);
280-
if (typeof maxInputTokens !== "number" || maxInputTokens <= 0) return ninety;
281-
const cappedMaxInput = applyProviderContextCap(maxInputTokens, contextCap) ?? maxInputTokens;
282-
return Math.min(ninety, cappedMaxInput, contextWindow);
280+
function nativeAutoCompactLimit(
281+
contextWindow: number,
282+
maxInputTokens: number | undefined,
283+
nativeDefault: number | undefined,
284+
configuredLimit: number | undefined,
285+
): number {
286+
const positiveConfigured = typeof configuredLimit === "number" && configuredLimit > 0
287+
? configuredLimit
288+
: undefined;
289+
const softLimit = nativeDefault === undefined
290+
? positiveConfigured
291+
: positiveConfigured === undefined
292+
? nativeDefault
293+
: Math.min(nativeDefault, positiveConfigured);
294+
return clampAutoCompactTokenLimit(contextWindow, maxInputTokens, softLimit);
283295
}
284296

285297
/**
@@ -302,7 +314,11 @@ function narrowNativeMaxContextWindow(
302314
return Math.min(value, Math.max(resolved, 1));
303315
}
304316

305-
export function applyNativeOpenAiContextOverride(entry: RawEntry, limits?: NativeContextLimitsInput): void {
317+
export function applyNativeOpenAiContextOverride(
318+
entry: RawEntry,
319+
limits?: NativeContextLimitsInput,
320+
modelAutoCompactTokenLimits?: Readonly<Record<string, number>>,
321+
): void {
306322
const nativeSlug = trustedAccountBoundNativeCatalogSlug(entry)
307323
?? (isNativeOpenAiEntry(entry) ? entry.slug as string : undefined);
308324
if (!nativeSlug) return;
@@ -317,7 +333,8 @@ export function applyNativeOpenAiContextOverride(entry: RawEntry, limits?: Nativ
317333
entry.auto_compact_token_limit = nativeAutoCompactLimit(
318334
contextWindow,
319335
nativeOpenAiMaxInputTokens(nativeSlug, limits) ?? override.maxInputTokens,
320-
undefined,
336+
override.autoCompactTokenLimit,
337+
modelAutoCompactTokenLimits?.[nativeSlug],
321338
);
322339
}
323340
if (typeof override.maxContextWindow === "number") {
@@ -336,7 +353,18 @@ export function applyNativeOpenAiContextOverride(entry: RawEntry, limits?: Nativ
336353
entry.auto_compact_token_limit = nativeAutoCompactLimit(
337354
cappedContext,
338355
nativeOpenAiMaxInputTokens(nativeSlug, limits) ?? override?.maxInputTokens,
339-
undefined,
356+
override?.autoCompactTokenLimit,
357+
modelAutoCompactTokenLimits?.[nativeSlug],
358+
);
359+
}
360+
const configuredAutoCompact = modelAutoCompactTokenLimits?.[nativeSlug];
361+
const effectiveContext = typeof entry.context_window === "number" ? entry.context_window : undefined;
362+
if (typeof configuredAutoCompact === "number" && configuredAutoCompact > 0 && effectiveContext !== undefined) {
363+
entry.auto_compact_token_limit = nativeAutoCompactLimit(
364+
effectiveContext,
365+
nativeOpenAiMaxInputTokens(nativeSlug, limits) ?? override?.maxInputTokens,
366+
override?.autoCompactTokenLimit,
367+
configuredAutoCompact,
340368
);
341369
}
342370
const currentMax = typeof entry.max_context_window === "number" ? entry.max_context_window : undefined;

0 commit comments

Comments
 (0)