Skip to content

Commit f607233

Browse files
lidge-junjun
andauthored
feat(server): advertise api_types and capabilities on the raw /v1/models list (#3230)
* feat(server): advertise api_types and capabilities on the raw /v1/models list Cursor's local-agent runtime (the Private Inference build) enables its reasoning-effort control only when a model row carries api_types and, optionally, a capabilities object. Emit both on every row of the OpenAI-shape list: api_types is constant (chat_completions, responses, anthropic_messages; membership is load-bearing for Cursor's wire selector and guarded by a unit test), capabilities carries context_length, supports_vision and the reasoning_effort ladder when the catalog knows them. Plain OpenAI clients, Grok Build and the Codex catalog branch ignore the new keys. The combo e2e assertions that compared whole row literals now match on the combo-relevant shape while keeping is_combo presence explicit. * fix(server): include output_modalities so Cursor keeps enriched rows Cursor's local-agent runtime drops any api_types row whose capabilities lack output_modalities containing "text", which silently disabled the effort control on every row. Emit output_modalities: ["text"] and mirror input_modalities when the catalog knows them. Verified live in Cursor Private Inference 3.18.25: the picker shows Low/Medium/High/Extra High for gpt-5.6-sol and a High turn arrives as reasoning.effort=high on /v1/responses. * fix(server): report the effective context window, not the provider cap contextWindow is already narrowed by providerContextCaps; contextCap is the raw operator knob and is set on every row of a capped provider even when the cap did not bite, so preferring it over-reported models whose real window sits below the cap. Freeze the shared api_types constant and copy it per row. * fix(server): omit context_length when it floors to zero positiveInt accepted 0.5 and then floored it to 0, which would emit an invalid zero context length instead of omitting the field (CodeRabbit). * feat(server): advertise the native long-context tier for Cursor Max Mode Cursor's local-agent runtime shows a Context selector (default vs long window, long marked as costing more) when a model row carries a long-context threshold below its context_length. It reads that threshold only from pricing.overrides[].min_prompt_tokens; a cost.long_context object fails its row schema. For native GPT-5.6 rows advertise the family's 272k default and 922k opt-in pair through nativeOpenAiContextTier; routed rows have no separate tier and stay unchanged. * fix(catalog): drop the native long-context tier under any window lever below it A per-model window override or provider window below the long window must remove the tier the same way a provider cap does; otherwise the row keeps advertising 922k while the effective window is smaller (CodeRabbit). --------- Co-authored-by: jun <[email protected]>
1 parent 7aa64bb commit f607233

6 files changed

Lines changed: 362 additions & 18 deletions

File tree

‎src/codex/catalog.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
// Public surface preserved exactly; importers keep using "src/codex/catalog".
33
export { isMediaGenerationModelId, shouldExposeRoutedModel, readCodexCatalogPath, readCatalog, normalizeRoutedCatalogEntry, catalogModelSlug, filterSupportedNativeSlugs, catalogModelSupportsReasoningSummaries } from "./catalog/parsing";
44
export type { CatalogModel, MultiAgentMode } from "./catalog/parsing";
5-
export { accountBoundNativeOpenAiSlugs, accountBoundNativeOpenAiSlugsBySelector, CODEX_NATIVE_ALIAS_CATALOG_KIND, NATIVE_DAYBREAK_BLUE_MODEL, NATIVE_GPT56_CONTEXT_WINDOW, NATIVE_GPT56_MAX_INPUT_TOKENS, NATIVE_GPT56_OPT_IN_CONTEXT_WINDOW, NATIVE_OPENAI_CAPABILITY_ALIAS_MODELS, NATIVE_OPENAI_MODELS, configuredNativeAliasSlugs, desktopAllowlistSuppressedNativeSlugs, isNativeAliasCatalogEntry, isNativeOpenAiCapabilityAliasModel, nativeContextLimits, nativeOpenAiCapabilitySourceSlug, nativeOpenAiContextWindow, nativeOpenAiMaxInputTokens, disabledNativeSlugs, visibleNativeSlugs, desktopVisibleNativeSlugs, nativeModelRows, applyNativeVisibility, observedAccountBoundNativeEntries, observedAccountBoundNativeOpenAiSlugs, upstreamNativeEntry, nativeOpenAiSlugs, listCatalogNativeSlugs, nativeInputModalities, nativeReasoningEfforts, nativeDefaultReasoningEffort, shouldIncludeAccountBoundNativeOpenAi, shouldIncludeNativeOpenAi, type NativeContextLimits, type NativeContextLimitsInput } from "./catalog/metadata";
5+
export { accountBoundNativeOpenAiSlugs, accountBoundNativeOpenAiSlugsBySelector, CODEX_NATIVE_ALIAS_CATALOG_KIND, NATIVE_DAYBREAK_BLUE_MODEL, NATIVE_GPT56_CONTEXT_WINDOW, NATIVE_GPT56_MAX_INPUT_TOKENS, NATIVE_GPT56_OPT_IN_CONTEXT_WINDOW, NATIVE_OPENAI_CAPABILITY_ALIAS_MODELS, NATIVE_OPENAI_MODELS, configuredNativeAliasSlugs, desktopAllowlistSuppressedNativeSlugs, isNativeAliasCatalogEntry, isNativeOpenAiCapabilityAliasModel, nativeContextLimits, nativeOpenAiCapabilitySourceSlug, nativeOpenAiContextWindow, nativeOpenAiMaxInputTokens, nativeOpenAiContextTier, disabledNativeSlugs, visibleNativeSlugs, desktopVisibleNativeSlugs, nativeModelRows, applyNativeVisibility, observedAccountBoundNativeEntries, observedAccountBoundNativeOpenAiSlugs, upstreamNativeEntry, nativeOpenAiSlugs, listCatalogNativeSlugs, nativeInputModalities, nativeReasoningEfforts, nativeDefaultReasoningEffort, shouldIncludeAccountBoundNativeOpenAi, shouldIncludeNativeOpenAi, type NativeContextLimits, type NativeContextLimitsInput } from "./catalog/metadata";
66
export { isSpawnableCodexCandidate, codexExecInvocation, loadBundledCodexCatalog, materializeBundledCodexCatalog, loadCatalogTemplate } from "./catalog/bundled";
77
export { nativeEffortClamp, shouldApplyNativeEffortClamp, catalogModelEfforts, codexSupportedReasoningEfforts, clampedDefaultEffort, clampEntryToCodexSupportedEfforts, clampCatalogModelsToCodexSupport } from "./catalog/effort";
88
export { applyProviderConfigHints, isDatedVariantId, filterCatalogVisibleModels, gatherRoutedModels, clearGatherRoutedModelsInflight, augmentRoutedModelsWithRegistryOpenAiApiRows, augmentRoutedModelsWithMetadata, resolveComboCatalogMember, configuredComboTargetModelsByProvider } from "./catalog/provider-fetch";

‎src/codex/catalog/metadata.ts‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -271,6 +271,34 @@ export function nativeOpenAiContextWindow(slug: string, limits?: NativeContextLi
271271
return narrowToLimits(raw, slug, limits);
272272
}
273273

274+
/**
275+
* Long-context tier for a native slug as a (default, long) pair, for clients that let the user
276+
* pick a window per request (Cursor's local-agent "Context" selector). The pair is the family's
277+
* pinned default window and its opt-in ceiling, independent of whether the operator has
278+
* already opted the proxy into the long window: the selector exists so the client can choose.
279+
* Any user lever below the long window removes the tier: a per-model window override, the
280+
* provider-level window override, or a provider context cap. A lever at or above it leaves the
281+
* tier intact (the 922k/1050k opt-in values are the levers, not a request to shrink). Undefined
282+
* when the family has no separate tier or when the two windows coincide.
283+
*/
284+
export function nativeOpenAiContextTier(
285+
slug: string,
286+
limits?: NativeContextLimitsInput,
287+
): { defaultWindow: number; longWindow: number } | undefined {
288+
const override = NATIVE_OPENAI_CONTEXT_OVERRIDES[slug];
289+
const defaultWindow = positiveInt(override?.contextWindow);
290+
const longWindow = positiveInt(override?.maxContextWindow);
291+
if (defaultWindow === undefined || longWindow === undefined || longWindow <= defaultWindow) return undefined;
292+
const resolved = asLimits(limits);
293+
const levers = [
294+
positiveInt(resolved.modelWindows?.[slug]),
295+
positiveInt(resolved.providerWindow),
296+
positiveInt(resolved.cap),
297+
];
298+
if (levers.some(lever => lever !== undefined && lever < longWindow)) return undefined;
299+
return { defaultWindow, longWindow };
300+
}
301+
274302
/**
275303
* Largest input a native slug accepts, or undefined when no separate limit is known
276304
* (the caller then falls back to the context window).

‎src/server/index.ts‎

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -224,6 +224,7 @@ import {
224224
} from "../lib/package-tree-integrity";
225225
import { detectInstall } from "../update/index";
226226
import { readyProtocolMetadata } from "../remote/protocol";
227+
import { modelCapabilityFields } from "./models-capabilities";
227228

228229
export const MAX_WS_FRAME_BYTES = 50 * 1024 * 1024;
229230
const WEBSOCKET_IDLE_TIMEOUT_SECONDS = 0;
@@ -1345,7 +1346,7 @@ export function startServer(port?: number, deps: StartServerDeps = {}): Server<W
13451346
}
13461347
throw error;
13471348
}
1348-
const { accountBoundNativeOpenAiSlugsBySelector, applyNativeVisibility, buildCatalogEntries, configuredNativeAliasSlugs, desktopAllowlistSuppressedNativeSlugs, disabledNativeSlugs, exactComboCatalogSlugs, loadCatalogTemplate, NATIVE_OPENAI_MODELS, nativeContextLimits, nativeOpenAiSlugs, nativeReasoningEfforts, nativeDefaultReasoningEffort, orderForSubagents, filterCatalogVisibleModels, shouldIncludeAccountBoundNativeOpenAi, shouldIncludeNativeOpenAi, uniqueCatalogModelsForRawPublicList, visibleCodexAccountSelectors, visibleNativeSlugs, desktopVisibleNativeSlugs } = await import("../codex/catalog");
1349+
const { accountBoundNativeOpenAiSlugsBySelector, applyNativeVisibility, buildCatalogEntries, configuredNativeAliasSlugs, desktopAllowlistSuppressedNativeSlugs, disabledNativeSlugs, exactComboCatalogSlugs, loadCatalogTemplate, NATIVE_OPENAI_MODELS, nativeContextLimits, nativeInputModalities, nativeOpenAiContextWindow, nativeOpenAiContextTier, nativeOpenAiSlugs, nativeReasoningEfforts, nativeDefaultReasoningEffort, orderForSubagents, filterCatalogVisibleModels, shouldIncludeAccountBoundNativeOpenAi, shouldIncludeNativeOpenAi, uniqueCatalogModelsForRawPublicList, visibleCodexAccountSelectors, visibleNativeSlugs, desktopVisibleNativeSlugs } = await import("../codex/catalog");
13491350
const { ACCOUNT_GATED_NATIVE_OPENAI_MODELS } = await import("../codex/catalog/native-models");
13501351
const includeNativeOpenAi = shouldIncludeNativeOpenAi(config);
13511352
const includeAccountBoundNativeOpenAi = shouldIncludeAccountBoundNativeOpenAi(config);
@@ -1499,6 +1500,16 @@ export function startServer(port?: number, deps: StartServerDeps = {}): Server<W
14991500
reasoning_efforts: efforts.map(effort => grokEffortOption(effort, effort === defaultEffort)),
15001501
};
15011502
};
1503+
// Cursor's local-agent runtime (Private Inference build) reads api_types + capabilities
1504+
// to enable its effort control; every other consumer ignores them. See
1505+
// src/server/models-capabilities.ts.
1506+
const nativeLimits = nativeContextLimits(config);
1507+
const nativeContextInput = (metadataId: string) => {
1508+
const tier = nativeOpenAiContextTier(metadataId, nativeLimits);
1509+
return tier
1510+
? { contextWindow: tier.defaultWindow, longContextWindow: tier.longWindow }
1511+
: { contextWindow: nativeOpenAiContextWindow(metadataId, nativeLimits) };
1512+
};
15021513
const nativeModelRow = (id: string, metadataId = id) => ({
15031514
id,
15041515
object: "model",
@@ -1508,6 +1519,14 @@ export function startServer(port?: number, deps: StartServerDeps = {}): Server<W
15081519
nativeReasoningEfforts(metadataId),
15091520
nativeDefaultReasoningEffort(metadataId),
15101521
),
1522+
...modelCapabilityFields({
1523+
reasoningEfforts: nativeReasoningEfforts(metadataId),
1524+
// Cursor "Max Mode": advertise the family's default/long pair (272k/922k for
1525+
// GPT-5.6) so the client can pick per request; without a tier, the effective
1526+
// window is the only value.
1527+
...nativeContextInput(metadataId),
1528+
inputModalities: nativeInputModalities(metadataId),
1529+
}),
15111530
});
15121531
// Selector-active discovery follows the same complete supported set as the Codex catalog
15131532
// for both bare and qualified rows. Without selectors, the live catalog continues to own
@@ -1554,6 +1573,13 @@ export function startServer(port?: number, deps: StartServerDeps = {}): Server<W
15541573
...(isCombo ? { is_combo: true } : {}),
15551574
...(effective ? { alias_of: `${provider?.alias || m.provider}/${effective.alias}` } : {}),
15561575
...grokEffortFields(m.reasoningEfforts ?? [], m.defaultReasoningEffort),
1576+
...modelCapabilityFields({
1577+
reasoningEfforts: m.reasoningEfforts,
1578+
// contextWindow is already the post-cap effective value; contextCap is the raw
1579+
// operator knob and over-reports models whose real window sits below it.
1580+
contextWindow: m.contextWindow,
1581+
inputModalities: m.inputModalities,
1582+
}),
15571583
};
15581584
})),
15591585
];

‎src/server/models-capabilities.ts‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
/**
2+
* Extended capability advertisement for the OpenAI-shape `GET /v1/models` list.
3+
*
4+
* Cursor's local-agent runtime (the "Private Inference" build, `localMode=true`) enables its
5+
* reasoning-effort control only when at least one row in `data[]` carries `api_types` naming an
6+
* API family it can speak, optionally with a `capabilities` object. Plain OpenAI clients, Grok
7+
* Build, and the Codex catalog branch ignore both keys. Every OpenCodex route serves Chat
8+
* Completions, Responses and Anthropic Messages, streams, and accepts tool calls, so those are
9+
* constants; context length and vision come from catalog data when known and are omitted
10+
* otherwise, matching Cursor's optional-field schema.
11+
*/
12+
13+
/**
14+
* Membership is load-bearing for Cursor: its wire selector picks the Anthropic Messages path only
15+
* when NO OpenAI-family type (`chat_completions`/`responses`/`openai_chat`/`openai_responses`)
16+
* is present. Keep at least one OpenAI-family entry; a unit test guards this.
17+
*/
18+
export const OPENCODEX_MODEL_API_TYPES: readonly string[] = Object.freeze(["chat_completions", "responses", "anthropic_messages"]);
19+
20+
export const OPENAI_FAMILY_API_TYPES: ReadonlySet<string> = new Set(["chat_completions", "responses", "openai_chat", "openai_responses"]);
21+
22+
export interface ModelCapabilityInput {
23+
reasoningEfforts?: readonly string[];
24+
contextWindow?: number;
25+
/**
26+
* Larger opt-in window (Cursor "Max Mode"). When it exceeds contextWindow, the row advertises
27+
* the long window as context_length and the default window as the long-context threshold,
28+
* which makes Cursor's local runtime show a Context selector (default vs long, long marked
29+
* as costing more).
30+
*/
31+
longContextWindow?: number;
32+
inputModalities?: readonly string[];
33+
}
34+
35+
export interface ModelCapabilityFields {
36+
api_types: readonly string[];
37+
capabilities: {
38+
context_length?: number;
39+
/** Cursor's extended-row filter REQUIRES this to contain "text"; every route emits text. */
40+
output_modalities: string[];
41+
input_modalities?: string[];
42+
supports_tool_use: true;
43+
supports_streaming: true;
44+
supports_reasoning: boolean;
45+
supports_vision?: boolean;
46+
reasoning_effort?: string[];
47+
};
48+
/**
49+
* Cursor reads the long-context threshold from `pricing.overrides[].min_prompt_tokens`. That
50+
* key sits outside its validated capability schema, so it is the one place a threshold can
51+
* be carried without failing row validation (`cost.long_context` is rejected by that schema).
52+
*/
53+
pricing?: { overrides: Array<{ min_prompt_tokens: number }> };
54+
}
55+
56+
function positiveInt(value: unknown): number | undefined {
57+
if (typeof value !== "number" || !Number.isFinite(value)) return undefined;
58+
const floored = Math.floor(value);
59+
return floored > 0 ? floored : undefined;
60+
}
61+
62+
export function modelCapabilityFields(input: ModelCapabilityInput): ModelCapabilityFields {
63+
const efforts = (input.reasoningEfforts ?? []).filter(effort => typeof effort === "string" && effort.length > 0);
64+
const contextLength = positiveInt(input.contextWindow);
65+
const longContextLength = positiveInt(input.longContextWindow);
66+
const hasLongTier = contextLength !== undefined && longContextLength !== undefined && longContextLength > contextLength;
67+
const modalities = Array.isArray(input.inputModalities)
68+
? input.inputModalities.filter(modality => typeof modality === "string" && modality.length > 0)
69+
: undefined;
70+
const supportsVision = modalities !== undefined ? modalities.includes("image") : undefined;
71+
return {
72+
api_types: [...OPENCODEX_MODEL_API_TYPES],
73+
capabilities: {
74+
...(hasLongTier
75+
? { context_length: longContextLength }
76+
: contextLength !== undefined ? { context_length: contextLength } : {}),
77+
// Once a gateway advertises api_types, Cursor keeps only rows whose output_modalities
78+
// include "text"; omitting the key drops the row from the extended catalog.
79+
output_modalities: ["text"],
80+
...(modalities !== undefined && modalities.length > 0 ? { input_modalities: [...modalities] } : {}),
81+
supports_tool_use: true,
82+
supports_streaming: true,
83+
supports_reasoning: efforts.length > 0,
84+
...(supportsVision !== undefined ? { supports_vision: supportsVision } : {}),
85+
...(efforts.length > 0 ? { reasoning_effort: [...efforts] } : {}),
86+
},
87+
...(hasLongTier ? { pricing: { overrides: [{ min_prompt_tokens: contextLength }] } } : {}),
88+
};
89+
}

0 commit comments

Comments
 (0)