Skip to content

Commit 2ab9d94

Browse files
lidge-junjun
andauthored
feat(cursor): opt-in effort-variant rows for models outside Cursor's effort table (#3276)
* docs(devlog): wp3 amendments c,d * feat(cursor): opt-in effort-variant rows for models outside Cursor's effort table cursorEffortRows (default off) publishes one <id>--<effort> row per supported effort for models Cursor Private Inference renders no Reasoning control for, and resolves the base model plus effort from that id on /v1/responses, /v1/chat/completions and /v1/messages. Off, /v1/models is byte-identical. * test(cursor): avoid a token-looking fixture credential * fix(cursor): effort rows reject --none, skip detection for plain ids, normalize before combo dispatch --------- Co-authored-by: jun <[email protected]>
1 parent 862e914 commit 2ab9d94

13 files changed

Lines changed: 625 additions & 48 deletions

File tree

‎devlog/_plan/260902_cursor_bundle_effort_table/030_wp3_effort_variant_rows.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,15 @@ verbatim below. Amendments made at P of the wp3 cycle: (a) "table-less" must con
1717
as the static fallback, so the projection follows the installed bundle; (b) exact known full
1818
model ids take precedence over the synthetic grammar (open question 1 → yes).
1919

20+
Amendment (c), audit blocker 1 (005): on `/v1/messages` reuse the existing `effortOverride`
21+
slot (`claude-messages.ts:603/649`, written as `output_config.effort` before translation and
22+
already respected by `anthropicToResponsesTranslation`) instead of injecting the internal
23+
Responses `reasoning.effort`: `effortOverride = effortRow?.effort ?? extractOcxEffortDirective(...)`.
24+
The `none` rung the lane worried about is never published as a row (Cursor's own ladders have
25+
no `none`; filter it from the row set), so the translator's exclusion of `none` is moot.
26+
Amendment (d): `tableLess` in the status route uses `predictCursorEffort(id, table, supportsReasoning).ladder === null`
27+
(wp1 landed the `supportsReasoning` parameter).
28+
2029
---
2130

2231
No files were modified. The untracked `devlog/_plan/260902_cursor_bundle_effort_table/` appeared concurrently and was left untouched. No tests were run.
@@ -413,4 +422,3 @@ Do not run the repository-wide suite in this lane.
413422
- Should the dashboard merely report `effortRows`, or render them inline under each table-less base? This lane recommends the API contract now and leaves presentation to the UI/UX lane.
414423
- Should synthetic rows be added for table-less aliases of otherwise table-matched models? Recommended: yes—the matcher sees the public ID, so an alias such as `opus` genuinely has no Cursor control.
415424

416-

‎docs-site/src/content/docs/reference/configuration.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,15 @@ Aliases are optional short request names. They never change the native model id
4848

4949
Aliases match case-insensitively. A model alias works as `or/opus` or, when globally unique, bare `opus`; an ambiguous bare alias reports its qualified candidates. Codex model pickers show the qualified alias while preserving the canonical `provider/model` routing id. A provider's `defaultAliases` value overrides `defaultModelAliases`. Built-ins are skipped when multiple models in one provider match the same pattern.
5050

51+
### Cursor effort rows
52+
53+
`cursorEffortRows` is an optional boolean and defaults to `false`. When enabled, the raw OpenAI-style
54+
`/v1/models` list adds `<base-id>--<effort>` selectors for reasoning-capable models that Cursor Private
55+
Inference does not match in its installed effort table. Selecting a generated row routes the base model
56+
and applies that row's effort; models Cursor already recognizes receive no variants. The flag reserves a
57+
terminal `--<declared-effort>` suffix for generated selectors, except when the complete value is already
58+
a known configured model id. Cursor may require a model-list refresh or restart after this setting changes.
59+
5160
Valid values in `config.json` override built-in defaults. Missing optional fields use the defaults
5261
documented on the domain pages. `OPENCODEX_HOME` takes precedence over the default configuration
5362
directory. Fields that accept an environment reference, such as `apiKey: "${PROVIDER_API_KEY}"`,

‎gui/src/pages/integrations/cursor-api.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ export interface CursorModelExpectation {
1313
id: string;
1414
reasoning: string[] | null;
1515
family: string | null;
16+
tableLess: boolean;
17+
effortRows: string[];
1618
context: { defaultWindow: number; longWindow: number } | null;
1719
}
1820

‎src/config.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1046,6 +1046,8 @@ const configSchema = z.object({
10461046
providers: z.record(z.string(), providerConfigSchema),
10471047
defaultProvider: z.string().min(1).default("openai"),
10481048
defaultModelAliases: z.boolean().optional(),
1049+
// Malformed hand edits disable this opt-in projection without rejecting providers.
1050+
cursorEffortRows: z.boolean().optional().catch(false),
10491051
// Future versions remain opaque through passthrough-compatible whole-config saves.
10501052
// Only version 1 grants deletion authority in the rebase path.
10511053
configRebaseProvenance: z.unknown().optional(),

‎src/server/chat-completions.ts‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ import {
4646
type TranslatorBudget,
4747
} from "../lib/translator-budget";
4848
import { handleNativeChatCompletions, isNativeChatRouteEligible } from "./chat-native";
49+
import { parseRequestEffortRowId } from "./effort-row";
4950

5051
type Rec = Record<string, unknown>;
5152

@@ -104,6 +105,8 @@ async function handleChatCompletionsWithBudget(
104105
}
105106

106107
const requestedModel = chatBody.model as string;
108+
const effortRow = parseRequestEffortRowId(requestedModel, config);
109+
if (effortRow) chatBody.model = effortRow.baseId;
107110
const stream = chatBody.stream === true;
108111
// Best-effort Grok attribution: the managed fence stamps this header on every model
109112
// it registers (extra_headers, sent verbatim by upstream Grok). Dashboard usage
@@ -113,7 +116,7 @@ async function handleChatCompletionsWithBudget(
113116
let settledRoute: ReturnType<typeof routeModel> | null = null;
114117
let chatNativeRoute: ReturnType<typeof routeModel> | null = null;
115118
try {
116-
const route = routeModel(config, requestedModel, evidenceFromBody(chatBody));
119+
const route = routeModel(config, chatBody.model as string, evidenceFromBody(chatBody));
117120
// Settle the wire once so every branch below reads the adapter this model will
118121
// actually use, not the provider-wide default (#404).
119122
route.provider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, "chat");
@@ -133,7 +136,7 @@ async function handleChatCompletionsWithBudget(
133136
if (chatBody.tools !== undefined) parts.push(JSON.stringify(chatBody.tools));
134137
logCtx.usageLogInputTokens = Math.max(1, estimateTokens(parts.join("\n"), requestedModel));
135138
}
136-
if (isNativeChatRouteEligible(route, chatBody)) chatNativeRoute = route;
139+
if (!effortRow && isNativeChatRouteEligible(route, chatBody)) chatNativeRoute = route;
137140
} catch (err) {
138141
if (err instanceof NoEligiblePolicyCandidateError) {
139142
logCtx.routeDecision = err.trace;
@@ -162,6 +165,12 @@ async function handleChatCompletionsWithBudget(
162165
// Validate the full Chat boundary after routing. Native Chat keeps `chatBody` as
163166
// its wire source; this Responses projection is used only by the fallback path.
164167
internalBody = chatCompletionsToResponsesBody(chatBody);
168+
if (effortRow) {
169+
internalBody.reasoning = {
170+
...(isRec(internalBody.reasoning) ? internalBody.reasoning : {}),
171+
effort: effortRow.effort,
172+
};
173+
}
165174
} catch (err) {
166175
const overflow = isTranslatorBudgetExceededError(err);
167176
const status = overflow ? 413 : err instanceof ChatCompletionsRequestError ? 400 : 500;

‎src/server/claude-messages.ts‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,10 @@ import {
5050
isTranslatorBudgetExceededError,
5151
type TranslatorBudget,
5252
} from "../lib/translator-budget";
53+
import {
54+
parseRequestEffortRowId,
55+
type ParsedEffortRowId,
56+
} from "./effort-row";
5357

5458
type Rec = Record<string, unknown>;
5559

@@ -600,7 +604,9 @@ async function handleClaudeMessagesWithBudget(
600604
let anthropicBody: unknown;
601605
let internalBody: Rec;
602606
let cacheKeySource: ClaudeCacheKeySource = null;
603-
let effortOverride: ReturnType<typeof extractOcxEffortDirective> = null;
607+
let effortOverride: string | null = null;
608+
let effortRow: ParsedEffortRowId | null = null;
609+
let requestedModel = "";
604610
try {
605611
anthropicBody = await readAnthropicBody(req, translatorBudget);
606612
// Defensive [1m] strip (devlog 138): clients normally remove the context-variant
@@ -620,6 +626,14 @@ async function handleClaudeMessagesWithBudget(
620626
effortOverride = extractOcxEffortDirective(anthropicBody);
621627
}
622628
}
629+
if (isRec(anthropicBody) && typeof anthropicBody.model === "string") {
630+
requestedModel = anthropicBody.model;
631+
effortRow = parseRequestEffortRowId(requestedModel, config);
632+
if (effortRow) {
633+
anthropicBody.model = effortRow.baseId;
634+
effortOverride = effortRow.effort;
635+
}
636+
}
623637
// Debug capture (opt-in allowlist scalars) BEFORE the passthrough branch so
624638
// native, routed, and disabled-alias paths are all observable (devlog 130 B1).
625639
captureClaudeInbound(
@@ -643,7 +657,7 @@ async function handleClaudeMessagesWithBudget(
643657
);
644658
if (claudeConversationId) logCtx.conversationId = claudeConversationId;
645659
}
646-
if (isRec(anthropicBody) && wantsNativePassthrough(req, config, requestPolicy, anthropicBody.model)) {
660+
if (!effortRow && isRec(anthropicBody) && wantsNativePassthrough(req, config, requestPolicy, anthropicBody.model)) {
647661
return await anthropicNativePassthrough(req, config, logCtx, logIds, anthropicBody, "/v1/messages");
648662
}
649663
if (isRec(anthropicBody) && effortOverride) {
@@ -669,7 +683,7 @@ async function handleClaudeMessagesWithBudget(
669683
);
670684
}
671685

672-
const requestedModel = (anthropicBody as Rec).model as string;
686+
if (!requestedModel) requestedModel = (anthropicBody as Rec).model as string;
673687
const stream = internalBody.stream === true;
674688
// Routed adapters only support streamed turns; always stream internally and fold
675689
// the translated Anthropic SSE into a message JSON for non-streaming clients.

‎src/server/effort-row.ts‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
import { comboModelId, comboPublicModelId } from "../combos/types";
2+
import { detectCursorInstalls } from "../integrations/cursor-detect";
3+
import {
4+
loadCursorEffortTable,
5+
type CursorEffortTable,
6+
} from "../integrations/cursor-effort-table";
7+
import {
8+
canonicalizeReasoningEfforts,
9+
isDeclaredReasoningEffort,
10+
} from "../reasoning-effort";
11+
import { knownModelIdsForProvider } from "../router";
12+
import { policyModelId, policyPublicModelId } from "../routing/profile";
13+
import type { OcxConfig } from "../types";
14+
import { routedSlug } from "../providers/slug-codec";
15+
import { predictCursorEffort } from "./models-capabilities";
16+
17+
const EFFORT_ROW_SEPARATOR = "--";
18+
19+
export interface ParsedEffortRowId {
20+
baseId: string;
21+
effort: string;
22+
}
23+
24+
export type EffortRowKnownIds = ReadonlySet<string> | ((id: string) => boolean);
25+
26+
export interface EffortRowOptions {
27+
knownIds?: EffortRowKnownIds;
28+
table?: CursorEffortTable | null;
29+
supportsReasoning?: boolean;
30+
}
31+
32+
function isKnownId(knownIds: EffortRowKnownIds | undefined, id: string): boolean {
33+
return typeof knownIds === "function" ? knownIds(id) : knownIds?.has(id) === true;
34+
}
35+
36+
export function effortRowId(baseId: string, effort: string): string {
37+
return `${baseId}${EFFORT_ROW_SEPARATOR}${effort}`;
38+
}
39+
40+
/**
41+
* Exact configured/public ids that must beat the synthetic terminal-suffix grammar.
42+
* This is request-local because live-model cache contents can change while the server runs.
43+
*/
44+
export function knownEffortRowIds(config: OcxConfig): Set<string> {
45+
const ids = new Set<string>();
46+
for (const [providerName, provider] of Object.entries(config.providers)) {
47+
const known = knownModelIdsForProvider(providerName, provider, config);
48+
const namespaces = [providerName, provider.alias].filter((value): value is string => (
49+
typeof value === "string" && value.length > 0
50+
));
51+
for (const id of known) {
52+
ids.add(id);
53+
ids.add(routedSlug(providerName, id));
54+
for (const namespace of namespaces) ids.add(`${namespace}/${id}`);
55+
}
56+
for (const alias of Object.values(provider.modelAliases ?? {})) {
57+
ids.add(alias);
58+
for (const namespace of namespaces) ids.add(`${namespace}/${alias}`);
59+
}
60+
}
61+
for (const [id, combo] of Object.entries(config.combos ?? {})) {
62+
ids.add(comboModelId(id));
63+
ids.add(comboPublicModelId(id, combo));
64+
}
65+
for (const [id, profile] of Object.entries(config.routingProfiles ?? {})) {
66+
ids.add(policyModelId(id));
67+
ids.add(policyPublicModelId(id, profile));
68+
}
69+
return ids;
70+
}
71+
72+
/** Resolve the installed Private Inference effort table once for the current request. */
73+
export function loadDetectedCursorEffortTable(): CursorEffortTable | null {
74+
const privateInference = detectCursorInstalls().find(install => install.build === "private-inference");
75+
return loadCursorEffortTable(privateInference);
76+
}
77+
78+
export function parseEffortRowId(
79+
id: string,
80+
config: Pick<OcxConfig, "cursorEffortRows">,
81+
options: EffortRowOptions = {},
82+
): ParsedEffortRowId | null {
83+
if (config.cursorEffortRows !== true || isKnownId(options.knownIds, id)) return null;
84+
85+
const separator = id.lastIndexOf(EFFORT_ROW_SEPARATOR);
86+
if (separator <= 0) return null;
87+
const baseId = id.slice(0, separator);
88+
const effort = id.slice(separator + EFFORT_ROW_SEPARATOR.length);
89+
// "none" is never published as a row (discovery filters it), so it is never accepted either.
90+
if (effort === "none" || !isDeclaredReasoningEffort(effort)) return null;
91+
if (predictCursorEffort(baseId, options.table ?? null, options.supportsReasoning).ladder !== null) {
92+
return null;
93+
}
94+
return { baseId, effort };
95+
}
96+
97+
/** Parse one ingress selector against the current config and installed Cursor table. */
98+
export function parseRequestEffortRowId(id: string, config: OcxConfig): ParsedEffortRowId | null {
99+
if (config.cursorEffortRows !== true) return null;
100+
// Ordinary ids carry no separator; bail before the known-id scan and install detection so
101+
// the flag costs nothing on the request path for models that are not effort rows.
102+
if (id.lastIndexOf(EFFORT_ROW_SEPARATOR) <= 0) return null;
103+
return parseEffortRowId(id, config, {
104+
knownIds: knownEffortRowIds(config),
105+
table: loadDetectedCursorEffortTable(),
106+
});
107+
}
108+
109+
export function expandCursorEffortRow<T extends { id: string }>(
110+
row: T,
111+
efforts: readonly string[] | undefined,
112+
config: Pick<OcxConfig, "cursorEffortRows">,
113+
options: EffortRowOptions = {},
114+
): T[] {
115+
if (config.cursorEffortRows !== true) return [row];
116+
117+
const supported = canonicalizeReasoningEfforts(
118+
(efforts ?? []).filter(effort => effort !== "none" && isDeclaredReasoningEffort(effort)),
119+
);
120+
const supportsReasoning = options.supportsReasoning ?? supported.length > 0;
121+
if (predictCursorEffort(row.id, options.table ?? null, supportsReasoning).ladder !== null) {
122+
return [row];
123+
}
124+
return [
125+
row,
126+
...supported
127+
.map(effort => effortRowId(row.id, effort))
128+
.filter(id => !isKnownId(options.knownIds, id))
129+
.map(id => ({ ...row, id })),
130+
];
131+
}

0 commit comments

Comments
 (0)