Skip to content

Commit a7dd239

Browse files
im-tylerclaude
andcommitted
fix(models): give toggle-kind models a thinking control in the CLI
GLM, Kimi and Qwen models had no way to turn thinking on or off from the CLI, while the web app and workspace handled them correctly. The recorded conclusion was that fixing it needed an upstream change or a 16th overlay patch. That was wrong, and the wrong note is why it sat for a day. Upstream's reasoningToggle handles only @ai-sdk/alibaba and @ai-sdk/cohere and returns {} for everything else, openai-compatible included (transform.ts:1707-1715). An empty variant set falls back to the id heuristics in variants(), which blocklist glm, kimi and qwen outright (:781-793). So declaring `toggle` honestly is worse than declaring nothing at all: it routes straight into a blocklist. reasoningVariants takes the `effort` arm before the toggle arm (:1662), and effort IS implemented for openai-compatible. A toggle is two states, so it is now declared as effort with values [null, "high"]. Upstream maps null to its "none" variant; Fylun's gateway maps "none" to thinking-off at translate.ts:275 — checked before any effort normalisation — and anything else to on. Verified against the pinned upstream ProviderTransform: all eight toggle models now yield {none:{reasoningEffort:"none"}, high:{reasoningEffort:"high"}}. The registry still says toggle, which is true. Only the advertised wire vocabulary changed, to the one upstream can act on. The comment that blocked this claimed nulls could not be emitted because the gateway "would normalise UP to the strongest level rather than off". False — explicitlyOff short-circuits first. Replaced with what actually happens. Budget-kind models still fall to the heuristics and get low/medium/high instead of their declared min/max. Controllable, so a fidelity gap rather than a missing control; noted in the README rather than worked around. Upstream: anomalyco/opencode#42793 is open against v1.18.18, the version pinned here, and a one-line PR #44064 was closed unmerged by a bot on a template technicality. The hole affects 634 models across 88 providers, so it is worth commenting on — but nothing here waits on it now. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01CfMxUBzUH4UP3reJ71j59K
1 parent e41940c commit a7dd239

4 files changed

Lines changed: 158 additions & 82 deletions

File tree

‎README.md‎

Lines changed: 33 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -90,12 +90,39 @@ codesign --force --sign - ~/.local/bin/fylun-code-bin
9090
reads `reasoning_options` directly off the catalog entry — the heuristics are
9191
the fallback for catalogs that don't declare it. `gen-fylun-models.mjs` now
9292
emits `reasoning_options` from each model's registry `ThinkingSupport`, so
93-
the guessing path is never reached. Still open upstream: `reasoningToggle`
94-
and `reasoningBudget` both return nothing for `@ai-sdk/openai-compatible`, so
95-
toggle-kind and budget-kind models fall through to the same heuristics and
96-
the GLM/Kimi/Qwen toggles remain uncontrollable. Fixing that needs an
97-
upstream change or a new patch; declaring the shapes honestly is the half
98-
that belongs here.
93+
the guessing path is never reached.
94+
95+
**Amended again 2026-08-31 — "needs an upstream change or a new patch" was
96+
wrong, and it cost the toggle models a control for a day.** `reasoningToggle`
97+
and `reasoningBudget` do both return nothing for `@ai-sdk/openai-compatible`
98+
(`transform.ts:1707-1715`, `:1846`), so declaring `toggle` honestly is
99+
actively WORSE than declaring nothing: an empty variant set falls back to the
100+
very id blocklist described above, and the model ends up with no control at
101+
all. But `reasoningVariants` takes the `effort` arm BEFORE the toggle arm
102+
(`:1662`), and effort IS implemented for openai-compatible. A toggle has
103+
exactly two states, so it is now declared as `[{ type: "effort", values:
104+
[null, "high"] }]` — upstream maps `null` to its "none" variant, and Fylun's
105+
gateway maps "none" to thinking-off at `translate.ts:275`, checked before any
106+
effort normalisation, and anything else to on. Verified against the pinned
107+
upstream `ProviderTransform`: `{none:{reasoningEffort:"none"},
108+
high:{reasoningEffort:"high"}}` for all eight toggle models. No patch.
109+
110+
The registry still says `toggle`, which is the truth; only the advertised
111+
wire vocabulary changed, to the one upstream can act on. The earlier claim
112+
that nulls could not be emitted because the gateway "would normalise UP to
113+
the strongest level rather than off" was simply false, and was what hid this.
114+
115+
Budget-kind models (Haiku 4.5, the Gemini three) still fall to the heuristics
116+
and get low/medium/high rather than their declared min/max. That is
117+
controllable — the gateway maps effort words to budgets and clamps to the
118+
model's range — so it is a fidelity gap, not a missing control, and the only
119+
real loss is that Gemini 3 Flash's budget floor of 0 has no "off" variant.
120+
121+
Upstream tracking: `anomalyco/opencode#42793` (open, filed against v1.18.18,
122+
the pinned version) and `#39933`. A one-line PR `#44064` was closed unmerged
123+
by a bot on a PR-template technicality without human review. Worth commenting
124+
on #42793 with the reproduction — the hole affects 634 models across 88
125+
providers on live models.dev — but Fylun no longer depends on it landing.
99126

100127
### Undocumented patches
101128

‎distribution/models-fylun.json‎

Lines changed: 40 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -281,7 +281,11 @@
281281
"reasoning": true,
282282
"reasoning_options": [
283283
{
284-
"type": "toggle"
284+
"type": "effort",
285+
"values": [
286+
null,
287+
"high"
288+
]
285289
}
286290
],
287291
"temperature": false,
@@ -317,7 +321,11 @@
317321
"reasoning": true,
318322
"reasoning_options": [
319323
{
320-
"type": "toggle"
324+
"type": "effort",
325+
"values": [
326+
null,
327+
"high"
328+
]
321329
}
322330
],
323331
"temperature": false,
@@ -846,7 +854,11 @@
846854
"reasoning": true,
847855
"reasoning_options": [
848856
{
849-
"type": "toggle"
857+
"type": "effort",
858+
"values": [
859+
null,
860+
"high"
861+
]
850862
}
851863
],
852864
"temperature": true,
@@ -877,7 +889,11 @@
877889
"reasoning": true,
878890
"reasoning_options": [
879891
{
880-
"type": "toggle"
892+
"type": "effort",
893+
"values": [
894+
null,
895+
"high"
896+
]
881897
}
882898
],
883899
"temperature": true,
@@ -908,7 +924,11 @@
908924
"reasoning": true,
909925
"reasoning_options": [
910926
{
911-
"type": "toggle"
927+
"type": "effort",
928+
"values": [
929+
null,
930+
"high"
931+
]
912932
}
913933
],
914934
"temperature": true,
@@ -939,7 +959,11 @@
939959
"reasoning": true,
940960
"reasoning_options": [
941961
{
942-
"type": "toggle"
962+
"type": "effort",
963+
"values": [
964+
null,
965+
"high"
966+
]
943967
}
944968
],
945969
"temperature": true,
@@ -1058,7 +1082,11 @@
10581082
"reasoning": true,
10591083
"reasoning_options": [
10601084
{
1061-
"type": "toggle"
1085+
"type": "effort",
1086+
"values": [
1087+
null,
1088+
"high"
1089+
]
10621090
}
10631091
],
10641092
"temperature": true,
@@ -1089,7 +1117,11 @@
10891117
"reasoning": true,
10901118
"reasoning_options": [
10911119
{
1092-
"type": "toggle"
1120+
"type": "effort",
1121+
"values": [
1122+
null,
1123+
"high"
1124+
]
10931125
}
10941126
],
10951127
"temperature": true,

‎distribution/models.json‎

Lines changed: 60 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -281,7 +281,11 @@
281281
"reasoning": true,
282282
"reasoning_options": [
283283
{
284-
"type": "toggle"
284+
"type": "effort",
285+
"values": [
286+
null,
287+
"high"
288+
]
285289
}
286290
],
287291
"temperature": false,
@@ -317,7 +321,11 @@
317321
"reasoning": true,
318322
"reasoning_options": [
319323
{
320-
"type": "toggle"
324+
"type": "effort",
325+
"values": [
326+
null,
327+
"high"
328+
]
321329
}
322330
],
323331
"temperature": false,
@@ -846,7 +854,11 @@
846854
"reasoning": true,
847855
"reasoning_options": [
848856
{
849-
"type": "toggle"
857+
"type": "effort",
858+
"values": [
859+
null,
860+
"high"
861+
]
850862
}
851863
],
852864
"temperature": true,
@@ -877,7 +889,11 @@
877889
"reasoning": true,
878890
"reasoning_options": [
879891
{
880-
"type": "toggle"
892+
"type": "effort",
893+
"values": [
894+
null,
895+
"high"
896+
]
881897
}
882898
],
883899
"temperature": true,
@@ -908,7 +924,11 @@
908924
"reasoning": true,
909925
"reasoning_options": [
910926
{
911-
"type": "toggle"
927+
"type": "effort",
928+
"values": [
929+
null,
930+
"high"
931+
]
912932
}
913933
],
914934
"temperature": true,
@@ -939,7 +959,11 @@
939959
"reasoning": true,
940960
"reasoning_options": [
941961
{
942-
"type": "toggle"
962+
"type": "effort",
963+
"values": [
964+
null,
965+
"high"
966+
]
943967
}
944968
],
945969
"temperature": true,
@@ -1058,7 +1082,11 @@
10581082
"reasoning": true,
10591083
"reasoning_options": [
10601084
{
1061-
"type": "toggle"
1085+
"type": "effort",
1086+
"values": [
1087+
null,
1088+
"high"
1089+
]
10621090
}
10631091
],
10641092
"temperature": true,
@@ -1089,7 +1117,11 @@
10891117
"reasoning": true,
10901118
"reasoning_options": [
10911119
{
1092-
"type": "toggle"
1120+
"type": "effort",
1121+
"values": [
1122+
null,
1123+
"high"
1124+
]
10931125
}
10941126
],
10951127
"temperature": true,
@@ -20009,9 +20041,9 @@
2000920041
"output": 128000
2001020042
},
2001120043
"cost": {
20012-
"input": 0.132,
20013-
"output": 0.528,
20014-
"cache_read": 0.033
20044+
"input": 0.0825,
20045+
"output": 0.33,
20046+
"cache_read": 0.020625
2001520047
}
2001620048
},
2001720049
"tencent/hy-mt2-7b": {
@@ -21686,9 +21718,9 @@
2168621718
"output": 384000
2168721719
},
2168821720
"cost": {
21689-
"input": 0.088606,
21690-
"output": 0.177212,
21691-
"cache_read": 0.017721
21721+
"input": 0.08246,
21722+
"output": 0.16492,
21723+
"cache_read": 0.016492
2169221724
}
2169321725
},
2169421726
"deepseek/deepseek-v4-pro-0813": {
@@ -21730,9 +21762,9 @@
2173021762
"output": 384000
2173121763
},
2173221764
"cost": {
21733-
"input": 1.32,
21734-
"output": 3.96,
21735-
"cache_read": 0.044
21765+
"input": 0.66,
21766+
"output": 1.98,
21767+
"cache_read": 0.022
2173621768
}
2173721769
},
2173821770
"deepseek/deepseek-v4-pro": {
@@ -21774,12 +21806,12 @@
2177421806
"open_weights": true,
2177521807
"limit": {
2177621808
"context": 1048576,
21777-
"output": 393216
21809+
"output": 384000
2177821810
},
2177921811
"cost": {
21780-
"input": 1.6,
21781-
"output": 3.2,
21782-
"cache_read": 0.135
21812+
"input": 1.034256,
21813+
"output": 2.068512,
21814+
"cache_read": 0.086188
2178321815
}
2178421816
},
2178521817
"deepseek/deepseek-v3.2": {
@@ -22108,9 +22140,9 @@
2210822140
"output": 384000
2210922141
},
2211022142
"cost": {
22111-
"input": 0.44,
22112-
"output": 1.32,
22113-
"cache_read": 0.014
22143+
"input": 0.22,
22144+
"output": 0.66,
22145+
"cache_read": 0.007
2211422146
}
2211522147
},
2211622148
"arcee-ai/trinity-large-thinking": {
@@ -30915,12 +30947,12 @@
3091530947
"open_weights": true,
3091630948
"limit": {
3091730949
"context": 1048576,
30918-
"output": 262144
30950+
"output": 131072
3091930951
},
3092030952
"cost": {
3092130953
"input": 2,
3092230954
"output": 6,
30923-
"cache_read": 0.25
30955+
"cache_read": 0.2
3092430956
}
3092530957
},
3092630958
"liquid/lfm-2.5-2.6b:free": {
@@ -33350,45 +33382,14 @@
3335033382
"open_weights": false,
3335133383
"limit": {
3335233384
"context": 262144,
33353-
"output": 80000
33385+
"output": 235929
3335433386
},
3335533387
"cost": {
3335633388
"input": 0.74,
3335733389
"output": 2.96,
3335833390
"cache_read": 0.15
3335933391
}
3336033392
},
33361-
"kwaipilot/kat-coder-air-v2.5": {
33362-
"id": "kwaipilot/kat-coder-air-v2.5",
33363-
"name": "KAT-Coder-Air V2.5",
33364-
"description": "Coding model for repository understanding, refactors, and agentic engineering tasks",
33365-
"family": "kat-coder",
33366-
"attachment": false,
33367-
"reasoning": false,
33368-
"tool_call": true,
33369-
"structured_output": true,
33370-
"temperature": true,
33371-
"release_date": "2026-07-10",
33372-
"last_updated": "2026-07-10",
33373-
"modalities": {
33374-
"input": [
33375-
"text"
33376-
],
33377-
"output": [
33378-
"text"
33379-
]
33380-
},
33381-
"open_weights": false,
33382-
"limit": {
33383-
"context": 256000,
33384-
"output": 80000
33385-
},
33386-
"cost": {
33387-
"input": 0.15,
33388-
"output": 0.6,
33389-
"cache_read": 0.03
33390-
}
33391-
},
3339233393
"kwaipilot/kat-coder-pro-v2": {
3339333394
"id": "kwaipilot/kat-coder-pro-v2",
3339433395
"name": "KAT-Coder-Pro V2",
@@ -33412,7 +33413,7 @@
3341233413
"open_weights": false,
3341333414
"limit": {
3341433415
"context": 262144,
33415-
"output": 80000
33416+
"output": 144000
3341633417
},
3341733418
"cost": {
3341833419
"input": 0.3,

0 commit comments

Comments
 (0)