Repository navigation
Commit 99d72a5
committed
Extend the reasoning-text fix to Gemini: includeThoughts, and split thought parts out of the answer
GeminiProvider.chat() had the same "generates reasoning, bills for it,
discards it" bug as Groq/Cerebras, but one layer earlier and in two
compounding parts:
1. The request never set thinkingConfig.includeThoughts. thinkingLevel
and thinkingBudget only control how much the model thinks; Gemini
only returns thought content in the response when includeThoughts is
explicitly requested. It never was, so thought content was generated,
billed, and never even arrived at the gateway to discard.
2. Even if it had arrived, the response parser would have contaminated
the answer with it: text = "".join(p.get("text","") for p in parts
if "text" in p) concatenates every part with a text key, with no
check of the sibling part["thought"] boolean Gemini sets on reasoning
parts.
These two had to ship together, not just be bundled for tidiness: turning
on includeThoughts without also fixing the parser would have introduced a
live contamination bug that doesn't exist today, since today no thought
parts return at all.
Fix, same file, same method:
- Request side: add "includeThoughts": True to both existing
thinkingConfig dict literals (the thinkingLevel branch and the
thinkingBudget branch).
- Response side: split parts by their thought flag -- thought-flagged
parts join into reasoning_text, everything else joins into text as
before. Add reasoning_text to the return dict, reusing the same
schema field and routes/chat.py wiring this PR already added for
Groq/Cerebras -- no further schema changes needed.
Confirmed by reading every class in providers.py and every provider
registry construction site: NVIDIA, OpenRouter, and GitHub need no
separate work (all three inherit chat() unchanged from
OpenAICompatProvider, already covered by this PR's Groq/Cerebras fix).
Ollama is separately covered by theschoolofai#29. No Anthropic/Claude chat provider
exists in this codebase, and the embedding/STT/TTS provider hierarchies
are a different domain, out of scope for this bug.
Explicitly out of scope for this change, named rather than silently
missed:
- output_tokens undercounting thoughtsTokenCount (Gemini's usage
fields are additive, unlike OpenAI-compat's completion_tokens, which
already includes reasoning tokens as a documented subset) -- a
separate, real cost-accounting gap, logged for its own fix.
- googleapis/python-genai#2121: some Gemini configurations return
thought content as a plain "THOUGHT:"-prefixed text part instead of
setting part.thought = True. Named in a code comment, not fixed.
- maxOutputTokens defaults: thinking and the visible answer share one
token budget, and a tight max_tokens can starve the answer once
thinking is turned on. A deployment-time config choice, not a code
change.
3 new tests in tests/test_provider_reasoning.py (extending the existing
file, not a new one): reasoning_text captured from a thought part; text
excludes thought content when both are present -- the actual regression
test for the contamination bug, confirmed RED before the fix producing
literally "scratch work: 6*7=4242"; reasoning_text is None when no
thought parts return. 18 passed on the full file. Full suite: 471
passed, the same two pre-existing failures already cited in this PR's
original body, nothing new.1 parent fc7e75d commit 99d72a5
2 files changed
Lines changed: 142 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
850 | 850 | | |
851 | 851 | | |
852 | 852 | | |
853 | | - | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
854 | 857 | | |
855 | 858 | | |
856 | | - | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
857 | 863 | | |
858 | 864 | | |
859 | 865 | | |
| |||
885 | 891 | | |
886 | 892 | | |
887 | 893 | | |
888 | | - | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
889 | 911 | | |
890 | 912 | | |
891 | 913 | | |
| |||
920 | 942 | | |
921 | 943 | | |
922 | 944 | | |
| 945 | + | |
923 | 946 | | |
924 | 947 | | |
925 | 948 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
221 | 221 | | |
222 | 222 | | |
223 | 223 | | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
224 | 340 | | |
225 | 341 | | |
226 | 342 | | |
| |||
0 commit comments