Repository navigation
Commit 282a4f8
committed
fix: disable Gemini thinking — it was eating the output budget and
truncating batch responses
ROOT CAUSE of "the email has no insights".
Evidence from the 2026-08-21 run: 11 of 14 episodes carried
classification_reason="LLM batch entry missing; metadata fallback",
score exactly 50.0, zero key_ideas. Only 3 episodes got real LLM output.
Broken down by category:
startup 2 of 6 succeeded (partial -> truncated response)
personal_growth 1 of 2 succeeded (partial)
ai_retail 0 of 6 succeeded (empty -> whole batch lost)
"batch entry missing" means the JSON array came back with FEWER objects
than episodes sent. Two shapes, one cause.
gemini-2.5-flash defaults to DYNAMIC THINKING, and thinking tokens are
billed against maxOutputTokens. With max_tokens=15000 (3000/episode x 5)
reasoning could consume the entire budget before any JSON was emitted.
Google documents thinkingBudget=0 as the way to disable it for 2.5 Flash;
the known upstream issue confirms the failure mode:
"A MAX_TOKENS finish reason appears to be given if
thoughts_token_count + output_token_count > max_output_tokens ...
If MAX_TOKENS is triggered, the response text is empty"
googleapis/python-genai#782
Empty text -> whole batch throws (ai_retail). Truncated text -> json_repair
salvages a prefix (startup, personal_growth). Both degrade silently.
Fixes:
1. generationConfig.thinkingConfig.thinkingBudget = 0. The actual fix.
2. _extract_gemini_text() replaces
candidates[0].content.parts[0].text, which had TWO latent bugs:
- empty parts raised a bare "IndexError: list index out of range",
swallowed by the batch ranker into metadata-only scoring with no
hint that thinking tokens were responsible;
- it read only parts[0], silently dropping the rest when Gemini
splits output across multiple parts (truncating the JSON array).
Now joins all parts and raises naming finishReason, thoughtsTokenCount
and promptTokenCount.
3. stage2_batch_rank logs "returned N entries for M episodes — TRUNCATED"
once, instead of M separate lines that look like M unrelated faults.
4. _BATCH_SIZE 5 -> 3 and token_budget_per_episode 3000 -> 5000, so a
truncation costs 3 episodes instead of 5 and there is real headroom.
Verified against the exact response shapes from the upstream issue:
MAX_TOKENS with empty parts, missing content key, no candidates (safety
block), healthy response, and multi-part output. Confirmed the old code
raised IndexError on the production case. 7 tests added.1 parent 4929a85 commit 282a4f8
1 file changed
Lines changed: 81 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
0 commit comments