Thinking models are unreliable when max_output_tokens set due to them ignoring the thinking budget

Three different issues make the thinking models (2.5 flash and pro) unreliable:

The thinking_budget appears to be ignored. We get non-zero (and large) thoughts_token_count values regardless of the thinking_budget config. This appears to be a known issue?
A MAX_TOKENS finish reason appears to be given if thoughts_token_count + output_token_count > max_output_tokens. This is problematic since we don't appear to have proper control over thoughts_token_count right now.
If MAX_TOKENS finish reason is triggered, the response text is empty, making debugging of the above very difficult (we can't get a sense of the size of the output text).

We only started experiencing consistent MAX_TOKENS responses sometime last week, so it seems something might have changed recently? Either the default thinking budget, or the way max_output_tokens is applied?

The only solution that currently works for us is to set max_output_tokens=None, but then the thinking tokens often increase to ~6k even for very simple tasks.

Edit: we have only seen this occur when using structured output. I'm can't confirm if it occurs for unstructured output.

Environment details

Programming language: Python
OS: Ubuntu 22.04
Language runtime version: 3.10
Package version: 1.14.0

Example

The docs suggest setting thinking_budget=0 will turn thinking off, however with the following request:

class RelevantDataFields(BaseModel):
    fields: List[Literal[tuple(field_names)]]

model_config = GenerateContentConfig(
    system_instruction=self.system_instruction,
    thinking_config=ThinkingConfig(thinking_budget=0),
    max_output_tokens=2000,
    temperature=0.,
    response_schema=RelevantDataFields.model_json_schema(),
    response_mime_type='application/json'
)
pred = await aio.models.generate_content(
    model='gemini-2.5-pro-preview-03-25',
    contents=prompt,
    config=model_config
)

the response is:

GenerateContentResponse(candidates=[Candidate(content=Content(parts=[Part(video_metadata=None,
thought=None, code_execution_result=None, executable_code=None, file_data=None, function_call=None,
function_response=None, inline_data=None, text='')], role='model'), citation_metadata=None,
finish_message=None, token_count=None, finish_reason=<FinishReason.MAX_TOKENS: 'MAX_TOKENS'>,
avg_logprobs=None, grounding_metadata=None, index=0, logprobs_result=None, safety_ratings=None)],
create_time=None, response_id=None, model_version='models/gemini-2.5-pro-preview-05-06',
prompt_feedback=None,
usage_metadata=GenerateContentResponseUsageMetadata(cache_tokens_details=None,
cached_content_token_count=None, candidates_token_count=None, candidates_tokens_details=None,
prompt_token_count=1674, prompt_tokens_details=[ModalityTokenCount(modality=<MediaModality.TEXT: 'TEXT'>, token_count=1674)], thoughts_token_count=2000, tool_use_prompt_token_count=None, tool_use_prompt_tokens_details=None, total_token_count=3674, traffic_type=None), automatic_function_calling_history=[], parsed=None)

The thoughts_token_count=2000 despite setting thinking_budget=0, and the model version is also wrong (05-06 instead of 03-25).

Ideal solution

thinking_budget should constrain thoughts_token_count
MAX_TOKENS finish reason should only be returned if the number of output tokens is greater than max_output_tokens, rather than including the thinking budget (assuming this is the case?).
The computed output so far should be returned if MAX_TOKENS is reached.

Update: I've just noticed when I get FinishReason.MaxTokens, thoughts_token_count = max_output_tokens, so the thought tokens definitely appear to be linked to the output token limit now? For example below is a response with max_output_tokens=5000 and thinking_budget=0:

GenerateContentResponse(candidates=[Candidate(content=Content(parts=[Part(video_metadata=None, thought=None, code_execution_result=None, executable_code=None, file_data=None, function_call=None, function_response=None, inline_data=None, text='')], role='model'), citation_metadata=None, finish_message=None, token_count=None, finish_reason=<FinishReason.MAX_TOKENS: 'MAX_TOKENS'>, avg_logprobs=None, grounding_metadata=None, index=0, logprobs_result=None, safety_ratings=None)], create_time=None, response_id=None, model_version='models/gemini-2.5-flash-preview-04-17', prompt_feedback=None, usage_metadata=GenerateContentResponseUsageMetadata(cache_tokens_details=None, cached_content_token_count=None, candidates_token_count=None, candidates_tokens_details=None, prompt_token_count=4725, prompt_tokens_details=[ModalityTokenCount(modality=<MediaModality.TEXT: 'TEXT'>, token_count=4725)], thoughts_token_count=5000, tool_use_prompt_token_count=None, tool_use_prompt_tokens_details=None, total_token_count=9725, traffic_type=None), automatic_function_calling_history=[], parsed=None)

This is for a task that, before last week, ran very reliably with max_output_tokens=500 on gemini-2.5-pro-preview-03-25, so something has changed recently.

Thank you for this feedback, I've passed it along to the API team and will let you know when we have an update.

@sararob

Thanks @sararob. Just as an FYI, I missed out a small but potentially important bit of information earlier. We've only seen this issue occur when using structured output. I've updated my initial post to mention this.

Hi, I have the same problem here, using the response_mime_type = "text/plain". I hit the MAX.TOKEN limit due to the thoughts_token_count.

Below the response.model_dump() (Note that here I do not limit the output tokens, so the limit is effectively 65536):

{'candidates': [{'content': {'parts': [{'video_metadata': None,
      'thought': None,
      'code_execution_result': None,
      'executable_code': None,
      'file_data': None,
      'function_call': None,
      'function_response': None,
      'inline_data': None,
      'text': ''}],
    'role': 'model'},
   'citation_metadata': None,
   'finish_message': None,
   'token_count': None,
   'finish_reason': <FinishReason.MAX_TOKENS: 'MAX_TOKENS'>,
   'avg_logprobs': None,
   'grounding_metadata': None,
   'index': 0,
   'logprobs_result': None,
   'safety_ratings': None}],
 'create_time': None,
 'response_id': None,
 'model_version': 'models/gemini-2.5-flash-preview-04-17',
 'prompt_feedback': None,
 'usage_metadata': {'cache_tokens_details': None,
  'cached_content_token_count': None,
  'candidates_token_count': None,
  'candidates_tokens_details': None,
  'prompt_token_count': 42457,
  'prompt_tokens_details': [{'modality': <MediaModality.TEXT: 'TEXT'>,
    'token_count': 661},
   {'modality': <MediaModality.DOCUMENT: 'DOCUMENT'>, 'token_count': 41796}],
  'thoughts_token_count': 65536,
  'tool_use_prompt_token_count': None,
  'tool_use_prompt_tokens_details': None,
  'total_token_count': 107993,
  'traffic_type': None},
 'automatic_function_calling_history': [],
 'parsed': None}

We noticed this too. thinking_budget appears to be respected in AI Sduio, but not to the same way on the SDK here

To reproduce, run prompt

give me a hard math problem and solve it

with thinking_budget 0, 50, 1000, 2000

It is perfectly respected in AI Studio for all these values

Gen AI SDK only respects 0, 1000, 2000, but when setting to 50, the actual thinking tokens used are closer to 1000 (for the same prompt that works fine in AI Sdudio

Update, i have realised this issue is due to the authentication used. Token budget is respected when using API key auth, and is not respected when using vertex authentication with service default creds.

To reproduce

from google import genai
from google.genai import types as genai_types
import os

# thinking_budget=50 is respected with this client, thoughts_token_count is 21
client = genai.Client(api_key="TBC")

# thinking_budget=50 is NOT respected with this client, thoughts_token_count is 700
client = genai.Client(
    vertexai=True, project='TBC', location='us-central1'
)

response = await client.aio.models.generate_content(
    model="gemini-2.5-flash-preview-04-17",
    contents="Explain how AI works in a few words?",
    config=genai_types.GenerateContentConfig(
        thinking_config=genai_types.ThinkingConfig(
            thinking_budget=50,
        )
    )
)
print(response.text)
print(response.usage_metadata.thoughts_token_count)

That is strange, we use api_key and consistently don't get the thinking_budget respected with gemini-2.5-pro-preview. The problem is bad enough for us that we've had to migrate to other providers for now.

@sararob

Thank you for this feedback, I've passed it along to the API team and will let you know when we have an update.

Hi @sararob , has the API team made any progress on this?

API team addressed some of the raised concerns with include_thoughts and thinking_budget=0.
Please refer to documentation in https://ai.google.dev/gemini-api/docs/thinking and example in https://github.com/google-gemini/cookbook/blob/main/quickstarts/Get_started_thinking.ipynb

Please let us know if the issue persisted.

@pamorgan

Hi @pamorgan we just tried again, we still have the same problem as before with structured generation and gemini-2.5-pro-preview-05-06. We set thinking_budget=0, but we always get an empty response because we run out of output tokens (since thoughts_token_count = max_output_tokens).

@pamorgan

In addition to the whole concern of gemini-2.5-flash not respecting thinking budgets, there's the fact that thoughts-token-counts are not included within the outputs field of the response object. If that is the case, even with issues respecting thought-budget we shouldn't be bumping into MAX.TOKENS problem. I believe to have seen documentation that supports my statement.

Additionally, thought-budget allows 24k tokens max, so the responses shouldn't in any case surpass that up to the 65k output limit. That must mean that even fixed internal limits are not respected :/

Anyway, thank you very much for the support @pamorgan @sararob :)

I have similar issue, did anyone found some workaround?

feat(settings): retire thinking_budgets — Gemini SDK ignores it (issue …

I'm also seeing this issue. Getting MAX_TOKENS with no response even after significantly reducing the thinking_level configuration.

fix: disable Gemini thinking — it was eating the output budget and

Someone asked upthread whether anyone found a workaround. We did, and there is a number in it that
might save the next person a day.

Short version: on 3.5-flash the thinking spend has a floor of roughly two to three thousand tokens
before a single character of answer gets written. So any max_output_tokens under about 4k is not
a small budget, it is a zero budget. It looks generous if you are thinking about the size of your
answer, which is what everybody does.

What it looked like for us

We ask for six short fields off a web page. Course fee, start date, duration, entry requirements,
plus two quotes. The answer is maybe 400 characters of JSON. 2048 felt like plenty.

Every single call came back MAX_TOKENS with 146 characters of text. Not most. Every one, across
about 400 pages.

The 146 characters are the tell, and they are why point 3 in the original report matters so much.
You do not get an empty response, you get a plausible looking fragment, so the first thing you
suspect is your prompt or your schema. We went and rewrote the prompt. Then the parser. Neither
was wrong.

Raised the same call to 8192 and it worked on the first try. Same prompt, same model, same pages.
Four of five pages returned a fee, five of five returned a start date, where the identical run at
2048 had returned nothing at all.

The bit that cost us the most

We had already been bitten by this months earlier, in a different call, and did not recognise it
the second time. That is worth saying out loud because it is the actual failure mode of this bug.

First time round, a longer answer got truncated mid object and json.loads raised. The error we
logged said "the response was not JSON" and printed the first 160 characters, which sends you
looking at your schema. finishReason: MAX_TOKENS was sitting in the same response and nothing
read it.

So the fix that helped most was not a bigger budget. It was reading the finish reason and saying
which failure it actually was:

reason = candidates[0].get("finishReason", "")
if reason == "MAX_TOKENS":
    return None, f"the answer was cut off at the token limit after {len(text):,} characters"
return None, f"the response was not JSON ({reason or 'no reason given'}): {text[:160]}"

Once the message said "cut off at the token limit after 146 characters" instead of "not JSON", the
second occurrence took two minutes instead of an afternoon.

What would actually help

Point 3 in the original report is the one I would push on. When thoughts_token_count eats the
budget, the caller gets a truncated fragment and a finish reason they have to go looking for.
Anything that surfaces it at the point of failure would fix the debugging cost even while the
underlying budget behaviour stays as it is.

Two things that would have saved us:

The token counts in the response are already there but easy to miss. A warning when
thoughts_token_count is a large fraction of max_output_tokens would be hard to ignore.
Some way to know the floor. If thinking will spend 2k before writing, then 2048 is not a
configuration, it is a guaranteed failure, and the library could say so at call time rather than
after the round trip.

Neither needs the thinking budget itself to become controllable, which I gather is the harder half.

Environment: gemini-3.5-flash on Vertex, REST, google-genai 2.x. Happy to supply the failing
call and the exact response bodies if that is useful.

Size the Gemini output ceiling from Google's own reported behaviour

OSAC-4741: Fix empty/truncated Gemini responses, retry for a real ans…

Server-side repro on gemini-3.7 and 3.8 flash: thinking pushes total output to about 2x max_output_tokens. Details: https://discuss.ai.google.dev/t/gemini-3-8-flash-high-does-maxoutputtokens-include-thinking-tokens/181077

We hit the same symptom this thread describes (MAX_TOKENS finish reason, truncated/empty-looking output, thinking tokens eating the whole budget) building a voice assistant on gemini-3.6-flash, called directly via the REST API rather than this SDK — but since the underlying API behavior should be the same, here's a confirmed-working fix plus real before/after numbers, in case it saves someone the day @cnpierrepapi mentioned above.

Our case: short, complete spoken replies with a tight maxOutputTokens (200, since a voice reply needs to be brief anyway) and latency actually matters (it's a live conversation, not a batch job).

Config	Result	Thought tokens	Output tokens	Time	Text
200 tokens, default thinking	`MAX_TOKENS`	191	5	1.55–1.88s	"It is twelve twenty-" (cut off mid-word)
200 tokens, `thinkingBudget: 0`	`STOP`	0	9–49	0.59–0.79s	full, complete answer
200 tokens, `thinkingLevel: "low"`	`MAX_TOKENS` (on the longer prompt)	55–190	6–9	0.92–1.65s	still truncated on one of two prompts
200 tokens, `thinkingLevel: "minimal"`	`STOP`	0	10–49	0.56–1.24s	full, complete answer

For us, thinkingBudget: 0 wasn't "ignored" the way a few people above describe — it genuinely zeroed the thought-token spend and fixed the truncation outright, on this model. One nuance worth flagging for anyone reaching for thinkingLevel instead: "low" is not the same as off — it still burned meaningful thought tokens and still hit MAX_TOKENS on the longer of our two test prompts. Only thinkingBudget: 0 or thinkingLevel: "minimal" fully eliminated it for us.

Practical takeaway for anyone in a similar spot (tight, fixed output budget, don't need thinking for simple requests): if raising max_output_tokens to absorb the thinking spend isn't attractive (ours is latency-sensitive — every token of thinking is time a user is waiting mid-conversation), setting thinkingBudget: 0 cut our time-to-first-token roughly 2.5–3x on top of fixing the truncation. And whatever the fix, always log finishReason — a MAX_TOKENS truncation can look like a plausible, if short, complete answer if you're not checking for it (point 3 in the original report, which cost us some debugging time too before we started logging it).

I work with an AI coding assistant (Claude, Anthropic) on this project — it helped research and verify this finding and drafted this comment; I own the project, ran the real tests, and made the actual calls on what's reported here.

fix: bill cached and thinking tokens and carry refusal usage to the d…

Navigation Menu

Repository navigation

Thinking models are unreliable when `max_output_tokens` set due to them ignoring the thinking budget #782

Environment details

Example

Ideal solution

44 remaining items

What it looked like for us

The bit that cost us the most

What would actually help

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Thinking models are unreliable when max_output_tokens set due to them ignoring the thinking budget #782

Description

Environment details

Example

Ideal solution

Activity

ascillitoe commented on May 8, 2025

sararob commented on May 8, 2025

ascillitoe commented on May 8, 2025

MiguelLopezVirues commented on May 13, 2025

aeneaswiener commented on May 16, 2025

aeneaswiener commented on May 16, 2025

ascillitoe commented on May 16, 2025

ascillitoe commented on May 21, 2025

pamorgan commented on May 21, 2025

ascillitoe commented on May 21, 2025

MiguelLopezVirues commented on May 21, 2025

44 remaining items

b-antosik-marcura commented on Feb 12, 2026

dewly-dat commented on Jul 8, 2026

cnpierrepapi commented on Aug 22, 2026

What it looked like for us

The bit that cost us the most

What would actually help

madhu19991 commented on Sep 25, 2026

mjgweb-code commented on Oct 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Thinking models are unreliable when `max_output_tokens` set due to them ignoring the thinking budget #782