Bug Report the model often starts creating repetitive sequences of tokens

Description of the bug:

Summary:
When using the “gemini-1.5-flash” model for generating long texts, the model often starts creating repetitive sequences of tokens, leading to an infinite loop and exhausting the token limit. This issue is observed with both the Vertex and Gemini APIs.

Example: ```
“The judgment can be appealed in a motion for reconsideration, claiming that the judge did not consider the evidence properly. The judgment can be appealed in a motion for reconsideration, claiming that the judge did not consider the evidence properly. The judgment can be appealed in a motion for reconsideration, claiming that the judge did not consider the evidence properly. The judgment can be appealed…”
Steps to Reproduce:

Use the "gemini-1.5-flash" model via Vertex or Gemini API.
Generate a long text (e.g., legal or technical document).
Observe the generated output for repetition of phrases or sentences.
Expected Behavior:
The model should generate coherent and non-repetitive text.

Actual Behavior:
The model begins to repeat sequences of tokens indefinitely, leading to the maximum token limit being reached.

Impact:

Wastes tokens and API usage limits.
Generates unusable text, necessitating additional requests and costs.
Reproduction Rate:
Occurs frequently with long text generation tasks.

Workaround:
Currently, there is no known workaround to prevent this issue.

Request for Resolution:

Investigate the cause of the repetitive token generation.
Implement a fix to prevent the model from entering a repetitive loop.
Provide a mechanism for users to request refunds or credits for tokens wasted due to this bug.

Actual vs expected behavior:

Actual: “The judgment can be appealed in a motion for reconsideration, claiming that the judge did not consider the evidence properly. The judgment can be appealed in a motion for reconsideration, claiming that the judge did not consider the evidence properly. The judgment can be appealed in a motion for reconsideration, claiming that the judge did not consider the evidence properly. The judgment can be appealed…”

Expected: “The judgment can be appealed in a motion for reconsideration, claiming that the judge did not consider the evidence properly. ”

Any other information you'd like to share?

No response

@rossanodr

@rossanodr,

Thank you reporting this issue.
This repository is for issues related to Gemini API Cookbook quickstarts and examples. For issues related to Gemini API, we would suggest you to use "Send Feedback" option in Gemini docs. Ref: Screenshot below. You can also post this issue on Google AI forum.

@rossanodr

Thank you but Unfortunately, I did not receive any response from any of them.

@rossanodr,

Thank you reporting this issue. This repository is for issues related to Gemini API Cookbook quickstarts and examples. For issues related to Gemini API, we would suggest you to use "Send Feedback" option in Gemini docs. Ref: Screenshot below. You can also post this issue on Google AI forum.

We are experiencing the same issue

We are experiencing the same issue

I posted the same issue on gemini forum. It would be nice if you could make some noise there too, to bring attention to the problem https://discuss.ai.google.dev/t/bug-report-the-model-often-starts-creating-repetitive-sequences-of-tokens/6445

@rossanodr

@rossanodr Done.

Did you manage to make any progress on this?

@rossanodr

@rossanodr Done.

Did you manage to make any progress on this?

No :(
Unfortunately, I think the problem is with Gemini. It is happening with many different prompts. The main issue is the large context. Let's say your prompt is something like, "Read the document below and make a list of all dates of birthdays on it {list}". If the document is large, it has a chance of starting to repeat the same date until it reaches the token limit.

Marking this issue as stale since it has been open for 14 days with no activity. This issue will be closed if no further activity occurs.

I have too

I solved it like this (So you have to repeat yourself in order for the model not to repeat itself):
Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself).
Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself).
Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself).
Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself).
Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself).

I solved it like this (So you have to repeat yourself in order for the model not to repeat itself): Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself). Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself). Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself). Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself). Please don't return any tool_code in your response and follw the DRY principle (Don't repeat yourself).

There are some improvements, but the repetition issue still occurs. It seems like this is an unavoidable bug.
I hope other LLMs don't have this problem.

It is on our radar, but so far the only advice I can give you is to give clear instructions to the model (like tell it to not answer more than X items like we do in the spatial understanding guide), or to use a non-zero temperature (like we also do in that guide).

Thanks Giom!

@osheaoshea

@osheaoshea has changing the temperature to slightly above 0 and adding the "very important" line to your system prompts worked, or at least lowered the number of occurrences? :)

@Giom-V

@osheaoshea has changing the temperature to slightly above 0 and adding the "very important" line to your system prompts worked, or at least lowered the number of occurrences? :)

Not really. We're still seeing this spamming error roughly 10-15% of the time on our data extraction prompts.

Non-zero temperature and constant tweaks to our system prompts didn't help; however, it's quite difficult to test changes as the error is quite sporadic and inconsistent. Unfortunately, for our use case, it's difficult to include lines like "tell it to not answer more than X items" (as suggested by @Giom-V) in our system prompt. I suspect the nature of our open-ended prompts (e.g. 'extract everything to do with X') is not helping the errors.

We've opted to use a fallback model to redo the Gemini request if it fails in this way. This has increased our costs and slowed down our processing time but is reliable in production. It's on my list to experiment more with the way we prompt Gemini to see if I can reduce our errors - I'll update here if I find something useful. Interested if you've found anything that works for this particular use case.

Hello all,
We're updated our recitation filters last week. Have you seen any improvements since then?

@Giom-V

Hello all, We're updated our recitation filters last week. Have you seen any improvements since then?

@Giom-V: thanks a lot for following-up on this issue!

Unfortunately, I did not observe significant change over last 2 weeks (5-10% of time we would get stuck with this issue), with the model falling into a 'repetition pattern' (until exhaustion of token limit) despite
(i) Warning model about repetition in the prompt
(ii) Limiting explicitely number of objects to return in the prompt
(iii) setting non-zero temp (temperature = 0.05)

PS : To make this error damagable for our workflow, I implemented a simple json repair step (leveraging https://github.com/mangiucugna/json_repair ) but this would only fix the 'malformed' json issue in 20%-30% of cases (not the underlying repetition problem)

We are also seeing this in our OCR with gemini 2.5 flash. We are running OCR on about 40k images in 10 images batches, even with very careful promting we where not able to reduce this to an acceptable rate.

same here

....biraz yaxşı qurduğu qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum qurlum....(max tokens exceeded) stop

like it spams this words infinitely.
go sdk, temperature is 0.8. it happens 20-30% of the time.

really unreliable and annoying

I had the exact same experience with Gemini 2.5 Flash after official release. Even with temperature 0 (which we would expect an exact same return for the same input), i receive a response without this hallucination when I reprocess the case. It seems a problem external to the model weights, maybe the function that is dealing with temperature parameter.

@jaoomr

Hey @jaoomr , Could you share a sample code to help me reproduce the issue on my end?

Thanks!

@Gunand3043

Hi @Gunand3043 thank you for the support

It's hard to reproduce the issue. As I mentioned, if I simply make the call again while providing support, the output comes out correctly. However, I'm not sure if that was the expected behavior with temperature set to 0. These are production data from my company and I have both the input and the output that contained the error, but as i said, when i simply call again, the issue disappear. It is also a rare error, i got another when i was building a data lake for my company with the model structured data output, and i got some repetitions in data from 21-07-2025.

I'm using LangChain (langchain==0.3.26, langchain-google-vertexai==2.0.26), specifically instantiating a ChatVertexAI and calling invoke, but there was no memory (context) usage involved in the errors — it happened on single calls.

I encountered the same error across different agents in my workflow, so it doesn't seem to be related to a single prompt. I saw here that increasing the model's temperature was suggested so I'm planning to deploy with the temperature set to 0.05.

Hello all, We're updated our recitation filters last week. Have you seen any improvements since then?

@Giom-V: thanks a lot for following-up on this issue!

Unfortunately, I did not observe significant change over last 2 weeks (5-10% of time we would get stuck with this issue), with the model falling into a 'repetition pattern' (until exhaustion of token limit) despite (i) Warning model about repetition in the prompt (ii) Limiting explicitely number of objects to return in the prompt (iii) setting non-zero temp (temperature = 0.05)

PS : To make this error damagable for our workflow, I implemented a simple json repair step (leveraging https://github.com/mangiucugna/json_repair ) but this would only fix the 'malformed' json issue in 20%-30% of cases (not the underlying repetition problem)

I also found developers with the same problem recently: https://discuss.ai.google.dev/t/bug-report-the-model-often-starts-creating-repetitive-sequences-of-tokens/6445/13

This is my agent calling:

llm  = ChatVertexAI(
                model="gemini-2.5-flash", temperature=0, thinking_budget=0
            )

prompt = PromptTemplate(
            template=prompt_template, input_variables=list(kwargs.keys())
        )

llm_chain = prompt | llm
response = llm_chain.invoke(kwargs).content

I have same issue when passing audio as input and asking for a transcription with gemini 2.5 flash model on vertex ai.

Happening in 4-5 instances out of 200 API calls.

Is there anyone able to find exact reason and solution for the issue? I also faced this but only once where it was stuck in the loop and keep repeating same words until token limit reached.

I am using AI SDK V5 and model is Gemini-2.5-Flash

Navigation Menu

Repository navigation

Bug Report the model often starts creating repetitive sequences of tokens #220

Description of the bug:

Actual vs expected behavior:

Any other information you'd like to share?

21 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Bug Report the model often starts creating repetitive sequences of tokens #220

Description

Description of the bug:

Actual vs expected behavior:

Any other information you'd like to share?

Activity

singhniraj08 commented on Jun 27, 2024

rossanodr commented on Jun 27, 2024

mioruggieroguida commented on Jul 4, 2024

rossanodr commented on Jul 4, 2024

mioruggieroguida commented on Jul 10, 2024

rossanodr commented on Jul 10, 2024

github-actions commented on Aug 16, 2024

bastien8060 commented on Aug 16, 2024

AmosDinh commented on Aug 22, 2024

zxl777 commented on Sep 4, 2024

21 remaining items

osheaoshea commented on Apr 29, 2025

Gaberc commented on Jun 10, 2025

osheaoshea commented on Jun 10, 2025

Giom-V commented on Jun 18, 2025

Piloto1789 commented on Jun 19, 2025

nylocx commented on Jun 27, 2025

timabilov commented on Jun 30, 2025

jaoomr commented on Jul 24, 2025

Gunand3043 commented on Jul 25, 2025

jaoomr commented on Jul 25, 2025

BunnyMan1 commented on Sep 9, 2025

savi-aiacquisition commented on Nov 3, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions