[BUG] OpenAI Compatible Qwen3.8:27B condense context token usage

Describe the bug
When using Qwen3.8:27B running in llama.cpp as an openai compatible endpoint, Although I have the max context window set to 262144 in the server, when I attempt to condense context at 181.1k tokens used, I get an error for Failed to condense context - (304897 tokens) exceeds the available context size 262144 tokens

To Reproduce
Steps to reproduce the behavior:

Run Qwen3.8:27B server via llama.cpp using the following command:

llama-server.exe --ctx-size 262144 -m "C:\models\Qwen3.8-27B-Q4_K_M.gguf" --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --temp 1.0 --port 8080 --host 0.0.0.0 --chat-template-file "C:\models\chat_template.jinja" -ngl 99 -fa 1 --cache-type-k q4_0 --cache-type-v q4_0 --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1

setup OpenAI Compatible provider to connect to that server with Model Reasoning Effort High.
Set Context Window Size in Zoo Code to 262144
Continue a conversation until token usage bar at the top shows 181.1k / 262.0k
Click the Intelligently Condense Context button
See error Failed to condense context 304897 tokens used.

Expected behavior
If Token usage for the current conversation was actually that long, I believe my llama.cpp server would have rejected it before this point. Or if the reason for the error is the actual request to intelligently condense context would take that many tokens, then I would expect to be able to see that number somewhere so that I can condense before it grows beyond the point where I have to let it fail and truncate the conversation.

What version of zoo are you running
3.78.0

Additional context
I just wanted to say I appreciate what you guys are doing with this project. It's extremely helpful and intuitive.

The same thing is happening to me. I've spent the past two hours looking at prompt dumps (--log-prompts-dir) and history_item.json/ui_messages.json. I've realized that every task I've ran today while not monitoring them had condensing fail and truncated instead. When trying to manually condense now at 106,500 / 200,000, it fails with an attempt to send 281,671 tokens. I've had it attempt to send over 470,000 tokens when closer to my context limit.

My reasoning effort is at Medium, using the UD-Q4 quant of 3.8 27B. This happens instantly, it's not like it tries to condense for a while and then fails; llama is flat-out rejecting the initial prompt as too large:

46.00.600.546 E srv send_error: task id = 3, error: request (281671 tokens) exceeds the available context size (200192 tokens), try increasing it

You can see that the context really was just ~106,500 right before this:

561.49.931.139 I slot release: id 0 | task 2160 | stop processing: n_tokens = 106543, truncated = 0

Something interesting - when saving the prompts, the file is over 15,000 lines when trying to use the condense button. If I manually send the condense prompt myself (that is, I paste it as part of the conversation), the entire prompt is 6,600 lines which is much more reasonable/expected. The prompt when I manually sent it contained soft-truncated file reads, terminal outputs, etc. The condense button has full, raw outputs of all file reads, terminal outputs, tool outputs, etc. If a tool read a 3,000 line file 20 turns ago, all 3,000 lines are included when I use the condense button. When sending the prompt manually, it's truncated. I don't know if the typical condense button is meant to include full files, full outputs, etc or not but that seems to be the primary difference here. I can provide both files (the prompts the model received when using the button vs sending it manually) if it helps.

I had Gemini analyze both files because they're too massive for me to do more than skim; I haven't independently verified this, but it added:

Comparing the two files side-by-side reveals three major differences beyond un-truncated tool calls:

Mass Duplication of Message History
The Manual File: Contains a single, linear chronological timeline of the conversation from turn 1 to your final message.

The Condense File: Duplicates entire chunks of the history. Because the extension hit a serialization error at lines 87–94, it restarted the message array assembly mid-stream without resetting its buffer. It appends the first half of the conversation history, hits the cutoff, and then appends the entire history array a second time underneath it.

Conversation Depth and Bounds
Both files start at the exact same initial task setup prompt (message 0).

Both files end at the same final turn state.

The Difference: The manual file moves cleanly from start to end once (~6,600 lines). The condense file starts at message 0, traverses forward, loops back to message 0 mid-prompt, and processes forward to the end a second time (~15,000 lines).

Duplicated System Reminders & Capability Headers
In the failed condense dump:

Tool Schemas: The JSON definitions for system capabilities (read_file, apply_diff, execute_command) appear at the top of the file, and then appear again in full inside the second appended block.

Injection: System reminders and environment states (active workspace details, git status, editor paths) are injected twice—once near the middle cutoff and again at the very bottom right before the closing instructions.

The issue isn't just that past file reads are being restored to raw text; the extension's condensation serializer is literally appending two complete copies of your task context into a single wire request.

The only solution at the moment seems to be manually prompting the model to condense and starting a new task with the output.

Edit - A few things I've tried that did not make a difference:

Disabling MTP, ngram, both
Disabling reasoning
Returning to my known, stable Qwen3.6 27B setup (for some reason this resulted in the failed prompt being over 450,000 tokens? Loading the 3.8 27B model back in and hitting the condense button yet again had it back to ~281,500, so somehow the model and/or it's configuration is relevant?)
Reverting to Zoo version 3.76.0

As far as I can tell, once a task gets into this state it's stuck there.

The same thing is happening to me. I've spent the past two hours looking at prompt dumps (--log-prompts-dir) and history_item.json/ui_messages.json. I've realized that every task I've ran today while not monitoring them had condensing fail and truncated instead. When trying to manually condense now at 106,500 / 200,000, it fails with an attempt to send 281,671 tokens. I've had it attempt to send over 470,000 tokens when closer to my context limit.

My reasoning effort is at Medium, using the UD-Q4 quant of 3.8 27B. This happens instantly, it's not like it tries to condense for a while and then fails; llama is flat-out rejecting the initial prompt as too large:

46.00.600.546 E srv send_error: task id = 3, error: request (281671 tokens) exceeds the available context size (200192 tokens), try increasing it

You can see that the context really was just ~106,500 right before this:

561.49.931.139 I slot release: id 0 | task 2160 | stop processing: n_tokens = 106543, truncated = 0

Something interesting - when saving the prompts, the file is over 15,000 lines when trying to use the condense button. If I manually send the condense prompt myself (that is, I paste it as part of the conversation), the entire prompt is 6,600 lines which is much more reasonable/expected. The prompt when I manually sent it contained soft-truncated file reads, terminal outputs, etc. The condense button has full, raw outputs of all file reads, terminal outputs, tool outputs, etc. If a tool read a 3,000 line file 20 turns ago, all 3,000 lines are included when I use the condense button. When sending the prompt manually, it's truncated. I don't know if the typical condense button is meant to include full files, full outputs, etc or not but that seems to be the primary difference here. I can provide both files (the prompts the model received when using the button vs sending it manually) if it helps.

I had Gemini analyze both files because they're too massive for me to do more than skim; I haven't independently verified this, but it added:

Comparing the two files side-by-side reveals three major differences beyond un-truncated tool calls:

Mass Duplication of Message History
The Manual File: Contains a single, linear chronological timeline of the conversation from turn 1 to your final message.

The Condense File: Duplicates entire chunks of the history. Because the extension hit a serialization error at lines 87–94, it restarted the message array assembly mid-stream without resetting its buffer. It appends the first half of the conversation history, hits the cutoff, and then appends the entire history array a second time underneath it.

Conversation Depth and Bounds
Both files start at the exact same initial task setup prompt (message 0).

Both files end at the same final turn state.
The Difference: The manual file moves cleanly from start to end once (~6,600 lines). The condense file starts at message 0, traverses forward, loops back to message 0 mid-prompt, and processes forward to the end a second time (~15,000 lines).

Duplicated System Reminders & Capability Headers
In the failed condense dump:

Tool Schemas: The JSON definitions for system capabilities (read_file, apply_diff, execute_command) appear at the top of the file, and then appear again in full inside the second appended block.
Injection: System reminders and environment states (active workspace details, git status, editor paths) are injected twice—once near the middle cutoff and again at the very bottom right before the closing instructions.
The issue isn't just that past file reads are being restored to raw text; the extension's condensation serializer is literally appending two complete copies of your task context into a single wire request.

The only solution at the moment seems to be manually prompting the model to condense and starting a new task with the output.

Edit - A few things I've tried that did not make a difference:
* Disabling MTP, ngram, both

* Disabling reasoning

* Returning to my known, stable Qwen3.6 27B setup (for some reason this resulted in the failed prompt being over 450,000 tokens? Loading the 3.8 27B model back in and hitting the condense button yet again had it back to ~281,500, so somehow the model and/or it's configuration is relevant?)

* Reverting to Zoo version 3.76.0
As far as I can tell, once a task gets into this state it's stuck there.

Thanks for the much more in-depth reply than my initial post. After reverting to 3.76.0 have you continued to see the issue with 3.8 in new chats? I've mostly been requesting the model write continued plans in a markdown file before my context limit runs out and starting a new chat. it's a little annoying, but might just be a difference in the way 3.8 handles prompts and history?

Thanks for the much more in-depth reply than my initial post. After reverting to 3.76.0 have you continued to see the issue with 3.8 in new chats? I've mostly been requesting the model write continued plans in a markdown file before my context limit runs out and starting a new chat. it's a little annoying, but might just be a difference in the way 3.8 handles prompts and history?

When reverting didn't seem to help, I updated back to the current release. Oddly enough, I've had another task running in an identical setup (Still 3.8 27B, nothing else is different) and I just came back to check on a long task that's been running and it's condensing right now. It accepted the prompt, ingested it, and is currently spitting out tokens for the output. I honestly don't know what's different here - it triggered condensing automatically at 163,600 / 200,000 tokens. It seems like this isn't an "always broken" problem, but rather a "something happens that breaks condensing permanently for the task" one - it's just a matter of figuring out what that "something" is. The only big difference I can see is this task didn't have any huge file reads; while it ran a ton of console commands, file reads were allocated to subtasks. My last one that failed had read several large files - do you remember if the task you ran where condensing failed had multiple and/or large file reads? All I can think to do now is try to narrow down the cause.

The condense just completed successfully as I finished typing this and looks normal outside of the analysis and summary tags being visible.

For what it's worth, I too am experiencing this problem exactly as described using 3.80.

do you remember if the task you ran where condensing failed had multiple and/or large file reads? All I can think to do now is try to narrow down the cause.

Hmm, I'd say this has been fairly consistent since I switched from 3.6 to 3.8 - One project I was working on had several very large file reads, and another was creating a project in an empty folder adding multiple files if you would count that as multiple file reads.
I'm going to test using a different gguf author other than unsloth, and maybe try something other than llama.cpp to see if those have any effect on it. Like you mentioned though, once a task is having that issue, it's already too late - switching to another model to attempt to condense doesn't fix it.

Alright, so I spent the last few hours working with Qwen:3.8 looking at Zoo Code's code, and added logging in a test environment, and it seems to think the problem is calls to condense don't strip out reasoning blocks the same way a normal conversation does. So, although the status bar at the top shows a smaller percentage of the full context window, when you try to condense, it sends everything that's collected thus far and fails. It also claims there's an issue with how it handled reading a very large file where it added a massive chunk of context, and that was stuck from that point on, so every attempt at condensing would fail from there.

I'll probably have it implement the suggested fixes for my local branch and see if that helps, but I would be hesitant to create a pull request without testing multiple other features.

I also tested changing to ggml-org's version of Qwen:3.8-Q4_K_M still using llama.cpp and see the same behavior without MTP turned on.

For anybody watching this, the fixes Qwen:3.8 implemented seem to have addressed the issue so far. This will require a good bit more testing most likely, but I pushed the changes out to a fork under my name in github: https://github.com/RemainIndoors1/Zoo-Code/tree/fix/condense-context-token-inflation

Anyway, I tested filling up context to 160,000 tokens and condensed manually with no issues. I believe these changes would affect all reasoning models, and I've only tested for Qwen:3.8 so far.

Also worth noting, I'm mostly a .Net developer with little-to-no experience with typescript, so I'm leaning pretty heavily on Qwen in this case. If anybody else wants to test that branch though, it's pretty easy, you can pull that branch and open the folder in VS Code, then install the dependencies via pnpm install and run it in a new window with F5 (the instructions are in the README). But for now, I'm going to keep working on my own stuff, and I'll report back if I see any issues.

well, RIP, looks like removing reasoning wasn't the whole story. I let a conversation run until it automatically condensed context at about 220,000 / 262,000 and it failed, but the attempted condense request was smaller overall:

Failed to condense context
Cannot condense context: the conversation (278015 tokens) is larger than the model's context window allows for a condense request (236169 tokens). Truncating the oldest messages instead.

well, RIP, looks like removing reasoning wasn't the whole story. I let a conversation run until it automatically condensed context at about 220,000 / 262,000 and it failed, but the attempted condense request was smaller overall:

Yeah, like I said in an earlier reply I also tried quite a few different changes to determine the cause and it didn't seem to be any one single thing. I do have the two prompts saved - the first when hitting the condense button, and the second being the identical state where the condense prompt was submitted as part of the normal conversation - and its almost a 300% difference; the only thing that I can think of that would cause such an explosion in size is tool outputs being included verbatim (like file reads, for instance). The problem is that I don't know what baseline, normal behavior is supposed to be when automatic condensing is triggered. I assumed that tool call outputs like that would be condensed/truncated if not outright ignored, relying on the model's response to the data rather than the data itself. Even if that's the case, though, and we determine that it really is file reads and other things being incorrectly included at full size, that's only half of the puzzle solved - the next question is why. Considering that nothing has changed for me other than switching from 3.6 to 3.8 I would have to imagine that there is some sort of difference in how tools are being used that results in zoo failing to detect tool output in certain circumstances when building the condense prompt? These are wild guesses, though, not backed by any evidence.

Can you tell I'm ADHD? I bet you can.

lol. don't feel bad, I have OCD with a flawed attention span modifier. I work on a project until I find something that would help that project, then switch to that. I'm currently about 5 projects deep.

I'll take your research into consideration and see if I can get Qwen to help figure out what all it would take to make condense work the way it's "supposed to", but like you said, it's hard to tell what baseline, normal behavior is supposed to be without a much greater understanding of the project. Re: what's different with Qwen:3.8? - one thing is apparently there are some hard-coded models in Zoo Code that decide what to do for things like removing reasoning during a call to condense context, and Qwen:3.8 isn't one of them, so that's definitely part of the problem.

Overall though, it does seem like reasoning was a huge chunk of it. My first test of 181,000 exploding to more than 300,000 tokens, and after changes 220,000 only jumping to about 278,015 is a big improvement.

Also, there is a slider in Zoo Code settings to change what percentage of context usage to trigger a condense automatically, so the current branch is arguably usable if it's configured right, but I'd still like to figure out how to fix the overall problem.

And I think I have a better understanding of where the problem is. The token count you see from that error isn't actually reported back from the API you're connected to. it's an estimated token count applied to all models using Tiktoken instead of the actual tokenizer's count from the API, then they apply a 1.5 x Fudge Factor (

Zoo-Code/src/utils/tiktoken.ts

Line 5 in 78c712a

const TOKEN_FUDGE_FACTOR = 1.5

) because it's a generic token count that isn't the same for all models since we're using a Generic OpenAI Compatible endpoint, and it's accounting for any model you might be connecting to.

So, in theory the 278015 tokens it's reporting and failing in my last test is not the actual number of tokens, but an estimated token count based on what tiktoken returns. On the flip side of that, the token tracker at the top of the chat is using the actual token count reported from the API, so when it attempts to condense, it errors out and never sends to the API.

TLDR; removing reasoning might have been (most of) the actual solution (TM), but it's not all inclusive, because Zoo Code prevents an attempted condense based on the estimated token size. So, using Generic OpenAI Compatible endpoint isn't ideal if you want an accurate representation of actual token usage.

I'm going to see what Qwen suggests, but so far it looks like that's the problem.

So, my current proposed solution to this problem since estimated token counts aren't reliably accurate for all models, and by the time you try to send a condense request it might already be too late - is when a condense request fails, instead of truncating, try condensing subsections of the conversation in a few passes to avoid any message loss. I'm not sure how the maintainers of this project would feel about that, but in theory that would be a viable solution to the problem, so I'm rolling forward with that as a test for now. Might be a hard deny if it ever gets to PR review. lol

Just an update on that, initial testing for the condense in subsections approach seems to work so far. I setup a 50,000 token limit llama.cpp server and ran a conversation until it automatically condensed a few times without truncation. Then, I verified with the model in chat that it still had a pretty good grasp of the conversation up until that point. But, it's getting pretty late and there are still some changes I'm working through to avoid affecting other models that are succeeding with the full context, so I'm not going to push an update to the fork I created until after I've had a chance to test more in depth.

The root of the problem is that manageContext() computes prevContextTokens = totalTokens + lastMessageTokens [lines 318–326] where totalTokens is a stale provider-reported count and only the last message is counted locally. Most of a new tool-result batch is invisible to the trigger, so condense fires late — when the context is already far over the window.

I'm currently testing a fix using a cascading approach where we first estimate the token count that wold be sent to the model for a condense operation - including tool calls and file reads. If count is below the model limit then proceed as normal; if not then apply a folding algorithm to stale read-file results inline keeping the last N reads full. Repeat token estimation and if still over test if any single tool-result exceeds the token window and truncate with an omission marker. If the token estimation is still too high then apply a segmented subsection condense with a final merge condense over the sub-summaries.

I'm a C++ dev so ts is not my happy place. I'll report if it works. Looks positive so far.

The root of the problem is that manageContext() computes prevContextTokens = totalTokens + lastMessageTokens [lines 318–326] where totalTokens is a stale provider-reported count and only the last message is counted locally. Most of a new tool-result batch is invisible to the trigger, so condense fires late — when the context is already far over the window.

I'm currently testing a fix using a cascading approach where we first estimate the token count that wold be sent to the model for a condense operation - including tool calls and file reads. If count is below the model limit then proceed as normal; if not then apply a folding algorithm to stale read-file results inline keeping the last N reads full. Repeat token estimation and if still over test if any single tool-result exceeds the token window and truncate with an omission marker. If the token estimation is still too high then apply a segmented subsection condense with a final merge condense over the sub-summaries.

I'm a C++ dev so ts is not my happy place. I'll report if it works. Looks positive so far.

Yeah, that's not dissimilar from the road I was headed down. Apologies, I ramble a lot and leave out a lot of relevant details. 8 hours of work + 8 hours of this makes for a long day. I'm happy to defer to your version if you get something that works, but I may not be far off.

FYI, I pushed a version out to the fork I created under my name with the rolling condense approach if API calls fail for context size. I had to create it as a fork, because it blocked me creating a new branch in the main repo. I'm not really accustomed to modifying opensource github projects, so that might be normal for all I know.

I haven't had a chance to thoroughly test it, so if anybody wants to critique the changes or test them yourself, feel free. I have that branch installed directly in VS Code now, so I'll make changes if I run into problems, but assuming all goes well, I probably won't be blowing up this thread everyday with updates anymore. Otherwise, I'll check back to see if there's any interest in me trying to create a pull request for those changes.

I thought it might be that the token count was unreliable as well, but it perfectly matched the actual terminal output of the running model. If it said I was at 106,000 tokens and then a condense failed by saying "260,000 tokens is greater than your 200,000 context", the console output immediately beforehand stated that I was at 106,000 tokens.

I think it's a combination of multiple issues. Reasoning gets added to the request, which honestly I'm taking Qwen's word for it, because it's a lot of code to dig through, but supposedly certain hard-coded models were already doing that, which exlcudes Qwen3.8. The branch I have trims it out every time. I think plap's mention of prevContextTokens being miscalculated was also a factor, and using tiktoken for the estimated usage vs the token count in the bar at the top showing what the server's logs actually report. Also, the Fudge Factor of 1.5 x to decide whether or not to even try to send the condense request was arguably a part of it, not to mention the tool calls it rolls into the final condense message. But all of those alone didn't fix it, as the estimated tokens was still too high so it failed on an auto-condense.

The rolling condense was a failsafe measure I suggested and Qwen implemented to make sure it always at least attempts a condense for Qwen3.8 and thus far I've run through 2 full condenses with server context of 262k. They both worked as expected with the usual loss of detail, but Qwen picks back up and re-reads all of the files if it's in the middle of a task. The only downside so far is a full condense takes like 7 minutes on a 5090 with 128 GB of RAM. I could probably avoid that by changing the setting to auto-condense at a lower percentage to avoid the rolling condense, but I haven't tested that yet. *Edit - I have tested that now. a condense at 210k tokens ran about 3-4 minutes faster, and I've been through about 5 condenses in a single conversation now with no Truncation.

I'm really not a full time vibe coder, just kind of lazy when it comes to trying to fix tools and I haven't had the time to devote to reading through how all of Zoo Code works yet.

Dealing with the same issue. I've made a few vibe coded attempts here, but none that have entirely fixed it.
https://github.com/bitflippersanonymous/Zoo-Code/tree/context-compression

Dealing with the same issue. I've made a few vibe coded attempts here, but none that have entirely fixed it. https://github.com/bitflippersanonymous/Zoo-Code/tree/context-compression

I've had my fork installed directly in VS Code for at least a few weeks and haven't had any problems with context condenses if you want to try that. I've been through at least 30-40 full condenses at this point and no failures.

One quirk - after several condenses in the same conversation, sometimes the Zoo-Code panel goes full gray and you have to reload the window. I don't think any of those changes would have caused that, but not 100% sure. Also, it's at least a few releases behind the master branch.

I haven't pursued trying to create a PR, because that would be a lot of work on my part reading through and understanding everything well enough to discuss PR feedback, nobody's asked for it, and it's working fine for my purposes.

I've had my fork installed directly in VS Code for at least a few weeks and haven't had any problems with context condenses if you want to try that.

Thank you! I'll try that and see if I can 'vibe merge' the 'no images in context' part too.

fix(condense): strip reasoning and add chunked fallback for oversized…

fix(condense): harden context condensation against oversized context

@RemainIndoors1

@RemainIndoors1 the "panel goes full gray" occurs in mainline too, would be unrelated to your fork

fix(condense): summarize only the API-visible history

fix(condense): summarize only the API-visible history (Zoo-Code-Org#2300

Navigation Menu

Repository navigation

[BUG] OpenAI Compatible Qwen3.8:27B condense context token usage #1342

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[BUG] OpenAI Compatible Qwen3.8:27B condense context token usage #1342

Description

Activity

Extremity commented on Aug 23, 2026

RemainIndoors1 commented on Aug 23, 2026

Extremity commented on Aug 23, 2026

plap1016 commented on Aug 23, 2026

RemainIndoors1 commented on Aug 23, 2026

RemainIndoors1 commented on Aug 23, 2026

RemainIndoors1 commented on Aug 23, 2026

RemainIndoors1 commented on Aug 24, 2026

Extremity commented on Aug 24, 2026

RemainIndoors1 commented on Aug 24, 2026

RemainIndoors1 commented on Aug 24, 2026

RemainIndoors1 commented on Aug 25, 2026

RemainIndoors1 commented on Aug 25, 2026

plap1016 commented on Aug 25, 2026

RemainIndoors1 commented on Aug 25, 2026

RemainIndoors1 commented on Aug 26, 2026

Extremity commented on Aug 27, 2026

RemainIndoors1 commented on Aug 28, 2026

bitflippersanonymous commented on Sep 14, 2026

RemainIndoors1 commented on Sep 14, 2026

bitflippersanonymous commented on Sep 14, 2026

Wil-Storch commented on Sep 23, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions