Repository navigation
Make conversation history summarization threshold public #270528
Description
Problem
The github.copilot.chat.advanced.summarizeAgentConversationHistoryThreshold setting appears to trigger when the context window fills up. It is currently marked as INTERNAL, making it unreliable to users who need to optimize LLM performance. Research and anecdotal evidence suggest that filling the entire context window can degrade LLM performance, with some organizations recommending starting new chats at ~40% context utilization.
I've found that performance degrades, sometimes significantly, when I am filling up the context window. This solution would mean I would summarize data more often, but also might improve both quality and latency.
Proposed Solution
Make the setting public and enhance it with three configuration modes:
1. Hard token number (existing)
{
"github.copilot.chat.summarizeAgentConversationHistoryThreshold": 80000
}2. Percentage-based threshold
{
"github.copilot.chat.summarizeAgentConversationHistoryThreshold": "40%"
}For a model with 200K max prompt tokens, this would trigger at 80K tokens.
3. Per-model configuration
{
"github.copilot.chat.summarizeAgentConversationHistoryThreshold": {
"gpt-4o": "40%",
"claude-3.5-sonnet": 100000,
"default": "50%"
}
}(Note: the above shows three different data types for that key. Having only one key makes it easy to find, but makes the code more complex for validation. Having three keys (such as github.copilot.chat.summarizeAgentConversationHistoryThresholdPercent) would make them harder to find, but easier for the code)
Rationale
- Different models have different performance characteristics at various context utilization levels
- Users should be able to optimize for quality vs. context retention based on their use case
- Percentage-based configuration is more portable across models with different context windows
Implementation Notes
The existing infrastructure in src/platform/endpoint/node/chatEndpoint.ts already provides modelMaxPromptTokens, which can be used to calculate percentage-based thresholds. The configuration system in src/platform/configuration/common/configurationService.ts supports complex types that could accommodate this schema.
Activity
Ovid commentedon Oct 15, 2025
I realize this isn’t a top priority (you all have a lot on your plates), but this issue is really hurting my workflow.
I’m doing a lot of spec-driven development, and when a task is larger than expected, the context window fills up and the model starts to drift. Guardrails get ignored, code quality drops, and sometimes the model even gets a bit “irritable.” (Claude Sonnet 4.5 once got somewhat snippy after a long session—definitely unexpected behavior.)
Right now, history summarization feels like Java's stop-the-world garbage collection: it halts everything, burns compute, and I leave to get a coffee. Allowing more frequent but smaller summaries would be like incremental GC. It would be slightly more frequent interruptions, but far less disruptive.
Reasoning
In short: smaller, smarter summarization could turn an occasional “context cliff” into smooth terrain. Even a hidden setting or experimental flag would help those of us doing long-form, high-context work.
Ovid commentedon Oct 16, 2025
And possibly give us the opportunity to fine-tune the summarization prompt as well, but allowing us to revert to the original version if our version is suboptimal.
roblourens commentedon Dec 11, 2025
That setting can be used as a dev setting today but it's easy to shoot yourself in the foot so I wouldn't promote it.
And yes, there are still issues with summarization that we need to revisit, there are some open issues about this.