Repository navigation
fix(core): bound compaction request size - #36589
kitlangton wants to merge 2 commits into
Conversation
|
Hi! Thanks for this work — bounding the serialized request below the 10 MiB body limit and chunked summarization looks like a solid fix for the HTTP 413 wedge. I'm looking into a related but (I believe) distinct failure mode: My idea is a model-free pre-prune before compaction (drop oldest messages / drop tool outputs / strip media locally, purely by size), so there's always some headroom to run a bounded summarization afterwards. Does your approach leave room for that, or would it conflict with the compaction flow you've built here? Happy to coordinate and avoid overlap. |
|
Automated PR Cleanup Thank you for contributing to opencode. Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions. This PR was closed because it matched the following cleanup criteria:
PRs created within the last month are not affected by this cleanup. If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate. Thanks again for taking the time to contribute. |
What
Large sessions no longer become permanently wedged when their model context fits but the serialized inference request exceeds the service's 10 MiB body limit.
Before, auto-compaction only considered model token pressure. A session at roughly 240k of a 1M-token context could therefore skip compaction and receive HTTP 413 forever. Compaction itself could also exceed an upstream body limit because it summarized the entire old history in one request.
After this change, OpenCode compacts before the request reaches 8 MiB and rolls large histories through bounded summary requests while preserving a final recent tail.
How
packages/core/src/session/runner/llm.tsbuilds the semantic request before the compaction decision and measures its system, messages, and tools payload.packages/core/src/session/compaction.tstreats 8 MiB as independent compaction pressure and summarizes old history in 32k-token chunks, carrying each structured summary into the next chunk.Scope
This does not raise the inference service's 10 MiB request-body limit. It keeps OpenCode requests safely below that independent transport boundary.
Testing
bun run test test/session-compaction.test.tsinpackages/core: 6 passedbun typecheckinpackages/coresession.compaction.ended, and then completed the requested response in the same threadFlow
flowchart TD A[Build semantic LLM request] --> B{Token pressure or body >= 8 MiB?} B -- no --> C[Send normal model request] B -- yes --> D[Keep recent tail] D --> E[Split older history into bounded chunks] E --> F[Summarize chunk] F --> G{More chunks?} G -- yes --> H[Carry summary into next chunk] H --> F G -- no --> I[Persist final summary + recent tail] I --> A