fix(anthropic): keep cache breakpoints within Anthropic's limit of 4 with deferred tools#3692
Merged
Merged
Conversation
Deferred tools add a cache_control breakpoint on the tool list, which could push the total past Anthropic's limit of 4. Reduce message breakpoints from 2 to 1 when deferred tools are present. Assisted-By: Claude
Regression test covering the worst-case scenario (2 system breakpoints, deferred tools, long conversation) for both standard and beta converters. Assisted-By: Claude
docker-agent
left a comment
docker-agent
left a comment
Contributor
There was a problem hiding this comment.
Assessment: 🟢 APPROVE
aheritier
added
area/providers/anthropic
rumpl
approved these changes
Jul 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commit 83956fc added a
cache_controlbreakpoint on the tool list when deferred tools are present. Combined with the existing breakpoints applied to system blocks (up to 2) and the last conversation messages (up to 2), requests could exceed Anthropic's hard limit of 4cache_controlblocks and return API errors at runtime.applyMessageCacheControlandapplyBetaMessageCacheControlnow accept a breakpoint budget. When deferred tools are in the request, only the last message receives a breakpoint instead of the last two, keeping the total at or below 4. The non-deferred path is unchanged: 2 system + 2 messages = 4. With deferred tools the worst case is 2 system + 1 tools + 1 message = 4. Cache efficiency is preserved because Anthropic's caching layer automatically looks back roughly 20 blocks for prior hits, so a single message breakpoint still gets cache hits on subsequent turns.New tests assert that a worst-case request — two system breakpoints, deferred tools, and a multi-turn conversation — stays within the 4-block limit for both the standard and beta converters, and that the message breakpoint count drops from two to one when deferred tools are present.