-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Anthropic prompt caching behavior #6941
Copy link
Copy link
Closed
Description
Describe the bug
The Anthropic prompt caching implementation had two issues that affected cache efficiency:
-
Cache prefix continuity broken: MOIM (Message of Instruction Mode) messages were being inserted after the last assistant message, which broke the cache prefix continuity required by Anthropic's caching system.
-
Cache control logic complexity: The cache_control logic used nested loops and didn't properly exclude MOIM messages (
<info-msg>) from caching, potentially caching invalid content types.
To Reproduce
Steps to reproduce the behavior:
- Use goose with Anthropic provider (Claude models)
- Have a conversation with multiple turns
- Observe MOIM insertion point in conversation
- Check that cache_control is applied to MOIM messages when it shouldn't be
- Notice reduced cache hit rates due to broken cache prefix
Expected behavior
- MOIM messages should be inserted at the end of the conversation to preserve Anthropic's cache prefix continuity
- Only valid content block types (text, image, tool_use, tool_result) should have cache_control applied
- MOIM messages (
<info-msg>) should be excluded from caching
Please provide the following information
- OS & Arch: [All platforms]
- Interface: [UI / CLI]
- Version: [Affected versions prior to commit 64c4bc1]
- Extensions enabled: [All]
- Provider & Model: [Anthropic – Claude models with prompt caching enabled]
Additional context
This fix improves:
- Cache hit rates for Anthropic conversations
- Cost efficiency when using prompt caching
- Code maintainability with simplified cache control logic
The issue affected all users of goose with Anthropic provider when prompt caching was enabled.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels