Skip to content

Anthropic prompt caching behavior #6941

@GordianDziwis

Description

@GordianDziwis

Describe the bug

The Anthropic prompt caching implementation had two issues that affected cache efficiency:

  1. Cache prefix continuity broken: MOIM (Message of Instruction Mode) messages were being inserted after the last assistant message, which broke the cache prefix continuity required by Anthropic's caching system.

  2. Cache control logic complexity: The cache_control logic used nested loops and didn't properly exclude MOIM messages (<info-msg>) from caching, potentially caching invalid content types.


To Reproduce

Steps to reproduce the behavior:

  1. Use goose with Anthropic provider (Claude models)
  2. Have a conversation with multiple turns
  3. Observe MOIM insertion point in conversation
  4. Check that cache_control is applied to MOIM messages when it shouldn't be
  5. Notice reduced cache hit rates due to broken cache prefix

Expected behavior

  • MOIM messages should be inserted at the end of the conversation to preserve Anthropic's cache prefix continuity
  • Only valid content block types (text, image, tool_use, tool_result) should have cache_control applied
  • MOIM messages (<info-msg>) should be excluded from caching

Please provide the following information

  • OS & Arch: [All platforms]
  • Interface: [UI / CLI]
  • Version: [Affected versions prior to commit 64c4bc1]
  • Extensions enabled: [All]
  • Provider & Model: [Anthropic – Claude models with prompt caching enabled]

Additional context

This fix improves:

  • Cache hit rates for Anthropic conversations
  • Cost efficiency when using prompt caching
  • Code maintainability with simplified cache control logic

The issue affected all users of goose with Anthropic provider when prompt caching was enabled.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions