The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 21, 2026 2:30am

Greptile Summary

This PR adds support for {"location": "tool_config"} in cache_control_injection_points, enabling Bedrock Converse prompt caching of tool definitions by appending a {"cachePoint": {"type": "default"}} block to toolConfig.tools. It also changes the AnthropicCacheControlHook to forward non-message injection points to downstream provider-specific handling, rather than silently dropping them.

Key changes:

• litellm/types/integrations/anthropic_cache_control_hook.py: Adds CacheControlToolConfigInjectionPoint TypedDict and widens CacheControlInjectionPoint from a plain alias to a Union type.
• litellm/integrations/anthropic_cache_control_hook.py: Separates injection-point processing — message locations are consumed by the hook, while all others are written back into non_default_params for provider-specific handling.
• litellm/llms/bedrock/chat/converse_transformation.py: Inside _transform_request_helper (shared by both sync and async paths), pops cache_control_injection_points from additional_request_params before it can leak into additionalModelRequestFields, then appends the cachePoint block when a tool_config location is found and tools are present.
• Three new unit tests validate the expected behaviour (cachePoint appended, no-op when no tools, no-op without a tool_config point).

Architectural note: Because the hook now forwards any non-message injection point unconditionally, providers other than Bedrock will receive cache_control_injection_points in additional_request_params; those providers have no handler for it today (flagged in a prior review thread).

Confidence Score: 4/5

• Safe to merge for Bedrock users; minor risk that non-Bedrock providers receive unhandled injection points in their request bodies.
• The Bedrock-side implementation is correct and well-tested: cache_control_injection_points is properly popped before it can leak into additionalModelRequestFields, both sync and async paths share _transform_request_helper so neither is missed, and all three new tests are mock-only unit tests consistent with the repo rule. The hook-side change is intentional and necessary to route the new location type to Bedrock. The one point of risk — that the hook now forwards unknown locations to every provider — was identified in a prior review thread and the author is aware of it. No regressions to existing message-level caching behaviour are introduced.
• litellm/integrations/anthropic_cache_control_hook.py — the passthrough logic at line 76 will forward any unrecognised location values to all non-Bedrock providers without filtering.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["User calls litellm.completion() with\ncache_control_injection_points"] --> B["AnthropicCacheControlHook\nget_chat_completion_prompt"]
    B --> C{For each injection point}
    C -->|location == 'message'| D["_process_message_injection\nApply cache_control to messages"]
    C -->|any other location| E["Add to remaining_points"]
    D --> F{More points?}
    F -->|yes| C
    F -->|no| G{remaining_points\nnon-empty?}
    E --> F
    G -->|yes| H["Write remaining_points back\ninto non_default_params\n(cache_control_injection_points)"]
    G -->|no| I["non_default_params unchanged"]
    H --> J["AmazonConverseConfig\n_transform_request_helper"]
    I --> J
    J --> K["_prepare_request_params\ncache_control_injection_points →\nadditional_request_params"]
    K --> L["Pop cache_control_injection_points\nfrom additional_request_params"]
    L --> M{len bedrock_tools > 0\nAND cache_injection_points?}
    M -->|yes| N{Any point with\nlocation == 'tool_config'?}
    M -->|no| O["Discard injection points\n(no tools to cache)"]
    N -->|yes| P["Append cachePoint block\nto bedrock_tools list\nbreak"]
    N -->|no| Q["Silently discard\nunrecognised locations"]
    P --> R["Build ToolConfigBlock\nwith cachePoint at end\nof tools array"]
    R --> S["Bedrock Converse API request\nwith toolConfig.tools including cachePoint"]
    O --> T["Bedrock Converse API request\nwithout toolConfig"]

_{Last reviewed commit: "fix(types): add Cach..."}