fix(anthropic): strip undocumented keys from metadata before sending to API#24661
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR adds a post-merge metadata sanitization filter in Confidence Score: 4/5Safe to merge for the stated fix; two pre-existing concerns flagged in earlier review threads (missing The filter logic is correct for the primary bug being fixed and is well-covered by the new mock tests. Score is 4 rather than 5 because the previously flagged P1 concern — that a
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/chat/transformation.py | Adds a post-merge metadata filter that strips all keys except user_id before sending to the Anthropic API; the filter does not call _valid_user_id() (already flagged in previous threads), meaning invalid values like emails can still be forwarded via the direct optional_params path. |
| tests/llm_translation/test_anthropic_completion.py | Six new mock-only unit tests covering the new filter, inheritance through Vertex/Azure configs, and edge cases; one test (test_metadata_user_id_from_litellm_params_strips_extras) tests pre-existing extraction logic rather than the newly added filter. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[transform_request called] --> B{litellm_params.metadata has user_id?}
B -- Yes and valid --> C[optional_params.metadata set to user_id only]
B -- No or invalid --> D[optional_params.metadata unchanged]
C --> E{metadata in optional_params?}
D --> E
E -- No --> F[skip filter]
E -- Yes --> G{user_id key present in metadata?}
G -- Yes --> H[NEW: strip all other keys, keep only user_id]
G -- No --> I[NEW: pop metadata entirely]
H --> J[pop is_vertex_request]
I --> J
F --> J
J --> K[return final data dict]
Reviews (2): Last reviewed commit: "fix(anthropic): strip undocumented keys ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| _user_id = optional_params["metadata"].get("user_id") | ||
| if _user_id is not None: | ||
| optional_params["metadata"] = {"user_id": _user_id} | ||
| else: | ||
| optional_params.pop("metadata") |
There was a problem hiding this comment.
_valid_user_id validation creates inconsistency
The two existing paths that populate metadata.user_id both call _valid_user_id() to reject emails and phone numbers (Anthropic rejects those values — see the comment at line 1032: "anthropic fails on emails"). This new filter skips that check entirely.
If a caller passes metadata={"user_id": "[email protected]"} directly in optional_params, the new code will forward {"user_id": "[email protected]"} to Anthropic and receive an API error, while the same value arriving via the user parameter or litellm_params.metadata would be silently dropped.
| _user_id = optional_params["metadata"].get("user_id") | |
| if _user_id is not None: | |
| optional_params["metadata"] = {"user_id": _user_id} | |
| else: | |
| optional_params.pop("metadata") | |
| _user_id = optional_params["metadata"].get("user_id") | |
| if _user_id is not None and _valid_user_id(str(_user_id)): | |
| optional_params["metadata"] = {"user_id": _user_id} | |
| else: | |
| optional_params.pop("metadata") |
| ## Ensure metadata only contains user_id (only documented field in Anthropic Messages API) | ||
| if "metadata" in optional_params and isinstance( | ||
| optional_params["metadata"], dict | ||
| ): | ||
| _user_id = optional_params["metadata"].get("user_id") | ||
| if _user_id is not None: | ||
| optional_params["metadata"] = {"user_id": _user_id} | ||
| else: | ||
| optional_params.pop("metadata") |
There was a problem hiding this comment.
Extra metadata keys are dropped silently. A user who passes metadata={"user_id": "x", "trace_id": "y"} will see trace_id disappear with no indication of why. Other spots in this file already use litellm.verbose_logger.warning() for similar "we modified your input" situations (e.g. line 1366). Adding a warning here would make it much easier to debug.
| ## Ensure metadata only contains user_id (only documented field in Anthropic Messages API) | |
| if "metadata" in optional_params and isinstance( | |
| optional_params["metadata"], dict | |
| ): | |
| _user_id = optional_params["metadata"].get("user_id") | |
| if _user_id is not None: | |
| optional_params["metadata"] = {"user_id": _user_id} | |
| else: | |
| optional_params.pop("metadata") | |
| ## Ensure metadata only contains user_id (only documented field in Anthropic Messages API) | |
| if "metadata" in optional_params and isinstance( | |
| optional_params["metadata"], dict | |
| ): | |
| _user_id = optional_params["metadata"].get("user_id") | |
| _extra_keys = [k for k in optional_params["metadata"] if k != "user_id"] | |
| if _extra_keys: | |
| litellm.verbose_logger.warning( | |
| "Anthropic metadata only supports 'user_id'. " | |
| "Dropping unsupported keys: %s", | |
| _extra_keys, | |
| ) | |
| if _user_id is not None: | |
| optional_params["metadata"] = {"user_id": _user_id} | |
| else: | |
| optional_params.pop("metadata") |
Sameerlite
force-pushed
the
litellm_filter-metadata-user-id
branch
from
9f24b8e to
38e8003
Compare
2 participants
Summary
The Anthropic Messages API only supports
user_idas a field inside themetadataobject (per official documentation). Passing any other key causes the API to return an error. The same constraint applies to Vertex AI Anthropic and Azure AI Anthropic, which share the same API contract.This PR adds a final filter in
AnthropicConfig.transform_request()that runs after all metadata sources are merged, ensuring only{"user_id": <value>}is forwarded — ormetadatais dropped entirely if nouser_idis present.Why this is needed:
user_idis present inmetadatauser_idis a valid fieldmetadataenteredoptional_paramsvia any path other than the litellm-params handler (e.g., direct Anthropic param pass-through), extra keys would be forwarded unfilteredProviders covered: Anthropic, Vertex AI Anthropic, Azure AI Anthropic — all three inherit from
AnthropicConfig, so the single fix covers all of them.