fix(anthropic): accept dict-shape reasoning_effort from Responses bridge#28201
Conversation
Greptile SummaryFixes a silent bug where
Confidence Score: 5/5The change is a narrow, targeted coercion fix in a single elif branch of map_openai_params; it does not alter any existing call site behaviour for callers already passing a string, and it silently skips unresolvable inputs exactly as the old guard did. The production change is four lines of straightforward coercion logic. All downstream calls (_map_reasoning_effort, _is_adaptive_thinking_model, _raise_invalid_reasoning_effort) are unchanged, and the PR includes eight targeted parametrized unit tests that exercise adaptive, non-adaptive, and unparseable inputs with no network calls. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/chat/transformation.py | Removes the isinstance(value, str) guard on the reasoning_effort branch and coerces dict input to the effort string before calling _map_reasoning_effort; silently drops on no usable effort (matches prior behavior for unrecognized inputs). |
| tests/test_litellm/llws/anthropic/chat/test_anthropic_chat_transformation.py | Adds three new parametrized regression tests covering adaptive-model, non-adaptive-model, and unparseable-dict scenarios with pure in-process calls (no network I/O). |
Reviews (4): Last reviewed commit: "test(anthropic): convert unparseable-dic..." | Re-trigger Greptile
|
@greptile review again |
|
🤖 litellm-agent: This PR is currently BLOCKED from merge. Score: 0/5 ❌ Why blocked:
Details: Score docked for: all Phase B agent checks non-approving (karpathy + security + coverage gap). Fix the issues above and push an update — the bot will re-review automatically.
|
|
@greptile hmm review again |
1 similar comment
|
@greptile hmm review again |
|
🤖 litellm-agent: Auto-merge skipped — the staging branch Please rebase your branch onto |
dcead94 to
7d12fd0
Compare
|
@mateo-berri related to your work |
Issue BerriAI#28196 — the Responses->Chat parser (transformation.py:184-200) keeps the full dict as reasoning_effort when summary is set; that branch was added in BerriAI#25359. But the Anthropic transformation here still guarded on isinstance(value, str), silently dropping the param. Result: callers using the standard Reasoning(effort, summary) OpenAI-shaped object on Anthropic lose thinking entirely (0 reasoning_tokens, no thinking_blocks). Coerce dict -> string before mapping. Same shape tolerance that gpt_5_transformation._normalize_reasoning_effort_for_chat_completion already implements. summary is irrelevant for Anthropic's thinking_blocks. Adds two regression tests: one parametrized over string + dict shapes (with and without summary), one covering unparseable dict inputs (drops silently, no crash).
…ning_effort Per Greptile feedback on PR BerriAI#28198: the original regression test only exercised the adaptive (4.6+) path. Add a parametrized test for the non-adaptive branch (claude-sonnet-4-5) verifying that dict-shape reasoning_effort still maps to thinking.type='enabled' + budget_tokens, and that output_config is NOT set on pre-4.6 models.
…trize Per @greptile-apps inline review on PR BerriAI#28201 — matches the parametrize style of the two adjacent dict-shape tests and produces clearer failure messages (test ID per case instead of one collapsing for-loop).
7d12fd0 to
55f2d20
Compare
|
LGTM; thanks! |
d9af172
into
BerriAI:shin_agent_oss_staging_05_19_2026
* feat: add Xiaomi MiMo-V2.5-Pro and MiMo-V2.5 OpenRouter model entries (#27700) Squash-merged by litellm-agent from TorvaldUtne's PR. * fix(ui): trim whitespace from MCP inspector tool call inputs (#28203) Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * gemini-3.1-flash-lite pricing (#27933) * feat(model_prices): add gemini-3.1-flash-lite pricing with standard/batch/flex/priority tiers * fix pricing * add service tier --------- Co-authored-by: shin-berri <[email protected]> * fix: incorrect /v1/agents request example (#28131) * fix(anthropic): accept dict-shape reasoning_effort from Responses bridge (#28201) * fix(anthropic): accept dict-shape reasoning_effort from Responses bridge Issue #28196 — the Responses->Chat parser (transformation.py:184-200) keeps the full dict as reasoning_effort when summary is set; that branch was added in #25359. But the Anthropic transformation here still guarded on isinstance(value, str), silently dropping the param. Result: callers using the standard Reasoning(effort, summary) OpenAI-shaped object on Anthropic lose thinking entirely (0 reasoning_tokens, no thinking_blocks). Coerce dict -> string before mapping. Same shape tolerance that gpt_5_transformation._normalize_reasoning_effort_for_chat_completion already implements. summary is irrelevant for Anthropic's thinking_blocks. Adds two regression tests: one parametrized over string + dict shapes (with and without summary), one covering unparseable dict inputs (drops silently, no crash). * test(anthropic): add non-adaptive model coverage for dict-shape reasoning_effort Per Greptile feedback on PR #28198: the original regression test only exercised the adaptive (4.6+) path. Add a parametrized test for the non-adaptive branch (claude-sonnet-4-5) verifying that dict-shape reasoning_effort still maps to thinking.type='enabled' + budget_tokens, and that output_config is NOT set on pre-4.6 models. * test(anthropic): convert unparseable-dict test to @pytest.mark.parametrize Per @greptile-apps inline review on PR #28201 — matches the parametrize style of the two adjacent dict-shape tests and produces clearer failure messages (test ID per case instead of one collapsing for-loop). * feat: add pricing entry for openrouter/google/gemini-3.1-flash-lite (#28280) Squash-merged by litellm-agent from ro31337's PR. * fix(router): wrap aresponses streaming iterator for mid-stream fallbacks (#28215) Squash-merged by litellm-agent from cwang-otto's PR. * fix(router): unblock staging — mypy + coverage for aresponses streaming fallback (#28318) Squash-merged by litellm-agent from cwang-otto's PR. * fix(responses): forward timeout on completion transformation path (Anthropic, Bedrock, Vertex) (#28133) Squash-merged by litellm-agent from cwang-otto's PR. * feat(ui): add pause/resume Switch to the models table (#28151) Squash-merged by litellm-agent from Cyberfilo's PR. * fix(responses): merge sync completion kwargs to avoid duplicate keys Double-splatting litellm_completion_request and kwargs raised TypeError when metadata or service_tier were set. Match the async merge pattern. Co-authored-by: Cursor <[email protected]> * Use proxy base URL for CLI SSO form action (#28271) Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * fix(tests): add mistral/ministral-8b-2512 to cost map and backfill in conftest Mistral rotated the 'mistral/mistral-tiny' alias to return 'ministral-8b-2512' as the response model, which was missing from the cost map. This caused test_completion_mistral_api and test_completion_mistral_api_modified_input to fail in litellm.completion_cost lookup. - Add mistral/ministral-8b-2512 entry to both the in-tree model_prices_and_context_window.json and the bundled litellm/model_prices_and_context_window_backup.json (mirrors the existing openrouter/mistralai/ministral-8b-2512 pricing). - litellm.model_cost is loaded at import time from the URL pinned to main, so the new backup entry isn't visible at test runtime until it also lands on main. Backfill any entries missing from the remote-fetched map into litellm.model_cost in the local_testing conftest so cost-calculator lookups succeed on this branch. * fix(tests): drop unnecessary del of conftest backfill loop vars * fix(router): harden streaming fallback wrapper for bridge iterators - FallbackResponsesStreamWrapper now uses getattr fallbacks when copying attributes from the source iterator. The bridge path (LiteLLMCompletionStreamingIterator used by Anthropic/Bedrock/Vertex) does not call super().__init__ and is missing response, logging_obj (it uses litellm_logging_obj), responses_api_provider_config, start_time, request_data, call_type, and _hidden_params. Previously, wrapper construction raised AttributeError for any streaming fallback on the bridge path. - _aresponses_with_streaming_fallbacks now deep-copies the litellm_metadata (and metadata) dicts into fallback_kwargs. The primary attempt mutates this dict in place via _update_kwargs_with_deployment, so a shallow copy of kwargs was leaking primary-deployment fields (deployment, model_info, api_base) into the mid-stream fallback request. Co-authored-by: Yassin Kortam <[email protected]> * fix(router): use safe_deep_copy for fallback metadata snapshot The ban_copy_deepcopy_kwargs CI check rejects copy.deepcopy() on any variable whose name contains 'kwargs' (incl. fallback_kwargs). Swap the two copy.deepcopy(fallback_kwargs[...]) calls for safe_deep_copy, which handles non-picklable values (OTEL spans, etc.) by per-key deepcopy with fallback to the original reference. Co-authored-by: Yassin Kortam <[email protected]> * test(ci): skip chronically flaky build_and_test integration tests Both tests have been failing on every recent run of build_and_test against this PR's HEAD (1686967, 1688402, 1689993, 1690877), and the same two tests also fail intermittently on unrelated commits and other branches, independent of any code change in this PR (which only touches router fallback wrappers, the Anthropic Responses bridge, and unrelated UI/cost-map files). - tests.test_spend_logs.test_spend_logs: /spend/logs?request_id=... returns 500 even after a 20s wait for the spend log to be written. Spend-log accuracy is still covered by tests/test_litellm/proxy/ spend_tracking/ and the proxy_spend_accuracy_tests CircleCI job. - tests.test_team_members.test_add_multiple_members: /team/info?team_id= ... intermittently returns 404/400 mid-loop after add_team_member calls in the same fixture-created team. Single-member coverage in test_add_single_member already exercises the same endpoints, and team-member CRUD has dedicated unit coverage under tests/test_litellm/proxy/management_endpoints/. Skipping unblocks the build_and_test job until the underlying race in the dockerized integration setup is root-caused. * fix: preserve explicit timeout=0 in responses API handler Use 'timeout if timeout is not None else request_timeout' instead of 'timeout or request_timeout' so an explicit timeout=0/0.0 isn't silently replaced by the default request_timeout. Co-authored-by: Yassin Kortam <[email protected]> * fix(ui): guard model_info access in pause Switch with optional chaining * fix(ui): guard model_info access in pause Switch onChange handler Mirror the optional-chaining guard already applied to the isPausing check so a config-model row with a missing model_info cannot throw when the toggle's onChange fires. --------- Co-authored-by: TorvaldUtne <[email protected]> Co-authored-by: oss-agent-shin <[email protected]> Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: mubashir1osmani <[email protected]> Co-authored-by: Isha <[email protected]> Co-authored-by: cwang-otto <[email protected]> Co-authored-by: Roman Pushkin <[email protected]> Co-authored-by: Filippo Menghi <[email protected]> Co-authored-by: Cursor <[email protected]> Co-authored-by: boarder7395 <[email protected]> Co-authored-by: mateo-berri <[email protected]> Co-authored-by: Claude <[email protected]> Co-authored-by: Yassin Kortam <[email protected]>
* feat: add Xiaomi MiMo-V2.5-Pro and MiMo-V2.5 OpenRouter model entries (BerriAI#27700) Squash-merged by litellm-agent from TorvaldUtne's PR. * fix(ui): trim whitespace from MCP inspector tool call inputs (BerriAI#28203) Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * gemini-3.1-flash-lite pricing (BerriAI#27933) * feat(model_prices): add gemini-3.1-flash-lite pricing with standard/batch/flex/priority tiers * fix pricing * add service tier --------- Co-authored-by: shin-berri <[email protected]> * fix: incorrect /v1/agents request example (BerriAI#28131) * fix(anthropic): accept dict-shape reasoning_effort from Responses bridge (BerriAI#28201) * fix(anthropic): accept dict-shape reasoning_effort from Responses bridge Issue BerriAI#28196 — the Responses->Chat parser (transformation.py:184-200) keeps the full dict as reasoning_effort when summary is set; that branch was added in BerriAI#25359. But the Anthropic transformation here still guarded on isinstance(value, str), silently dropping the param. Result: callers using the standard Reasoning(effort, summary) OpenAI-shaped object on Anthropic lose thinking entirely (0 reasoning_tokens, no thinking_blocks). Coerce dict -> string before mapping. Same shape tolerance that gpt_5_transformation._normalize_reasoning_effort_for_chat_completion already implements. summary is irrelevant for Anthropic's thinking_blocks. Adds two regression tests: one parametrized over string + dict shapes (with and without summary), one covering unparseable dict inputs (drops silently, no crash). * test(anthropic): add non-adaptive model coverage for dict-shape reasoning_effort Per Greptile feedback on PR BerriAI#28198: the original regression test only exercised the adaptive (4.6+) path. Add a parametrized test for the non-adaptive branch (claude-sonnet-4-5) verifying that dict-shape reasoning_effort still maps to thinking.type='enabled' + budget_tokens, and that output_config is NOT set on pre-4.6 models. * test(anthropic): convert unparseable-dict test to @pytest.mark.parametrize Per @greptile-apps inline review on PR BerriAI#28201 — matches the parametrize style of the two adjacent dict-shape tests and produces clearer failure messages (test ID per case instead of one collapsing for-loop). * feat: add pricing entry for openrouter/google/gemini-3.1-flash-lite (BerriAI#28280) Squash-merged by litellm-agent from ro31337's PR. * fix(router): wrap aresponses streaming iterator for mid-stream fallbacks (BerriAI#28215) Squash-merged by litellm-agent from cwang-otto's PR. * fix(router): unblock staging — mypy + coverage for aresponses streaming fallback (BerriAI#28318) Squash-merged by litellm-agent from cwang-otto's PR. * fix(responses): forward timeout on completion transformation path (Anthropic, Bedrock, Vertex) (BerriAI#28133) Squash-merged by litellm-agent from cwang-otto's PR. * feat(ui): add pause/resume Switch to the models table (BerriAI#28151) Squash-merged by litellm-agent from Cyberfilo's PR. * fix(responses): merge sync completion kwargs to avoid duplicate keys Double-splatting litellm_completion_request and kwargs raised TypeError when metadata or service_tier were set. Match the async merge pattern. Co-authored-by: Cursor <[email protected]> * Use proxy base URL for CLI SSO form action (BerriAI#28271) Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * fix(tests): add mistral/ministral-8b-2512 to cost map and backfill in conftest Mistral rotated the 'mistral/mistral-tiny' alias to return 'ministral-8b-2512' as the response model, which was missing from the cost map. This caused test_completion_mistral_api and test_completion_mistral_api_modified_input to fail in litellm.completion_cost lookup. - Add mistral/ministral-8b-2512 entry to both the in-tree model_prices_and_context_window.json and the bundled litellm/model_prices_and_context_window_backup.json (mirrors the existing openrouter/mistralai/ministral-8b-2512 pricing). - litellm.model_cost is loaded at import time from the URL pinned to main, so the new backup entry isn't visible at test runtime until it also lands on main. Backfill any entries missing from the remote-fetched map into litellm.model_cost in the local_testing conftest so cost-calculator lookups succeed on this branch. * fix(tests): drop unnecessary del of conftest backfill loop vars * fix(router): harden streaming fallback wrapper for bridge iterators - FallbackResponsesStreamWrapper now uses getattr fallbacks when copying attributes from the source iterator. The bridge path (LiteLLMCompletionStreamingIterator used by Anthropic/Bedrock/Vertex) does not call super().__init__ and is missing response, logging_obj (it uses litellm_logging_obj), responses_api_provider_config, start_time, request_data, call_type, and _hidden_params. Previously, wrapper construction raised AttributeError for any streaming fallback on the bridge path. - _aresponses_with_streaming_fallbacks now deep-copies the litellm_metadata (and metadata) dicts into fallback_kwargs. The primary attempt mutates this dict in place via _update_kwargs_with_deployment, so a shallow copy of kwargs was leaking primary-deployment fields (deployment, model_info, api_base) into the mid-stream fallback request. Co-authored-by: Yassin Kortam <[email protected]> * fix(router): use safe_deep_copy for fallback metadata snapshot The ban_copy_deepcopy_kwargs CI check rejects copy.deepcopy() on any variable whose name contains 'kwargs' (incl. fallback_kwargs). Swap the two copy.deepcopy(fallback_kwargs[...]) calls for safe_deep_copy, which handles non-picklable values (OTEL spans, etc.) by per-key deepcopy with fallback to the original reference. Co-authored-by: Yassin Kortam <[email protected]> * test(ci): skip chronically flaky build_and_test integration tests Both tests have been failing on every recent run of build_and_test against this PR's HEAD (1686967, 1688402, 1689993, 1690877), and the same two tests also fail intermittently on unrelated commits and other branches, independent of any code change in this PR (which only touches router fallback wrappers, the Anthropic Responses bridge, and unrelated UI/cost-map files). - tests.test_spend_logs.test_spend_logs: /spend/logs?request_id=... returns 500 even after a 20s wait for the spend log to be written. Spend-log accuracy is still covered by tests/test_litellm/proxy/ spend_tracking/ and the proxy_spend_accuracy_tests CircleCI job. - tests.test_team_members.test_add_multiple_members: /team/info?team_id= ... intermittently returns 404/400 mid-loop after add_team_member calls in the same fixture-created team. Single-member coverage in test_add_single_member already exercises the same endpoints, and team-member CRUD has dedicated unit coverage under tests/test_litellm/proxy/management_endpoints/. Skipping unblocks the build_and_test job until the underlying race in the dockerized integration setup is root-caused. * fix: preserve explicit timeout=0 in responses API handler Use 'timeout if timeout is not None else request_timeout' instead of 'timeout or request_timeout' so an explicit timeout=0/0.0 isn't silently replaced by the default request_timeout. Co-authored-by: Yassin Kortam <[email protected]> * fix(ui): guard model_info access in pause Switch with optional chaining * fix(ui): guard model_info access in pause Switch onChange handler Mirror the optional-chaining guard already applied to the isPausing check so a config-model row with a missing model_info cannot throw when the toggle's onChange fires. --------- Co-authored-by: TorvaldUtne <[email protected]> Co-authored-by: oss-agent-shin <[email protected]> Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: mubashir1osmani <[email protected]> Co-authored-by: Isha <[email protected]> Co-authored-by: cwang-otto <[email protected]> Co-authored-by: Roman Pushkin <[email protected]> Co-authored-by: Filippo Menghi <[email protected]> Co-authored-by: Cursor <[email protected]> Co-authored-by: boarder7395 <[email protected]> Co-authored-by: mateo-berri <[email protected]> Co-authored-by: Claude <[email protected]> Co-authored-by: Yassin Kortam <[email protected]>
Summary
Fixes #28196 — Anthropic silently drops
reasoning_effortwhen it arrives as a dict (the shape the Responses→Chat parser produces wheneverreasoning.summaryis set on the request).Root cause recap
PR #25359 added a new branch to
transform_responses_api_request_to_chat_completion_request:But the Anthropic transformation downstream still guards on
isinstance(value, str):So when a caller sends the standard
Reasoning(effort="low", summary="concise")(e.g. viaaresponsesagainst an Anthropic-routed model), the dict fails the guard,thinkingis never set, and Anthropic produces 0 reasoning tokens. OpenAI'sgpt_5_transformation._normalize_reasoning_effort_for_chat_completionand Vertex Gemini's reasoning_effort handler both already accept both shapes; this PR brings Anthropic to the same tolerance.Change
In
AnthropicConfig.map_openai_params, coerce dict input to the effort string before calling_map_reasoning_effort. Drop silently if the dict has no usable effort (matches the prior behavior for unrecognized inputs).Tests
Three new regression tests in
tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py:test_reasoning_effort_accepts_dict_shape_for_adaptive_model— parametrized over"low",{"effort": "low"},{"effort": "low", "summary": "concise"},{"effort": "low", "summary": "detailed"}. Verifiesthinking.type='adaptive'+output_config.effortget set onclaude-sonnet-4-6.test_reasoning_effort_accepts_dict_shape_for_non_adaptive_model— parametrized over string + dict shapes againstclaude-sonnet-4-5. Verifiesthinking.type='enabled'+budget_tokensget set, andoutput_configis NOT set on pre-4.6 models. (Added per @greptile-apps feedback on fix(anthropic): accept dict-shape reasoning_effort from Responses bridge #28198 — covers the non-adaptive branch.)test_reasoning_effort_unparseable_dict_is_dropped— covers{"summary": "concise"}(missing effort),{"effort": None},{"effort": 123}(non-string). All silently dropped, no crash.All 8 new tests pass, full 26
reasoning_efforttest suite passes.Empirical verification (live API)
reasoning_tokensreasoning_effort="low"(string)claude-sonnet-4-6Reasoning(effort="low", summary="concise")(dict, pre-fix)claude-sonnet-4-6Reasoning(effort="low", summary="concise")(dict, post-fix)claude-sonnet-4-6Reasoning(effort="low", summary="concise")gpt-5.2reasoning_effort="low"(string)gemini/gemini-3-flash-previewReasoning(effort="low")(dict, no summary)gemini/gemini-3-flash-previewReasoning(effort="low", summary="concise")(dict, with summary)gemini/gemini-3-flash-previewGemini behaves correctly across all three shapes — its handler at
vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py:1150-1177already does the dict→string coercion. Bug was Anthropic-specific.Related
main, blocked by Guard workflow)