feat(gemini_tts): adding support to Gemini TTS languageCode parameters by johngarrido · Pull Request #29623 · BerriAI/litellm

johngarrido · 2026-06-03T21:30:54Z

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

50-55 passing tests: main is stable with minor issues.

45-49 passing tests: acceptable but needs attention

<= 40 passing tests: unstable; be careful with your merges and assess the risk.

Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:

Screenshots / Proof of Fix

Type

🆕 New Feature

Changes

Add languageCode support to Gemini TTS speechConfig, allowing users to specify a BCP-47 language code (e.g. en-US, pt-BR) via the audio parameter. This flows through _map_audio_params into generationConfig.speechConfig.languageCode in the Gemini API request.

CLAassistant · 2026-06-03T21:31:00Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

johngarrido · 2026-06-03T21:32:17Z

@greptileai

greptile-apps · 2026-06-03T21:32:42Z

Greptile Summary

This PR adds BCP-47 language_code support to Gemini TTS by threading the snake_case user-facing key through _map_audio_params into the Gemini API's languageCode field in speechConfig. The naming convention is consistent with existing keys (voice, format) and the docstring was updated to reflect the new parameter.

_map_audio_params now reads language_code from the input dict and writes languageCode to the SpeechConfig output, only when the key is present.
SpeechConfig TypedDict in vertex_ai.py gains an optional languageCode: str field.
Three focused unit tests and one parametrized end-to-end test (covering vertex_ai and gemini providers) are added, all using local transformation logic with no real network calls.

Confidence Score: 5/5

Safe to merge — the change is additive, conditionally applied only when language_code is present, and well-covered by new unit and end-to-end tests.

The implementation is minimal and isolated to the TTS mapping layer. The new field is only written when explicitly provided, so existing calls are unaffected. Tests exercise both the presence and absence of the field across multiple providers.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py	Adds `language_code` (snake_case) support in `_map_audio_params`, mapping it to the Gemini API's `languageCode` field in `speechConfig`. Also updates the docstring to document the new input/output.
litellm/types/llms/vertex_ai.py	Adds optional `languageCode: str` field to the `SpeechConfig` TypedDict, reflecting the new Gemini API parameter.
tests/test_litellm/llms/gemini/test_gemini_tts.py	Adds three unit tests and one parametrized end-to-end test covering the new `language_code` parameter, all using mocked/local transformation logic with no real network calls.

_{Reviews (4): Last reviewed commit: "Restoring files modified under enterpris..." | Re-trigger Greptile}

codecov · 2026-06-03T21:34:02Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

greptile-apps · 2026-06-03T21:35:03Z

Greptile Summary

This PR adds languageCode (BCP-47) support to Gemini TTS's speechConfig, allowing callers to pass a language code via the audio parameter dict. The remaining file changes across the enterprise/ tree are purely cosmetic Black-formatter reformats with no functional impact.

Core change: _map_audio_params now reads value[\"languageCode\"] and writes it to speech_config[\"languageCode\"]; SpeechConfig TypedDict gains the matching field.
Tests: Three targeted unit tests and one parametrized end-to-end transformation test cover the happy-path, missing-field, and multi-provider scenarios without any network calls.

Confidence Score: 4/5

Safe to merge; the feature change is small and well-tested, and all enterprise file changes are whitespace-only.

The TTS implementation is correct and the tests exercise the new code path end-to-end. The only concerns are minor: the _map_audio_params docstring omits languageCode, and the new user-facing key name ("languageCode") is camelCase while the existing audio keys ("voice", "format") are lowercase, creating a small discoverability gap for callers who guess "language_code".

litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py — the _map_audio_params function and its user-facing key convention are worth a second look.

Important Files Changed

Filename	Overview
litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py	Core TTS feature: adds `languageCode` support in `_map_audio_params` by reading `value["languageCode"]` and writing it to `speech_config["languageCode"]`. Implementation is correct and straightforward, but the method's docstring is not updated to document the new field.
litellm/types/llms/vertex_ai.py	Adds `languageCode: str` to `SpeechConfig` TypedDict. Correct and minimal change.
tests/test_litellm/llms/gemini/test_gemini_tts.py	Adds three unit tests and one parametrized end-to-end transformation test for `languageCode` support. All tests are pure mock/transformation tests with no network calls, appropriately placed in the unit test folder.
enterprise/litellm_enterprise/enterprise_callbacks/callback_controls.py	Formatting-only changes: fixes indentation of `is_callback_disabled_dynamically` (was double-indented as if it were nested) and reformats long lines. No functional change.
enterprise/litellm_enterprise/proxy/common_utils/check_batch_cost.py	Formatting-only changes: Black reformats long lines throughout the file. No functional change.
enterprise/litellm_enterprise/proxy/common_utils/check_responses_cost.py	Formatting-only changes: removes trailing whitespace and reformats multi-line expressions. No functional change.

Comments Outside Diff (1)

litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py, line 1108-1124 (link)

The _map_audio_params docstring still shows only voice and format in the expected input and output — languageCode is not mentioned. Reviewers and future contributors relying on the docstring alone won't know the field is supported.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (2): Last reviewed commit: "Adding support to Gemini TTS Language Co..." | Re-trigger Greptile}

…Code param in Docstring

Sameerlite · 2026-06-04T12:28:14Z

@greptileai

Sameerlite

,

Sameerlite · 2026-06-04T12:29:29Z

 from .base_email import BaseEmailLogger

-
 SENDGRID_API_ENDPOINT = "https://api.sendgrid.com/v3/mail/send"


Why have you changed files not related to your code? Can you only keep the files changed for this feat. Also make sure greptile has score of 5/5 on the latest commit

@Sameerlite I believe this happened after running linting / formatting checks as suggested in contributing guide: https://github.com/BerriAI/litellm/blob/litellm_internal_staging/CONTRIBUTING.md

I'll remove these changes btw to keep it focused on the feature.

…int/formatting checks

johngarrido · 2026-06-04T21:58:30Z

@greptileai

krrish-berri-2 · 2026-06-04T22:35:43Z

@johngarrido — could you add a screenshot or short video showing that this change works as expected? It really helps reviewers verify the fix quickly. Thanks!

johngarrido · 2026-06-05T18:52:05Z

Hi @krrish-berri-2, quick walkthrough video with explanation here:

https://drive.google.com/file/d/1qyiOtVQ6Lc9owXHWYmE8_u1g2Da7lcm1/view?usp=sharing

Best,

* feat(bedrock_mantle): add SigV4/IAM auth to Responses API route (fixes #29665) (#29788) * feat(responses): add default no-op sign_request to BaseResponsesAPIConfig * feat(responses): call sign_request after body is final, send signed bytes when signed * feat(bedrock_mantle): add SigV4 sign_request via composed BaseAWSLLM (bearer path) * test(bedrock_mantle): cover SigV4 access-key, AssumeRole, body bytes, region/auth consistency * feat(bedrock_mantle): defer auth to sign_request; validate_environment no longer requires bearer * docs(bedrock_mantle): document SigV4 + Bearer auth on Responses route * test(responses): cover fake-stream signing order and mantle bearer arg/env precedence * fix(bedrock_mantle): wrap all botocore credential errors with both-paths guidance * fix(bedrock_mantle): catch specific credential errors, not all BotoCoreError, so STS transport failures are not masked * fix(bedrock_mantle): sign the compact Responses route too, not just create * fix(github-copilot): route per-model on /v1/responses based on model info (#29747) * feat(focus): add GCS destination for FOCUS export (#29751) * test: add failing tests for FocusGCSDestination * feat: add FocusGCSDestination reusing GCSBucketBase auth * feat: register FocusGCSDestination in factory; export from __init__ * fix(focus): preserve GCS_PATH_SERVICE_ACCOUNT when service_account_json not in config * style: apply Black formatting to gcs_destination and tests * style: apply Black formatting to factory.py * fix(bedrock): omit empty additionalModelRequestFields and system from Converse API payload (#29565) Amazon Nova Pro (and other strict Bedrock models) return 400 Malformed input request when additionalModelRequestFields: {} or system: [] are present in the payload. Both fields are optional in CommonRequestObject (total=False) and must be omitted rather than sent as empty structures. Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]> * fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible in pass-through cost tracking (#29730) * fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible Azure OpenAI resources created via the newer "Azure AI Foundry" / Cognitive Services pathway live on `*.cognitiveservices.azure.com` subdomains, not the older `openai.azure.com`. Both are valid Azure OpenAI surfaces in production today. The OpenAI pass-through cost-tracking handler hard-codes only the older hostname in five places (four `is_openai_*_route` methods on OpenAIPassthroughLoggingHandler, plus is_openai_route on PassThroughEndpointLogging). As a result, calls from newer Azure deployments are silently classified as "not an OpenAI route", the dispatch into the cost-tracking handler is skipped, and tokens/cost never get extracted into LiteLLM_SpendLogs — the row gets written with prompt_tokens=0, completion_tokens=0, spend=0, model='unknown'. Reproduced 2026-06-04 against a real Azure OpenAI deployment on `*.cognitiveservices.azure.com` proxied through LiteLLM v1.88.0. Fix: factor the hostname check into a single helper `_is_openai_compatible_host` listing all three recognized surfaces (api.openai.com, openai.azure.com, cognitiveservices.azure.com), and have all five call sites delegate to it. Purely additive — never weakens recognition for the originally-supported hostnames. Adds a test `test_is_openai_route_recognizes_cognitiveservices_azure_com` that exercises all four `is_openai_*_route` static methods against `*.cognitiveservices.azure.com` URLs (positive cases per route + a small cross-route negative to confirm route-specific path matching still works on the new hostname). Out of scope for this PR (separate followup): - `openai_passthrough_handler` calls chat/completions `transform_response` on Responses API payloads (`output:` not `choices:`), which throws inside the dispatch and drops the SpendLogs row entirely. Recognized + tracked separately. * ci: trigger fresh run Empty commit to re-run checks. The previous auth-and-jwt failure was a transient HuggingFace Hub 429 rate-limit hitting tokenizer downloads in tests/proxy_unit_tests/test_custom_tokenizer_bug.py — unrelated to this PR's scope (hostname recognition in pass-through cost tracking). No code change. --------- Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * fix(responses): preserve forced-function tool_choice name in Responses to Chat transform (#29812) The Responses API forces a specific function with a top-level name ({"type": "function", "name": "X"}), but _transform_tool_choice only handled the nested Chat Completions shape and fell through to returning "required" for the flat form, silently dropping the function name and degrading a forced function call to force-any-tool. Map the flat Responses shape to the nested Chat shape, keeping the "required" fallback when no name is present. * Preserve x-anthropic-billing-header system blocks for first-party Anthropic (#29584) * Preserve x-anthropic-billing-header system blocks for first-party Anthropic PR #20951 strips system blocks beginning with "x-anthropic-billing-header:" for every Anthropic target. That block is how the first-party Anthropic API recognizes Claude Code subscription (OAuth) traffic, so dropping it makes requests that carry only that block, such as the auto-mode tool-safety classifier, fail with a misleading 429 rate_limit_error; normal turns still work because they also carry the "You are Claude Code" identity block. Gate the strip behind should_strip_billing_metadata(), defaulting to False on the first-party AnthropicConfig and AnthropicMessagesConfig so the block is kept, and overridden to True on the providers that reach these transforms and reject the block (Bedrock platform, Vertex, Azure for the chat path; Minimax, Azure, DeepSeek for the messages path). Behavior for those providers is unchanged. * Strip billing header on Bedrock invoke and Vertex messages pass-through Two more subclasses reach the gated strip but inherited keep-by-default. AmazonAnthropicClaudeConfig (Bedrock invoke) calls AnthropicConfig.transform_request, which calls translate_system_message, and VertexAIPartnerModelsAnthropicMessagesConfig (Vertex messages pass-through) calls super().transform_anthropic_messages_request. Override should_strip_billing_metadata() to True on both. Add a parametrized test asserting the flag for every first-party base (False) and provider subclass (True), covering all overrides, plus a translate_system_message regression test for the Bedrock invoke path. * fix(cache): log hashed cache keys (#29890) * fix(ui): save routing groups as list (#29889) * Revert "fix(ui): save routing groups as list (#29889)" (#29928) This reverts commit 9b1f78f. * feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider (#29842) * feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider Registers parasail in the openai_like JSON provider loader with both /v1/chat/completions and /v1/responses support. Parasail's Responses API rejects store:true and any request that omits store, so the loader gains a force_store_false special_handling flag; the parasail entry sets it and the generated Responses config overrides store=false on every call. This keeps callers from hitting "State storage not supported" and matches what Parasail's docs require. Adds the PARASAIL enum value, listing under openai_compatible_providers, provider documentation at docs/my-website/docs/providers/parasail.md, and a focused unit test file under tests/test_litellm/llms/parasail/ that covers JSON registration, chat URL construction, Responses URL construction with PARASAIL_API_BASE override, and the force_store_false regression in both the caller-sent-store=true and caller-omitted cases. * fix(parasail): register in provider_endpoints_support, drop in-repo docs Greptile review feedback. The provider doc belongs in the litellm-docs repo, not this one's docs/my-website tree; removing it here. Adds the parasail entry to provider_endpoints_support.json so the check_provider_folders_documented.py CI check passes (chat_completions and responses true; others false). * fix: normalize Anthropic passthrough server tool usage (#29827) * test(anthropic): cover server_tool_use dict cost tracking * fix: normalize Anthropic server tool usage (cherry picked from commit 982f726) * fix: keep server tool usage subscriptable (cherry picked from commit 70280b9) --------- Co-authored-by: Genmin <[email protected]> * fix(proxy): fix typo generic_role_mappoings -> generic_role_mappings in ui_sso.py (#29753) Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * feat(proxy): add disable_budget_reservation general setting (#27639) (#29493) * feat(proxy): add disable_budget_reservation general setting (#27639) * feat(proxy): register disable_budget_reservation in ConfigGeneralSettings (#27639) * docs(proxy): document disable_budget_reservation concurrency tradeoff (#27639) * ci: re-trigger flaky docker build (prisma generate ECONNRESET) * fix(proxy): warn and document budget enforcement tradeoff when disable_budget_reservation is set (#27639) * feat(gemini_tts): adding support to Gemini TTS languageCode parameters (#29623) * Adding support to Gemini TTS Language Code parameters * Mapping Gemini TTS languageCode param in Docstring * Use snake_case for language_code input keyMapping Gemini TTS languageCode param in Docstring * Restoring files modified under enterprise/litellm_enterprise due to lint/formatting checks --------- Co-authored-by: João Garrido <[email protected]> * feat(guardrails): capture user and model metadata in CrowdStrike AIDR (#29517) * fix(proxy): require OpenAI path segment for shared Azure Cognitive Services domains Address Greptile review: the `*.cognitiveservices.azure.com` / `*.openai.azure.com` domains are shared by every Azure Cognitive Service (Speech, Vision, Language, ...), so a hostname-only substring match misclassified non-OpenAI Azure traffic as OpenAI routes. - Replace the substring host test with suffix matching (rejects look-alike domains like cognitiveservices.azure.com.attacker.example). - Add `_is_openai_compatible_url` that requires an OpenAI-style path marker (`/openai/` or `/v1/`) on the shared Azure domains, and use it in PassThroughEndpointLogging.is_openai_route (previously hostname-only). - Add negative tests for Azure Speech/Vision paths and look-alike domains. Co-Authored-By: Claude Opus 4.8 <[email protected]> * fix: support Responses input in Redis semantic cache (#29581) * fix: support responses input in redis semantic cache * test: cover redis semantic prompt extraction * test: handle blank redis semantic text fallbacks * chore: remove async cache dead statement * test: cover redis semantic cache miss paths * fix: filter sensitive cache lookup kwargs * chore: rerun ci after huggingface rate limit * chore(ui): regenerate dashboard API types (npm run gen:api) Sync src/lib/http/schema.d.ts with the proxy OpenAPI spec: adds the disable_budget_reservation general-settings field and picks up the RateLimitError docstring reindent. Fixes the gen:api CI drift check. Co-Authored-By: Claude Opus 4.8 <[email protected]> * test(bedrock): assert empty additionalModelRequestFields is omitted The Converse transformer now drops an empty additionalModelRequestFields block instead of sending it as `{}`. Update test_bedrock_top_k_param so models without top_k support (llama3) assert the key is absent rather than equal to an empty dict. Co-Authored-By: Claude Opus 4.8 <[email protected]> --------- Co-authored-by: Kent <[email protected]> Co-authored-by: codgician <[email protected]> Co-authored-by: Praveen Ghuge <[email protected]> Co-authored-by: Roi <[email protected]> Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: Liam Scott <[email protected]> Co-authored-by: abhay23-AI <[email protected]> Co-authored-by: Ceder Dens <[email protected]> Co-authored-by: 冯基魁 <[email protected]> Co-authored-by: Kai Huang <[email protected]> Co-authored-by: rinto <[email protected]> Co-authored-by: Genmin <[email protected]> Co-authored-by: Arnav Bhilwariya <[email protected]> Co-authored-by: Armaan Sandhu <[email protected]> Co-authored-by: João Garrido <[email protected]> Co-authored-by: João Garrido <[email protected]> Co-authored-by: Kenan Yildirim <[email protected]> Co-authored-by: Dávid Balatoni <[email protected]>

* feat(bedrock_mantle): add SigV4/IAM auth to Responses API route (fixes BerriAI#29665) (BerriAI#29788) * feat(responses): add default no-op sign_request to BaseResponsesAPIConfig * feat(responses): call sign_request after body is final, send signed bytes when signed * feat(bedrock_mantle): add SigV4 sign_request via composed BaseAWSLLM (bearer path) * test(bedrock_mantle): cover SigV4 access-key, AssumeRole, body bytes, region/auth consistency * feat(bedrock_mantle): defer auth to sign_request; validate_environment no longer requires bearer * docs(bedrock_mantle): document SigV4 + Bearer auth on Responses route * test(responses): cover fake-stream signing order and mantle bearer arg/env precedence * fix(bedrock_mantle): wrap all botocore credential errors with both-paths guidance * fix(bedrock_mantle): catch specific credential errors, not all BotoCoreError, so STS transport failures are not masked * fix(bedrock_mantle): sign the compact Responses route too, not just create * fix(github-copilot): route per-model on /v1/responses based on model info (BerriAI#29747) * feat(focus): add GCS destination for FOCUS export (BerriAI#29751) * test: add failing tests for FocusGCSDestination * feat: add FocusGCSDestination reusing GCSBucketBase auth * feat: register FocusGCSDestination in factory; export from __init__ * fix(focus): preserve GCS_PATH_SERVICE_ACCOUNT when service_account_json not in config * style: apply Black formatting to gcs_destination and tests * style: apply Black formatting to factory.py * fix(bedrock): omit empty additionalModelRequestFields and system from Converse API payload (BerriAI#29565) Amazon Nova Pro (and other strict Bedrock models) return 400 Malformed input request when additionalModelRequestFields: {} or system: [] are present in the payload. Both fields are optional in CommonRequestObject (total=False) and must be omitted rather than sent as empty structures. Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]> * fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible in pass-through cost tracking (BerriAI#29730) * fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible Azure OpenAI resources created via the newer "Azure AI Foundry" / Cognitive Services pathway live on `*.cognitiveservices.azure.com` subdomains, not the older `openai.azure.com`. Both are valid Azure OpenAI surfaces in production today. The OpenAI pass-through cost-tracking handler hard-codes only the older hostname in five places (four `is_openai_*_route` methods on OpenAIPassthroughLoggingHandler, plus is_openai_route on PassThroughEndpointLogging). As a result, calls from newer Azure deployments are silently classified as "not an OpenAI route", the dispatch into the cost-tracking handler is skipped, and tokens/cost never get extracted into LiteLLM_SpendLogs — the row gets written with prompt_tokens=0, completion_tokens=0, spend=0, model='unknown'. Reproduced 2026-06-04 against a real Azure OpenAI deployment on `*.cognitiveservices.azure.com` proxied through LiteLLM v1.88.0. Fix: factor the hostname check into a single helper `_is_openai_compatible_host` listing all three recognized surfaces (api.openai.com, openai.azure.com, cognitiveservices.azure.com), and have all five call sites delegate to it. Purely additive — never weakens recognition for the originally-supported hostnames. Adds a test `test_is_openai_route_recognizes_cognitiveservices_azure_com` that exercises all four `is_openai_*_route` static methods against `*.cognitiveservices.azure.com` URLs (positive cases per route + a small cross-route negative to confirm route-specific path matching still works on the new hostname). Out of scope for this PR (separate followup): - `openai_passthrough_handler` calls chat/completions `transform_response` on Responses API payloads (`output:` not `choices:`), which throws inside the dispatch and drops the SpendLogs row entirely. Recognized + tracked separately. * ci: trigger fresh run Empty commit to re-run checks. The previous auth-and-jwt failure was a transient HuggingFace Hub 429 rate-limit hitting tokenizer downloads in tests/proxy_unit_tests/test_custom_tokenizer_bug.py — unrelated to this PR's scope (hostname recognition in pass-through cost tracking). No code change. --------- Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * fix(responses): preserve forced-function tool_choice name in Responses to Chat transform (BerriAI#29812) The Responses API forces a specific function with a top-level name ({"type": "function", "name": "X"}), but _transform_tool_choice only handled the nested Chat Completions shape and fell through to returning "required" for the flat form, silently dropping the function name and degrading a forced function call to force-any-tool. Map the flat Responses shape to the nested Chat shape, keeping the "required" fallback when no name is present. * Preserve x-anthropic-billing-header system blocks for first-party Anthropic (BerriAI#29584) * Preserve x-anthropic-billing-header system blocks for first-party Anthropic PR BerriAI#20951 strips system blocks beginning with "x-anthropic-billing-header:" for every Anthropic target. That block is how the first-party Anthropic API recognizes Claude Code subscription (OAuth) traffic, so dropping it makes requests that carry only that block, such as the auto-mode tool-safety classifier, fail with a misleading 429 rate_limit_error; normal turns still work because they also carry the "You are Claude Code" identity block. Gate the strip behind should_strip_billing_metadata(), defaulting to False on the first-party AnthropicConfig and AnthropicMessagesConfig so the block is kept, and overridden to True on the providers that reach these transforms and reject the block (Bedrock platform, Vertex, Azure for the chat path; Minimax, Azure, DeepSeek for the messages path). Behavior for those providers is unchanged. * Strip billing header on Bedrock invoke and Vertex messages pass-through Two more subclasses reach the gated strip but inherited keep-by-default. AmazonAnthropicClaudeConfig (Bedrock invoke) calls AnthropicConfig.transform_request, which calls translate_system_message, and VertexAIPartnerModelsAnthropicMessagesConfig (Vertex messages pass-through) calls super().transform_anthropic_messages_request. Override should_strip_billing_metadata() to True on both. Add a parametrized test asserting the flag for every first-party base (False) and provider subclass (True), covering all overrides, plus a translate_system_message regression test for the Bedrock invoke path. * fix(cache): log hashed cache keys (BerriAI#29890) * fix(ui): save routing groups as list (BerriAI#29889) * Revert "fix(ui): save routing groups as list (BerriAI#29889)" (BerriAI#29928) This reverts commit 9b1f78f. * feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider (BerriAI#29842) * feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider Registers parasail in the openai_like JSON provider loader with both /v1/chat/completions and /v1/responses support. Parasail's Responses API rejects store:true and any request that omits store, so the loader gains a force_store_false special_handling flag; the parasail entry sets it and the generated Responses config overrides store=false on every call. This keeps callers from hitting "State storage not supported" and matches what Parasail's docs require. Adds the PARASAIL enum value, listing under openai_compatible_providers, provider documentation at docs/my-website/docs/providers/parasail.md, and a focused unit test file under tests/test_litellm/llms/parasail/ that covers JSON registration, chat URL construction, Responses URL construction with PARASAIL_API_BASE override, and the force_store_false regression in both the caller-sent-store=true and caller-omitted cases. * fix(parasail): register in provider_endpoints_support, drop in-repo docs Greptile review feedback. The provider doc belongs in the litellm-docs repo, not this one's docs/my-website tree; removing it here. Adds the parasail entry to provider_endpoints_support.json so the check_provider_folders_documented.py CI check passes (chat_completions and responses true; others false). * fix: normalize Anthropic passthrough server tool usage (BerriAI#29827) * test(anthropic): cover server_tool_use dict cost tracking * fix: normalize Anthropic server tool usage (cherry picked from commit 982f726) * fix: keep server tool usage subscriptable (cherry picked from commit 70280b9) --------- Co-authored-by: Genmin <[email protected]> * fix(proxy): fix typo generic_role_mappoings -> generic_role_mappings in ui_sso.py (BerriAI#29753) Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * feat(proxy): add disable_budget_reservation general setting (BerriAI#27639) (BerriAI#29493) * feat(proxy): add disable_budget_reservation general setting (BerriAI#27639) * feat(proxy): register disable_budget_reservation in ConfigGeneralSettings (BerriAI#27639) * docs(proxy): document disable_budget_reservation concurrency tradeoff (BerriAI#27639) * ci: re-trigger flaky docker build (prisma generate ECONNRESET) * fix(proxy): warn and document budget enforcement tradeoff when disable_budget_reservation is set (BerriAI#27639) * feat(gemini_tts): adding support to Gemini TTS languageCode parameters (BerriAI#29623) * Adding support to Gemini TTS Language Code parameters * Mapping Gemini TTS languageCode param in Docstring * Use snake_case for language_code input keyMapping Gemini TTS languageCode param in Docstring * Restoring files modified under enterprise/litellm_enterprise due to lint/formatting checks --------- Co-authored-by: João Garrido <[email protected]> * feat(guardrails): capture user and model metadata in CrowdStrike AIDR (BerriAI#29517) * fix(proxy): require OpenAI path segment for shared Azure Cognitive Services domains Address Greptile review: the `*.cognitiveservices.azure.com` / `*.openai.azure.com` domains are shared by every Azure Cognitive Service (Speech, Vision, Language, ...), so a hostname-only substring match misclassified non-OpenAI Azure traffic as OpenAI routes. - Replace the substring host test with suffix matching (rejects look-alike domains like cognitiveservices.azure.com.attacker.example). - Add `_is_openai_compatible_url` that requires an OpenAI-style path marker (`/openai/` or `/v1/`) on the shared Azure domains, and use it in PassThroughEndpointLogging.is_openai_route (previously hostname-only). - Add negative tests for Azure Speech/Vision paths and look-alike domains. Co-Authored-By: Claude Opus 4.8 <[email protected]> * fix: support Responses input in Redis semantic cache (BerriAI#29581) * fix: support responses input in redis semantic cache * test: cover redis semantic prompt extraction * test: handle blank redis semantic text fallbacks * chore: remove async cache dead statement * test: cover redis semantic cache miss paths * fix: filter sensitive cache lookup kwargs * chore: rerun ci after huggingface rate limit * chore(ui): regenerate dashboard API types (npm run gen:api) Sync src/lib/http/schema.d.ts with the proxy OpenAPI spec: adds the disable_budget_reservation general-settings field and picks up the RateLimitError docstring reindent. Fixes the gen:api CI drift check. Co-Authored-By: Claude Opus 4.8 <[email protected]> * test(bedrock): assert empty additionalModelRequestFields is omitted The Converse transformer now drops an empty additionalModelRequestFields block instead of sending it as `{}`. Update test_bedrock_top_k_param so models without top_k support (llama3) assert the key is absent rather than equal to an empty dict. Co-Authored-By: Claude Opus 4.8 <[email protected]> --------- Co-authored-by: Kent <[email protected]> Co-authored-by: codgician <[email protected]> Co-authored-by: Praveen Ghuge <[email protected]> Co-authored-by: Roi <[email protected]> Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: Liam Scott <[email protected]> Co-authored-by: abhay23-AI <[email protected]> Co-authored-by: Ceder Dens <[email protected]> Co-authored-by: 冯基魁 <[email protected]> Co-authored-by: Kai Huang <[email protected]> Co-authored-by: rinto <[email protected]> Co-authored-by: Genmin <[email protected]> Co-authored-by: Arnav Bhilwariya <[email protected]> Co-authored-by: Armaan Sandhu <[email protected]> Co-authored-by: João Garrido <[email protected]> Co-authored-by: João Garrido <[email protected]> Co-authored-by: Kenan Yildirim <[email protected]> Co-authored-by: Dávid Balatoni <[email protected]>

* feat(bedrock_mantle): add SigV4/IAM auth to Responses API route (fixes BerriAI#29665) (BerriAI#29788) * feat(responses): add default no-op sign_request to BaseResponsesAPIConfig * feat(responses): call sign_request after body is final, send signed bytes when signed * feat(bedrock_mantle): add SigV4 sign_request via composed BaseAWSLLM (bearer path) * test(bedrock_mantle): cover SigV4 access-key, AssumeRole, body bytes, region/auth consistency * feat(bedrock_mantle): defer auth to sign_request; validate_environment no longer requires bearer * docs(bedrock_mantle): document SigV4 + Bearer auth on Responses route * test(responses): cover fake-stream signing order and mantle bearer arg/env precedence * fix(bedrock_mantle): wrap all botocore credential errors with both-paths guidance * fix(bedrock_mantle): catch specific credential errors, not all BotoCoreError, so STS transport failures are not masked * fix(bedrock_mantle): sign the compact Responses route too, not just create * fix(github-copilot): route per-model on /v1/responses based on model info (BerriAI#29747) * feat(focus): add GCS destination for FOCUS export (BerriAI#29751) * test: add failing tests for FocusGCSDestination * feat: add FocusGCSDestination reusing GCSBucketBase auth * feat: register FocusGCSDestination in factory; export from __init__ * fix(focus): preserve GCS_PATH_SERVICE_ACCOUNT when service_account_json not in config * style: apply Black formatting to gcs_destination and tests * style: apply Black formatting to factory.py * fix(bedrock): omit empty additionalModelRequestFields and system from Converse API payload (BerriAI#29565) Amazon Nova Pro (and other strict Bedrock models) return 400 Malformed input request when additionalModelRequestFields: {} or system: [] are present in the payload. Both fields are optional in CommonRequestObject (total=False) and must be omitted rather than sent as empty structures. Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible in pass-through cost tracking (BerriAI#29730) * fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible Azure OpenAI resources created via the newer "Azure AI Foundry" / Cognitive Services pathway live on `*.cognitiveservices.azure.com` subdomains, not the older `openai.azure.com`. Both are valid Azure OpenAI surfaces in production today. The OpenAI pass-through cost-tracking handler hard-codes only the older hostname in five places (four `is_openai_*_route` methods on OpenAIPassthroughLoggingHandler, plus is_openai_route on PassThroughEndpointLogging). As a result, calls from newer Azure deployments are silently classified as "not an OpenAI route", the dispatch into the cost-tracking handler is skipped, and tokens/cost never get extracted into LiteLLM_SpendLogs — the row gets written with prompt_tokens=0, completion_tokens=0, spend=0, model='unknown'. Reproduced 2026-06-04 against a real Azure OpenAI deployment on `*.cognitiveservices.azure.com` proxied through LiteLLM v1.88.0. Fix: factor the hostname check into a single helper `_is_openai_compatible_host` listing all three recognized surfaces (api.openai.com, openai.azure.com, cognitiveservices.azure.com), and have all five call sites delegate to it. Purely additive — never weakens recognition for the originally-supported hostnames. Adds a test `test_is_openai_route_recognizes_cognitiveservices_azure_com` that exercises all four `is_openai_*_route` static methods against `*.cognitiveservices.azure.com` URLs (positive cases per route + a small cross-route negative to confirm route-specific path matching still works on the new hostname). Out of scope for this PR (separate followup): - `openai_passthrough_handler` calls chat/completions `transform_response` on Responses API payloads (`output:` not `choices:`), which throws inside the dispatch and drops the SpendLogs row entirely. Recognized + tracked separately. * ci: trigger fresh run Empty commit to re-run checks. The previous auth-and-jwt failure was a transient HuggingFace Hub 429 rate-limit hitting tokenizer downloads in tests/proxy_unit_tests/test_custom_tokenizer_bug.py — unrelated to this PR's scope (hostname recognition in pass-through cost tracking). No code change. --------- Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * fix(responses): preserve forced-function tool_choice name in Responses to Chat transform (BerriAI#29812) The Responses API forces a specific function with a top-level name ({"type": "function", "name": "X"}), but _transform_tool_choice only handled the nested Chat Completions shape and fell through to returning "required" for the flat form, silently dropping the function name and degrading a forced function call to force-any-tool. Map the flat Responses shape to the nested Chat shape, keeping the "required" fallback when no name is present. * Preserve x-anthropic-billing-header system blocks for first-party Anthropic (BerriAI#29584) * Preserve x-anthropic-billing-header system blocks for first-party Anthropic PR BerriAI#20951 strips system blocks beginning with "x-anthropic-billing-header:" for every Anthropic target. That block is how the first-party Anthropic API recognizes Claude Code subscription (OAuth) traffic, so dropping it makes requests that carry only that block, such as the auto-mode tool-safety classifier, fail with a misleading 429 rate_limit_error; normal turns still work because they also carry the "You are Claude Code" identity block. Gate the strip behind should_strip_billing_metadata(), defaulting to False on the first-party AnthropicConfig and AnthropicMessagesConfig so the block is kept, and overridden to True on the providers that reach these transforms and reject the block (Bedrock platform, Vertex, Azure for the chat path; Minimax, Azure, DeepSeek for the messages path). Behavior for those providers is unchanged. * Strip billing header on Bedrock invoke and Vertex messages pass-through Two more subclasses reach the gated strip but inherited keep-by-default. AmazonAnthropicClaudeConfig (Bedrock invoke) calls AnthropicConfig.transform_request, which calls translate_system_message, and VertexAIPartnerModelsAnthropicMessagesConfig (Vertex messages pass-through) calls super().transform_anthropic_messages_request. Override should_strip_billing_metadata() to True on both. Add a parametrized test asserting the flag for every first-party base (False) and provider subclass (True), covering all overrides, plus a translate_system_message regression test for the Bedrock invoke path. * fix(cache): log hashed cache keys (BerriAI#29890) * fix(ui): save routing groups as list (BerriAI#29889) * Revert "fix(ui): save routing groups as list (BerriAI#29889)" (BerriAI#29928) This reverts commit 9b1f78f. * feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider (BerriAI#29842) * feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider Registers parasail in the openai_like JSON provider loader with both /v1/chat/completions and /v1/responses support. Parasail's Responses API rejects store:true and any request that omits store, so the loader gains a force_store_false special_handling flag; the parasail entry sets it and the generated Responses config overrides store=false on every call. This keeps callers from hitting "State storage not supported" and matches what Parasail's docs require. Adds the PARASAIL enum value, listing under openai_compatible_providers, provider documentation at docs/my-website/docs/providers/parasail.md, and a focused unit test file under tests/test_litellm/llms/parasail/ that covers JSON registration, chat URL construction, Responses URL construction with PARASAIL_API_BASE override, and the force_store_false regression in both the caller-sent-store=true and caller-omitted cases. * fix(parasail): register in provider_endpoints_support, drop in-repo docs Greptile review feedback. The provider doc belongs in the litellm-docs repo, not this one's docs/my-website tree; removing it here. Adds the parasail entry to provider_endpoints_support.json so the check_provider_folders_documented.py CI check passes (chat_completions and responses true; others false). * fix: normalize Anthropic passthrough server tool usage (BerriAI#29827) * test(anthropic): cover server_tool_use dict cost tracking * fix: normalize Anthropic server tool usage (cherry picked from commit 982f726) * fix: keep server tool usage subscriptable (cherry picked from commit 70280b9) --------- Co-authored-by: Genmin <[email protected]> * fix(proxy): fix typo generic_role_mappoings -> generic_role_mappings in ui_sso.py (BerriAI#29753) Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * feat(proxy): add disable_budget_reservation general setting (BerriAI#27639) (BerriAI#29493) * feat(proxy): add disable_budget_reservation general setting (BerriAI#27639) * feat(proxy): register disable_budget_reservation in ConfigGeneralSettings (BerriAI#27639) * docs(proxy): document disable_budget_reservation concurrency tradeoff (BerriAI#27639) * ci: re-trigger flaky docker build (prisma generate ECONNRESET) * fix(proxy): warn and document budget enforcement tradeoff when disable_budget_reservation is set (BerriAI#27639) * feat(gemini_tts): adding support to Gemini TTS languageCode parameters (BerriAI#29623) * Adding support to Gemini TTS Language Code parameters * Mapping Gemini TTS languageCode param in Docstring * Use snake_case for language_code input keyMapping Gemini TTS languageCode param in Docstring * Restoring files modified under enterprise/litellm_enterprise due to lint/formatting checks --------- Co-authored-by: João Garrido <[email protected]> * feat(guardrails): capture user and model metadata in CrowdStrike AIDR (BerriAI#29517) * fix(proxy): require OpenAI path segment for shared Azure Cognitive Services domains Address Greptile review: the `*.cognitiveservices.azure.com` / `*.openai.azure.com` domains are shared by every Azure Cognitive Service (Speech, Vision, Language, ...), so a hostname-only substring match misclassified non-OpenAI Azure traffic as OpenAI routes. - Replace the substring host test with suffix matching (rejects look-alike domains like cognitiveservices.azure.com.attacker.example). - Add `_is_openai_compatible_url` that requires an OpenAI-style path marker (`/openai/` or `/v1/`) on the shared Azure domains, and use it in PassThroughEndpointLogging.is_openai_route (previously hostname-only). - Add negative tests for Azure Speech/Vision paths and look-alike domains. * fix: support Responses input in Redis semantic cache (BerriAI#29581) * fix: support responses input in redis semantic cache * test: cover redis semantic prompt extraction * test: handle blank redis semantic text fallbacks * chore: remove async cache dead statement * test: cover redis semantic cache miss paths * fix: filter sensitive cache lookup kwargs * chore: rerun ci after huggingface rate limit * chore(ui): regenerate dashboard API types (npm run gen:api) Sync src/lib/http/schema.d.ts with the proxy OpenAPI spec: adds the disable_budget_reservation general-settings field and picks up the RateLimitError docstring reindent. Fixes the gen:api CI drift check. * test(bedrock): assert empty additionalModelRequestFields is omitted The Converse transformer now drops an empty additionalModelRequestFields block instead of sending it as `{}`. Update test_bedrock_top_k_param so models without top_k support (llama3) assert the key is absent rather than equal to an empty dict. --------- Co-authored-by: Kent <[email protected]> Co-authored-by: codgician <[email protected]> Co-authored-by: Praveen Ghuge <[email protected]> Co-authored-by: Roi <[email protected]> Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: Liam Scott <[email protected]> Co-authored-by: abhay23-AI <[email protected]> Co-authored-by: Ceder Dens <[email protected]> Co-authored-by: 冯基魁 <[email protected]> Co-authored-by: Kai Huang <[email protected]> Co-authored-by: rinto <[email protected]> Co-authored-by: Genmin <[email protected]> Co-authored-by: Arnav Bhilwariya <[email protected]> Co-authored-by: Armaan Sandhu <[email protected]> Co-authored-by: João Garrido <[email protected]> Co-authored-by: João Garrido <[email protected]> Co-authored-by: Kenan Yildirim <[email protected]> Co-authored-by: Dávid Balatoni <[email protected]>

Adding support to Gemini TTS Language Code parameters

0bfb05d

greptile-apps Bot reviewed Jun 3, 2026

View reviewed changes

Comment thread litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py Outdated

jowgarrido added 2 commits June 3, 2026 21:39

Mapping Gemini TTS languageCode param in Docstring

dd0f7a3

Use snake_case for language_code input keyMapping Gemini TTS language…

56c2c42

…Code param in Docstring

johngarrido changed the title ~~Adding support to Gemini TTS Language Code parameters~~ feat(gemini_tts): adding support to Gemini TTS languageCode parameters Jun 4, 2026

Sameerlite reviewed Jun 4, 2026

View reviewed changes

Restoring files modified under enterprise/litellm_enterprise due to l…

b9df5e0

…int/formatting checks

Sameerlite changed the base branch from litellm_internal_staging to litellm_oss_staging_080626 June 8, 2026 12:43

Sameerlite merged commit e487706 into BerriAI:litellm_oss_staging_080626 Jun 8, 2026
69 of 70 checks passed

johngarrido deleted the gemini-tts-language-params branch June 22, 2026 17:11

		from .base_email import BaseEmailLogger


		SENDGRID_API_ENDPOINT = "https://api.sendgrid.com/v3/mail/send"

Uh oh!

Uh oh!

Conversation

johngarrido commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

CI (LiteLLM team)

Screenshots / Proof of Fix

Type

Changes

Uh oh!

CLAassistant commented Jun 3, 2026

Uh oh!

johngarrido commented Jun 3, 2026

Uh oh!

greptile-apps Bot commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

codecov Bot commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

greptile-apps Bot commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Comments Outside Diff (1)

Uh oh!

Uh oh!

Sameerlite commented Jun 4, 2026

Uh oh!

Sameerlite left a comment

Choose a reason for hiding this comment

Uh oh!

Sameerlite Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

johngarrido Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johngarrido commented Jun 4, 2026

Uh oh!

krrish-berri-2 commented Jun 4, 2026

Uh oh!

johngarrido commented Jun 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

johngarrido commented Jun 3, 2026 •

edited

Loading

greptile-apps Bot commented Jun 3, 2026 •

edited

Loading

codecov Bot commented Jun 3, 2026 •

edited

Loading

greptile-apps Bot commented Jun 3, 2026 •

edited

Loading

johngarrido Jun 4, 2026 •

edited

Loading