[Feature][Bug Fix] Decouple Azure OpenAI Deployment ID from model name via base_model to fix gpt5 model routing by withomasmicrosoft · Pull Request #28490 · BerriAI/litellm

withomasmicrosoft · 2026-05-21T16:58:17Z

Relevant issues

[Feature]: Azure OpenAI decouple deployment ID from model name via base_model #28482

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review.

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

50-55 passing tests: main is stable with minor issues.

45-49 passing tests: acceptable but needs attention

<= 40 passing tests: unstable; be careful with your merges and assess the risk.

Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:

Screenshots / Proof of Fix

"""
Sample: Azure base_model parameter — Before & After

Problem:
  When you name your Azure deployment something like "my-gpt-5.2", LiteLLM
  detects it as a gpt-5 model but can't parse the exact version from the custom
  name. logprobs (supported on Azure gpt-5.2+) gets rejected because LiteLLM
  doesn't know the deployment is running gpt-5.2+.

Solution:
  Pass `base_model` to tell LiteLLM what the underlying model actually is.
"""

import litellm

# ===========================================================================
# BEFORE: Custom deployment name — logprobs incorrectly rejected
# ===========================================================================

# ❌ "my-gpt-5.2" contains "gpt-5" so LiteLLM routes to GPT-5 config, but
#    it can't determine the minor version from the custom name. logprobs is
#    rejected because LiteLLM doesn't know this is gpt-5.2+.

params_before = litellm.get_supported_openai_params(
    model="my-gpt-5.2", custom_llm_provider="azure"
)

print("=" * 60)
print("BEFORE (no base_model):")
print(f"  logprobs supported: {'logprobs' in params_before}")
print(f"  Supported params: {sorted(params_before)}")
print()

# ===========================================================================
# AFTER: Using base_model — LiteLLM detects the correct model version
# ===========================================================================

# ✅ With base_model="azure/gpt-5.2", LiteLLM knows this is gpt-5.2.
#    logprobs is correctly recognized as supported (Azure gpt-5.2+ supports it),
#    and the right config class (AzureOpenAIGPT5Config) is used with full
#    version awareness.

params_after = litellm.get_supported_openai_params(
    model="my-gpt-5.2", custom_llm_provider="azure", base_model="azure/gpt-5.2"
)

print("AFTER (with base_model='azure/gpt-5.2'):")
print(f"  logprobs supported: {'logprobs' in params_after}")
print(f"  Supported params: {sorted(params_after)}")
print("=" * 60)

# --- Assertions ---
assert "logprobs" not in params_before, "BEFORE: logprobs should NOT be supported"
assert "logprobs" in params_after, "AFTER: logprobs should be supported"

print("\n✅ All assertions passed!")
print("   Without base_model:  logprobs is NOT in supported params")
print("   With base_model:     logprobs IS in supported params")

Type

🆕 New Feature

Changes

feat(azure): decouple deployment ID from model name via base_model

Azure OpenAI deployments have arbitrary names (deployment IDs) that may not match the underlying model. Previously, model-type detection (o-series, gpt-5, etc.) relied on substring matching against the deployment name, causing misrouted configs and rejected params when deployment names were non-standard (e.g. 'my-deployment-id' for gpt-5.2). This change extends the existing base_model field to drive model-type detection, config selection, supported param resolution, and param mapping throughout the Azure call path: - _get_azure_config() uses base_model for is_o_series/is_gpt_5 checks - get_provider_chat_config() threads base_model for Azure - get_supported_openai_params() accepts and uses base_model - get_optional_params() accepts base_model and passes it to all Azure config method calls (get_supported_openai_params, map_openai_params) - azure.py completion handler uses base_model for GPT-5 detection - Config internal methods (e.g. is_model_gpt_5_2_model) now receive base_model so features like logprobs are correctly enabled Fully backward compatible - when base_model is unset, behavior is identical. Existing o_series/ and gpt5_series/ prefix workarounds continue to work. Usage in proxy config: model_list: - model_name: my-gpt5 litellm_params: model: azure/my-deployment-id model_info: base_model: azure/gpt-5.2 Fixes: non-standard deployment names like 'prefix-gpt-5.2' rejecting logprobs/top_logprobs despite the underlying model supporting them.

…enai-base-model-name

codecov · 2026-05-21T17:02:16Z

Codecov Report

❌ Patch coverage is 82.35294% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
.../litellm_core_utils/get_supported_openai_params.py	25.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

greptile-apps · 2026-05-21T17:04:39Z

Greptile Summary

This PR introduces a base_model parameter that lets callers decouple an Azure deployment name from the underlying model family, so that custom names like "my-gpt-5.2" still route to the correct config class (AzureOpenAIGPT5Config, AzureOpenAIO1Config, or the default AzureOpenAIConfig).

base_model is threaded through completion() → get_optional_params() → get_supported_openai_params() → ProviderConfigManager.get_provider_chat_config() → _get_azure_config(), and is stored in litellm_params so AzureChatCompletion.completion() can read it for GPT-5 request-data construction.
Azure is pulled out of the shared _PROVIDER_CONFIG_MAP dictionary into its own early-exit branch in get_provider_chat_config so the extra base_model argument can be forwarded without touching the map's (factory, needs_model) tuple interface.
A new test module covers config routing, supported-params, and get_optional_params mapping for both the standard and base_model-override code paths, with backward-compatibility cases for existing detection logic.

Confidence Score: 5/5

Safe to merge — the change is additive and fully backward-compatible; callers that don't pass base_model see identical behavior to before.

The base_model parameter defaults to None everywhere it is added, so no existing call sites are affected. The new Azure special-case path in get_provider_chat_config replicates the same three-way routing that was already in _get_azure_config, just with an extra detection_model = base_model or model substitution. The parameter is correctly stripped from passed_params before _check_valid_arg runs and is stored in litellm_params before it is needed in azure.py. The test suite is thorough, exercises all routing branches, and makes no real network calls.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/utils.py	Adds `base_model` to `get_optional_params`, `_get_azure_config`, and `get_provider_chat_config`; moves Azure to a special-case path before the provider config map so the parameter can be threaded through for model-type detection. The `base_model` is correctly popped from `passed_params` before `_check_valid_arg` runs so it is never forwarded to the Azure API.
litellm/litellm_core_utils/get_supported_openai_params.py	Adds `base_model` parameter; passes it through `get_provider_chat_config` and uses `base_model or model` for the `get_supported_openai_params` call on the provider config. The pre-existing Azure-specific fallback at lines 142-157 is now dead code for chat completions (Azure is in `LlmProvidersSet` and always returns a non-None config), but this was acknowledged in a prior thread.
litellm/llms/azure/azure.py	Uses `litellm_params.get("base_model") or model` for the GPT-5 detection check before building request data. O-series routing is handled upstream in `main.py` so only the GPT-5 vs. default split needs to happen here.
litellm/main.py	Passes `base_model` to `get_provider_chat_config`, adds it to `optional_param_args` (so it flows into `get_optional_params`), and uses `_azure_detection_model = base_model or model` for the o-series routing check. `base_model` is also forwarded to `get_litellm_params` so it is available in `litellm_params` for downstream handlers in `azure.py`.
tests/test_litellm/llms/azure/chat/test_azure_base_model_routing.py	New test file with comprehensive unit tests covering config routing, supported-params detection, and optional-param mapping for custom deployment names with `base_model`. All tests are pure unit tests with no real network calls.

_{Reviews (2): Last reviewed commit: "Addressing Greptile comments." | Re-trigger Greptile}

Sameerlite · 2026-05-22T11:34:16Z

@greptileai

oss-pr-review-agent-shin · 2026-05-22T11:39:45Z

🤖 litellm-agent: This PR is currently BLOCKED from merge.

Score: 3/5 ❌

Why blocked:

1 PR-related CI failure (Size gate: tests (+274) exceed code (+80) by more than 3× — over-specified or feature too thin. Add the oversized-ok label if intentional.) (pr_related_failures, -2 pts)

Details: Score docked for: 1 PR-related CI failure (Size gate: tests (+274) exceed code (+80) by more than 3× — over-specified or feature too thin. Add the oversized-ok label if intentional.).

Fix the issues above and push an update — the bot will re-review automatically.

Note: This bot is still in beta and might not always work as expected. Please share any feedback via Slack.

Sameerlite

LGTM, thanks!

* fix(anthropic): handle empty streaming tool calls (#28549) Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * [Feature][Bug Fix] Decouple Azure OpenAI Deployment ID from model name via base_model to fix gpt5 model routing (#28490) * feat(azure): decouple deployment ID from model name via base_model Azure OpenAI deployments have arbitrary names (deployment IDs) that may not match the underlying model. Previously, model-type detection (o-series, gpt-5, etc.) relied on substring matching against the deployment name, causing misrouted configs and rejected params when deployment names were non-standard (e.g. 'my-deployment-id' for gpt-5.2). This change extends the existing base_model field to drive model-type detection, config selection, supported param resolution, and param mapping throughout the Azure call path: - _get_azure_config() uses base_model for is_o_series/is_gpt_5 checks - get_provider_chat_config() threads base_model for Azure - get_supported_openai_params() accepts and uses base_model - get_optional_params() accepts base_model and passes it to all Azure config method calls (get_supported_openai_params, map_openai_params) - azure.py completion handler uses base_model for GPT-5 detection - Config internal methods (e.g. is_model_gpt_5_2_model) now receive base_model so features like logprobs are correctly enabled Fully backward compatible - when base_model is unset, behavior is identical. Existing o_series/ and gpt5_series/ prefix workarounds continue to work. Usage in proxy config: model_list: - model_name: my-gpt5 litellm_params: model: azure/my-deployment-id model_info: base_model: azure/gpt-5.2 Fixes: non-standard deployment names like 'prefix-gpt-5.2' rejecting logprobs/top_logprobs despite the underlying model supporting them. * Addressing Greptile comments. * gemini-3.1-flash-lite pricing (#27933) * feat(model_prices): add gemini-3.1-flash-lite pricing with standard/batch/flex/priority tiers * fix pricing * add service tier --------- Co-authored-by: shin-berri <[email protected]> * fix(openai-responses): strip Anthropic cache_control from Responses API requests (#28431) Squash-merged by litellm-agent from cwang-otto's PR. * Treat None litellm_provider as wildcard in _check_provider_match (#28523) Squash-merged by litellm-agent from adityasingh2400's PR. * fix greptile * fix: use _azure_detection_model in default Azure branch of get_supported_openai_params Co-authored-by: Yassin Kortam <[email protected]> * fix(openai-responses): strip cache_control on compact endpoint as well Co-authored-by: Yassin Kortam <[email protected]> --------- Co-authored-by: Felipe Garé <[email protected]> Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: withomasmicrosoft <[email protected]> Co-authored-by: mubashir1osmani <[email protected]> Co-authored-by: cwang-otto <[email protected]> Co-authored-by: Aditya Singh <[email protected]> Co-authored-by: Cursor Agent <[email protected]> Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic): handle empty streaming tool calls (BerriAI#28549) Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> * [Feature][Bug Fix] Decouple Azure OpenAI Deployment ID from model name via base_model to fix gpt5 model routing (BerriAI#28490) * feat(azure): decouple deployment ID from model name via base_model Azure OpenAI deployments have arbitrary names (deployment IDs) that may not match the underlying model. Previously, model-type detection (o-series, gpt-5, etc.) relied on substring matching against the deployment name, causing misrouted configs and rejected params when deployment names were non-standard (e.g. 'my-deployment-id' for gpt-5.2). This change extends the existing base_model field to drive model-type detection, config selection, supported param resolution, and param mapping throughout the Azure call path: - _get_azure_config() uses base_model for is_o_series/is_gpt_5 checks - get_provider_chat_config() threads base_model for Azure - get_supported_openai_params() accepts and uses base_model - get_optional_params() accepts base_model and passes it to all Azure config method calls (get_supported_openai_params, map_openai_params) - azure.py completion handler uses base_model for GPT-5 detection - Config internal methods (e.g. is_model_gpt_5_2_model) now receive base_model so features like logprobs are correctly enabled Fully backward compatible - when base_model is unset, behavior is identical. Existing o_series/ and gpt5_series/ prefix workarounds continue to work. Usage in proxy config: model_list: - model_name: my-gpt5 litellm_params: model: azure/my-deployment-id model_info: base_model: azure/gpt-5.2 Fixes: non-standard deployment names like 'prefix-gpt-5.2' rejecting logprobs/top_logprobs despite the underlying model supporting them. * Addressing Greptile comments. * gemini-3.1-flash-lite pricing (BerriAI#27933) * feat(model_prices): add gemini-3.1-flash-lite pricing with standard/batch/flex/priority tiers * fix pricing * add service tier --------- Co-authored-by: shin-berri <[email protected]> * fix(openai-responses): strip Anthropic cache_control from Responses API requests (BerriAI#28431) Squash-merged by litellm-agent from cwang-otto's PR. * Treat None litellm_provider as wildcard in _check_provider_match (BerriAI#28523) Squash-merged by litellm-agent from adityasingh2400's PR. * fix greptile * fix: use _azure_detection_model in default Azure branch of get_supported_openai_params Co-authored-by: Yassin Kortam <[email protected]> * fix(openai-responses): strip cache_control on compact endpoint as well Co-authored-by: Yassin Kortam <[email protected]> --------- Co-authored-by: Felipe Garé <[email protected]> Co-authored-by: shin-berri <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: withomasmicrosoft <[email protected]> Co-authored-by: mubashir1osmani <[email protected]> Co-authored-by: cwang-otto <[email protected]> Co-authored-by: Aditya Singh <[email protected]> Co-authored-by: Cursor Agent <[email protected]> Co-authored-by: Yassin Kortam <[email protected]>

withomasmicrosoft added 2 commits May 21, 2026 15:33

Merge branch 'BerriAI:litellm_internal_staging' into feature-azure-op…

8afc7e1

…enai-base-model-name

greptile-apps Bot reviewed May 21, 2026

View reviewed changes

Comment thread tests/test_litellm/llms/azure/chat/test_azure_base_model_routing.py

Comment thread tests/test_litellm/llms/azure/chat/test_azure_base_model_routing.py Outdated

Comment thread litellm/utils.py

Addressing Greptile comments.

d03a3d0

Sameerlite approved these changes May 22, 2026

View reviewed changes

Sameerlite changed the base branch from litellm_internal_staging to litellm_oss_staging_2 May 22, 2026 11:50

Sameerlite merged commit 1c80fa9 into BerriAI:litellm_oss_staging_2 May 22, 2026
44 checks passed

shenshouer mentioned this pull request May 25, 2026

fix(ui): paginate session logs in drawer sidebar with Shift+J/K shortcuts #28761

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[Feature][Bug Fix] Decouple Azure OpenAI Deployment ID from model name via base_model to fix gpt5 model routing#28490

[Feature][Bug Fix] Decouple Azure OpenAI Deployment ID from model name via base_model to fix gpt5 model routing#28490
Sameerlite merged 3 commits into
BerriAI:litellm_oss_staging_2from
withomasmicrosoft:feature-azure-openai-base-model-name

withomasmicrosoft commented May 21, 2026

Uh oh!

codecov Bot commented May 21, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented May 21, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sameerlite commented May 22, 2026

Uh oh!

oss-pr-review-agent-shin Bot commented May 22, 2026

Uh oh!

Sameerlite left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

withomasmicrosoft commented May 21, 2026

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

CI (LiteLLM team)

Screenshots / Proof of Fix

Type

Changes

Uh oh!

codecov Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

greptile-apps Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sameerlite commented May 22, 2026

Uh oh!

oss-pr-review-agent-shin Bot commented May 22, 2026

Uh oh!

Sameerlite left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov Bot commented May 21, 2026 •

edited

Loading

greptile-apps Bot commented May 21, 2026 •

edited

Loading