Skip to content

Day 0: gpt 5.4 mini and nano support#23958

Merged
Sameerlite merged 15 commits intoBerriAI:litellm_dev_sameer_16_march_weekfrom
Sameerlite:litellm_gpt-5.4_mini
Mar 20, 2026
Merged

Day 0: gpt 5.4 mini and nano support#23958
Sameerlite merged 15 commits intoBerriAI:litellm_dev_sameer_16_march_weekfrom
Sameerlite:litellm_gpt-5.4_mini

Conversation

@Sameerlite
Copy link
Copy Markdown
Collaborator

@Sameerlite Sameerlite commented Mar 18, 2026

Relevant issues

All the prices are correct and checked based on the website: https://developers.openai.com/api/docs/models/gpt-5.4-mini
Both of these models support xhigh and none and have been tested locally.
No actual test has been added for this model as that would concur cost, and as we have verified that these models work locally without any error based on all the info in the model cost map

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🆕 New Feature

Changes

Add _is_reasoning_effort_level_explicitly_disabled to use opt-out semantics
for minimal effort: unknown/unlisted models pass through, only blocked when
the model map explicitly sets supports_minimal_reasoning_effort=false.

xhigh keeps opt-in semantics (must be explicitly supported).

Adds test for unknown-model passthrough and explicit-disabled detection.

Made-with: Cursor
…, add xhigh tests

- Remove unused _get_model_info_helper import
- Normalize model via get_llm_provider in _is_reasoning_effort_level_explicitly_disabled
  so provider-prefixed names (openai/gpt-5.4-mini) resolve correctly
- Add test_gpt5_4_mini_allows_reasoning_effort_xhigh
- Add test_gpt5_4_nano_allows_reasoning_effort_xhigh
- Add test_gpt5_4_mini_provider_prefixed_rejects_minimal
- Extend test_gpt5_minimal_explicitly_disabled_check for openai/gpt-5.4-mini
@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq bot commented Mar 18, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing Sameerlite:litellm_gpt-5.4_mini (0dbed19) with main (cfeafbe)

Open in CodSpeed

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 18, 2026

Greptile Summary

This PR adds Day-0 support for gpt-5.4-mini and gpt-5.4-nano by registering them in the model cost map and introducing a new minimal reasoning-effort level with opt-out semantics. It also fixes a pre-existing bug where the xhigh drop path only cleared non_default_params but not optional_params.

Key changes:

  • New models: gpt-5.4-mini and gpt-5.4-nano added to both cost map files with supports_xhigh_reasoning_effort: true, supports_none_reasoning_effort: true, and supports_minimal_reasoning_effort: false.
  • Bug fix (xhigh): The xhigh drop path now correctly pops from both non_default_params and optional_params, resolving the inconsistency flagged in the previous review.
  • New minimal guard: An elif effective_effort == "minimal" branch uses the new _is_explicitly_disabled_factory opt-out helper — unknown models pass through, only explicit false entries are blocked.
  • New helper: _is_explicitly_disabled_factory in litellm/utils.py mirrors _supports_factory but inverts the semantics (returns True only when the key is explicitly False).
  • Concern: supports_minimal_reasoning_effort: true is added to ~25 pre-existing models. Under the opt-out semantics this is redundant (missing key = not disabled), but it also means these models will pass minimal through to OpenAI even if they do not actually support it. If any of those models (e.g. gpt-5-codex, gpt-5.1-codex) reject minimal on the live API, users will receive an upstream 400 rather than a clean LiteLLM error.

Confidence Score: 4/5

  • Safe to merge after verifying that the ~25 existing models newly tagged with supports_minimal_reasoning_effort: true actually support minimal on the live OpenAI API.
  • The core logic is correct and well-tested: the pre-existing xhigh optional_params drop bug has been fixed, the new minimal opt-out guard is sound, the _is_explicitly_disabled_factory helper is well-implemented, and all new tests are mock-only and comprehensive. The score is held at 4 (not 5) because supports_minimal_reasoning_effort: true is added to a large number of pre-existing models without per-model verification evidence — if any of those models reject minimal upstream, users will receive a raw OpenAI 400 instead of a clean LiteLLM error.
  • Pay close attention to the supports_minimal_reasoning_effort: true additions in model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json for pre-existing models.

Important Files Changed

Filename Overview
litellm/llms/openai/chat/gpt_5_transformation.py Adds minimal reasoning-effort guard with opt-out semantics; fixes pre-existing xhigh bug by now popping from both non_default_params and optional_params. Logic is sound; minor naming inconsistency in is_model_gpt_5_2_model now covers gpt-5.4-mini/nano.
litellm/utils.py New _is_explicitly_disabled_factory helper correctly mirrors _supports_factory with opt-out (explicit-false-only) semantics, including the two-stage bare-model-key fallback and exception guard.
model_prices_and_context_window.json Adds gpt-5.4-mini and gpt-5.4-nano with correct supports_xhigh/none_reasoning_effort: true and supports_minimal_reasoning_effort: false. Also adds supports_minimal_reasoning_effort: true to ~25 existing models — redundant under the opt-out semantics and could pass minimal through to models that don't actually support it.
tests/test_litellm/llms/openai/test_gpt5_transformation.py Comprehensive mock-only tests cover xhigh allow/reject/drop, minimal allow/reject/drop, dict-format effort normalization, provider-prefixed model lookup, and the unknown-model pass-through invariant. All consistent with the new model map entries.
litellm/model_prices_and_context_window_backup.json Identical changes to the primary model cost file; same redundant supports_minimal_reasoning_effort: true additions to existing models.
docs/my-website/blog/gpt_5_4_mini_nano/index.md New blog post documenting gpt-5.4-mini and gpt-5.4-nano usage; content is accurate relative to the model map additions.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[map_openai_params called] --> B{effective_effort?}
    B -- xhigh --> C{_supports_reasoning_effort_level?\nopt-in: explicit True required}
    C -- supported --> D[pass xhigh through]
    C -- not supported --> E{drop_params?}
    E -- yes --> F[pop from non_default_params\nAND optional_params]
    E -- no --> G[raise UnsupportedParamsError]
    B -- minimal --> H{_is_reasoning_effort_level_explicitly_disabled?\nopt-out: explicit False required}
    H -- explicitly disabled\ni.e. supports_minimal=false in map --> I{drop_params?}
    I -- yes --> J[pop from non_default_params\nAND optional_params]
    I -- no --> K[raise UnsupportedParamsError]
    H -- not disabled\nmissing key OR true --> L[pass minimal through]
    B -- other / none / null --> M[continue normal flow]
Loading

Comments Outside Diff (2)

  1. litellm/llms/openai/chat/gpt_5_transformation.py, line 78-81 (link)

    P2 Misleading method name covers gpt-5.4 variants

    is_model_gpt_5_2_model returns True for any model starting with gpt-5.4 (including the newly-added gpt-5.4-mini and gpt-5.4-nano), but the name implies it only covers the 5.2 family. While this predates this PR, the addition of new gpt-5.4-* models makes the confusion worse. If this method is called by external or downstream code to gate 5.2-specific behaviour, it will silently include the new mini/nano variants in that gate. Consider renaming it to is_model_gpt_5_2_or_later_model or splitting the gpt-5.4 branch into its own predicate to avoid unintended side-effects.

  2. model_prices_and_context_window.json, line 328-370 (link)

    P2 supports_minimal_reasoning_effort: true additions are redundant

    The opt-out semantics of _is_explicitly_disabled_factory treat a missing key identically to an explicit true — unknown models pass through. Adding "supports_minimal_reasoning_effort": true to every pre-existing model (gpt-5.1, gpt-5.2, gpt-5-pro, gpt-5-codex, gpt-5-nano, etc.) is therefore a no-op from a runtime perspective; it neither protects nor enables anything.

    The real risk is the reverse: if any of those models actually do not support minimal in the live API (e.g. gpt-5-codex, gpt-5.1-codex), they now have an explicit true that will pass minimal through to OpenAI, potentially surfacing a 400 from the upstream API rather than a clean LiteLLM error. Given the number of models touched (~25), it is worth double-checking each one before merging.

    The only entries that carry real meaning are the explicit false values on gpt-5.4-mini and gpt-5.4-nano, which correctly block minimal.

Last reviewed commit: "Add test for reasoni..."

@Sameerlite Sameerlite changed the base branch from main to litellm_dev_sameer_16_march_week March 20, 2026 10:55
@Sameerlite Sameerlite merged commit 2d02eaa into BerriAI:litellm_dev_sameer_16_march_week Mar 20, 2026
39 of 105 checks passed
Simon-Stone added a commit to Simon-Stone/litellm that referenced this pull request Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant