fix(caching): replay openai/responses bridge cache hits as chat streams by Sameerlite · Pull Request #28158 · BerriAI/litellm

Sameerlite · 2026-05-18T11:54:53Z

Summary

Fix cache-hit deserialization for openai/responses chat-completions bridge: chat-shaped Redis payloads (chatcmpl-* / choices) are replayed as acompletion streams instead of ResponsesAPIResponse (avoids 500 on cache hit).
Skip re-wrapping cached CustomStreamWrapper streams in the responses→chat iterator so replayed chunks keep delta.content instead of empty stop-only SSE.
Passthrough chat.completion.chunk dicts in translate_responses_chunk_to_openai_stream as a fallback.

Test plan

pytest tests/local_testing/test_caching_handler.py::test_convert_cached_aresponses_bridge_chat_completion_stream -v
pytest tests/local_testing/test_caching_handler.py::test_convert_cached_streaming_responses_result_to_iterator -v
pytest tests/test_litellm/completion_extras/test_litellm_responses_transformation_transformation.py::test_translate_responses_chunk_passthrough_chat_completion_chunk -v
Proxy: POST /v1/chat/completions with openai/responses/gpt-5.4, stream: true — second identical request returns streamed text (not empty delta + stop only)

Fixes LIT-3062

Before Issue reproduced

After

Note

Medium Risk
Changes cached responses/aresponses replay logic and streaming transformation paths, which can affect what types/callbacks are returned on cache hits and how streaming chunks are interpreted.

Overview
Fixes cache-hit replay for the OpenAI responses → chat-completions bridge by detecting chat-shaped cached payloads (e.g. chatcmpl-*/choices) and replaying them as chat completion ModelResponse/CustomStreamWrapper streams instead of coercing into ResponsesAPIResponse.

Updates the bridge handler to pass through cached ModelResponse results directly and to avoid re-wrapping already-cached CustomStreamWrapper streams, plus adds a fallback in stream translation to accept raw chat.completion.chunk-shaped dicts.

Test and infra tweaks add coverage for the new cache-shape detection/bridge behavior and make the VCR Redis persister treat deserialization/corrupt cached payloads as cache misses.

^{Reviewed by Cursor Bugbot for commit 07de34e. Bugbot is set up for automated code reviews on this repo. Configure here.}

When chat completions route through openai/responses, cached ModelResponse payloads under aresponses keys were deserialized as ResponsesAPIResponse (500) or re-translated as responses events (empty streaming deltas). Deserialize chat-shaped cache entries as acompletion and bypass the responses stream iterator for cached CustomStreamWrapper replay. Co-authored-by: Cursor <[email protected]>

codecov · 2026-05-18T11:57:51Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

greptile-apps · 2026-05-18T12:00:21Z

Greptile Summary

This PR fixes cache-hit replay for the openai/responses chat-completions bridge: when Redis returns a chatcmpl-*-shaped dict (stored by the underlying chat path), the caching handler now detects it via _is_chat_completion_cached_dict and replays it as a ModelResponse (non-streaming) or CustomStreamWrapper (streaming) instead of incorrectly coercing it into a ResponsesAPIResponse.

Caching handler (caching_handler.py): adds _is_chat_completion_cached_dict (checks object prefix first, choices fallback only when object is absent) and routes matching dicts through the existing chat-completion replay path.
Bridge handler (handler.py): adds a ModelResponse early-return to fix the previously broken non-streaming path, and skips re-wrapping CustomStreamWrapper(cached_response) streams through the Responses→chat iterator.
Stream iterator (transformation.py): adds a passthrough for chat.completion.chunk-shaped dicts in translate_responses_chunk_to_openai_stream as a defensive fallback.

Confidence Score: 5/5

Safe to merge — all three changed code paths are covered by new unit tests, no existing tests were weakened, and the previously flagged non-streaming bridge bug is now correctly handled.

All changed logic paths have direct test coverage. The previously flagged issue (bridge returning a ModelResponse with no handler) is fixed by the explicit elif isinstance(result, ModelResponse): return result branch in both sync and async bridge methods. The _is_chat_completion_cached_dict discriminator correctly prioritises the object field over the broad choices fallback.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/caching/caching_handler.py	Adds `_is_chat_completion_cached_dict` helper and routes chat-shaped `aresponses`/`responses` cache hits to the chat completion replay path; well-guarded with `object` prefix check before the `choices` fallback.
litellm/completion_extras/litellm_responses_transformation/handler.py	Adds `ModelResponse` early-return (fixes the previously-flagged non-streaming issue) and skips re-wrapping of already-preformatted cached chat streams; both sync and async paths are handled consistently.
litellm/completion_extras/litellm_responses_transformation/transformation.py	Adds passthrough in `translate_responses_chunk_to_openai_stream` for `chat.completion.chunk`-shaped dicts, returning `ModelResponseStream` directly instead of failing to match any Responses API event type.
tests/_vcr_redis_persister.py	Wraps deserialization in try/except so corrupt VCR Redis cassette payloads raise `CassetteNotFoundError` (cache miss) instead of propagating the exception; logs and warns appropriately.
tests/test_litellm/caching/test_caching_handler.py	Adds four new tests covering streaming/non-streaming for both the new chat-completion cache path and the legacy Responses API path; all mock-only, no network calls.

_{Reviews (9): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile}

… replay Co-authored-by: Yassin Kortam <[email protected]>

CLAassistant · 2026-05-18T12:05:23Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ Sameerlite
✅ mateo-berri
❌ cursoragent
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

cursor

Cursor Bugbot has reviewed your changes using high mode and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

✅ Fixed: Non-streaming chat cache branch returns unhandled ModelResponse
- Added an isinstance(result, ModelResponse) branch in the bridge handler's completion/acompletion to return the cached ModelResponse directly instead of falling through to the stream-collection path that would crash on it.
✅ Fixed: Test relies on kwarg absent in production flow
- Removed the dead kwargs.get("acompletion") is True check so use_chat_completion_cache relies solely on _is_chat_completion_cached_dict, and dropped the unrealistic acompletion: True from the test kwargs so the test exercises the actual production detection path.

Preview (bda3ee6f9d)

diff --git a/litellm/caching/caching_handler.py b/litellm/caching/caching_handler.py
--- a/litellm/caching/caching_handler.py
+++ b/litellm/caching/caching_handler.py
@@ -87,6 +87,13 @@
 in_memory_cache_obj = InMemoryCache()
 
 
+def _is_chat_completion_cached_dict(cached_result: dict) -> bool:
+    cached_id = cached_result.get("id")
+    if isinstance(cached_id, str) and cached_id.startswith("chatcmpl"):
+        return True
+    return "choices" in cached_result
+
+
 def _should_defer_streaming_cache_hit_callbacks(*, kwargs: Dict[str, Any]) -> bool:
     """
     When stream=True, do not run success callbacks at cache-hit time.
@@ -861,27 +868,47 @@
         elif (call_type == "aresponses" or call_type == "responses") and isinstance(
             cached_result, dict
         ):
-            from litellm.responses.streaming_iterator import (
-                CachedResponsesAPIStreamingIterator,
-            )
+            use_chat_completion_cache = _is_chat_completion_cached_dict(cached_result)
+            if use_chat_completion_cache:
+                if kwargs.get("stream", False) is True:
+                    bridge_call_type = (
+                        CallTypes.acompletion.value
+                        if call_type == "aresponses"
+                        else CallTypes.completion.value
+                    )
+                    cached_result = self._convert_cached_stream_response(
+                        cached_result=cached_result,
+                        call_type=bridge_call_type,
+                        logging_obj=logging_obj,
+                        model=model,
+                    )
+                else:
+                    cached_result = convert_to_model_response_object(
+                        response_object=cached_result,
+                        model_response_object=ModelResponse(),
+                    )
+            else:
+                from litellm.responses.streaming_iterator import (
+                    CachedResponsesAPIStreamingIterator,
+                )
 
-            response_obj = ResponsesAPIResponse(**cached_result)
-            if (
-                hasattr(response_obj, "_hidden_params")
-                and response_obj._hidden_params is not None
-                and isinstance(response_obj._hidden_params, dict)
-            ):
-                response_obj._hidden_params["cache_hit"] = True
+                response_obj = ResponsesAPIResponse(**cached_result)
+                if (
+                    hasattr(response_obj, "_hidden_params")
+                    and response_obj._hidden_params is not None
+                    and isinstance(response_obj._hidden_params, dict)
+                ):
+                    response_obj._hidden_params["cache_hit"] = True
 
-            if kwargs.get("stream", False) is True:
-                cached_result = CachedResponsesAPIStreamingIterator(
-                    response=response_obj,
-                    logging_obj=logging_obj,
-                    request_data=kwargs,
-                    call_type=call_type,
-                )
-            else:
-                cached_result = response_obj
+                if kwargs.get("stream", False) is True:
+                    cached_result = CachedResponsesAPIStreamingIterator(
+                        response=response_obj,
+                        logging_obj=logging_obj,
+                        request_data=kwargs,
+                        call_type=call_type,
+                    )
+                else:
+                    cached_result = response_obj
 
         if (
             hasattr(cached_result, "_hidden_params")

diff --git a/litellm/completion_extras/litellm_responses_transformation/handler.py b/litellm/completion_extras/litellm_responses_transformation/handler.py
--- a/litellm/completion_extras/litellm_responses_transformation/handler.py
+++ b/litellm/completion_extras/litellm_responses_transformation/handler.py
@@ -38,6 +38,15 @@
         return bool(stream)
 
     @staticmethod
+    def _is_preformatted_cached_chat_stream(result: Any) -> bool:
+        from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper
+
+        return (
+            isinstance(result, CustomStreamWrapper)
+            and result.custom_llm_provider == "cached_response"
+        )
+
+    @staticmethod
     def _coerce_response_object(
         response_obj: Any,
         hidden_params: Optional[dict],
@@ -177,6 +186,8 @@
             **request_data,
         )
 
+        from litellm.types.utils import ModelResponse
+
         stream = self._resolve_stream_flag(optional_params, litellm_params)
         if isinstance(result, ResponsesAPIResponse):
             return self.transformation_handler.transform_response(
@@ -192,6 +203,8 @@
                 api_key=kwargs.get("api_key"),
                 json_mode=kwargs.get("json_mode"),
             )
+        elif isinstance(result, ModelResponse):
+            return result
         elif not stream:
             responses_api_response = self._collect_response_from_stream(result)
             return self.transformation_handler.transform_response(
@@ -208,6 +221,10 @@
                 json_mode=kwargs.get("json_mode"),
             )
         else:
+            if self._is_preformatted_cached_chat_stream(result):
+                return self._apply_post_stream_processing(
+                    result, model, custom_llm_provider
+                )
             completion_stream = self.transformation_handler.get_model_response_iterator(
                 streaming_response=result,  # type: ignore
                 sync_stream=True,
@@ -256,6 +273,8 @@
             aresponses=True,
         )
 
+        from litellm.types.utils import ModelResponse
+
         stream = self._resolve_stream_flag(optional_params, litellm_params)
         if isinstance(result, ResponsesAPIResponse):
             return self.transformation_handler.transform_response(
@@ -271,6 +290,8 @@
                 api_key=kwargs.get("api_key"),
                 json_mode=kwargs.get("json_mode"),
             )
+        elif isinstance(result, ModelResponse):
+            return result
         elif not stream:
             responses_api_response = await self._collect_response_from_stream_async(
                 result
@@ -289,6 +310,10 @@
                 json_mode=kwargs.get("json_mode"),
             )
         else:
+            if self._is_preformatted_cached_chat_stream(result):
+                return self._apply_post_stream_processing(
+                    result, model, custom_llm_provider
+                )
             completion_stream = self.transformation_handler.get_model_response_iterator(
                 streaming_response=result,  # type: ignore
                 sync_stream=False,

diff --git a/litellm/completion_extras/litellm_responses_transformation/transformation.py b/litellm/completion_extras/litellm_responses_transformation/transformation.py
--- a/litellm/completion_extras/litellm_responses_transformation/transformation.py
+++ b/litellm/completion_extras/litellm_responses_transformation/transformation.py
@@ -1141,6 +1141,14 @@
         event_type = parsed_chunk.get("type")
         if isinstance(event_type, ResponsesAPIStreamEvents):
             event_type = event_type.value
+
+        if parsed_chunk.get("object") == "chat.completion.chunk" or (
+            event_type is None
+            and isinstance(parsed_chunk.get("choices"), list)
+            and parsed_chunk.get("choices")
+        ):
+            return ModelResponseStream(**parsed_chunk)
+
         verbose_logger.debug(f"Chat provider: Processing event type: {event_type}")
 
         if event_type == "response.created":

diff --git a/tests/local_testing/test_caching_handler.py b/tests/local_testing/test_caching_handler.py
--- a/tests/local_testing/test_caching_handler.py
+++ b/tests/local_testing/test_caching_handler.py
@@ -40,6 +40,7 @@
 from litellm.types.llms.openai import ResponsesAPIResponse
 from datetime import timedelta, datetime
 from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLogging
+from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper
 from litellm._logging import verbose_logger
 import logging
 
@@ -1072,6 +1073,58 @@
     )
 
 
+def test_convert_cached_aresponses_bridge_chat_completion_stream():
+    """
+    openai/responses chat-completions bridge caches ModelResponse JSON on aresponses
+    cache keys; replay must not call ResponsesAPIResponse(**chatcmpl_dict).
+    """
+    caching_handler = LLMCachingHandler(
+        original_function=aresponses, request_kwargs={}, start_time=datetime.now()
+    )
+    logging_obj = LiteLLMLogging(
+        litellm_call_id=str(datetime.now()),
+        call_type=CallTypes.aresponses.value,
+        model="gpt-5.4",
+        messages=[],
+        function_id=str(uuid.uuid4()),
+        stream=True,
+        start_time=datetime.now(),
+    )
+    cached_result = {
+        "id": "chatcmpl-bridge-cache-test",
+        "object": "chat.completion",
+        "created": int(time.time()),
+        "model": "gpt-5.4",
+        "choices": [
+            {
+                "index": 0,
+                "message": {"role": "assistant", "content": "Hi!"},
+                "finish_reason": "stop",
+            }
+        ],
+        "usage": {
+            "prompt_tokens": 7,
+            "completion_tokens": 11,
+            "total_tokens": 18,
+        },
+    }
+
+    result = caching_handler._convert_cached_result_to_model_response(
+        cached_result=cached_result,
+        call_type=CallTypes.aresponses.value,
+        kwargs={
+            "model": "gpt-5.4",
+            "stream": True,
+            "messages": [{"role": "user", "content": "hi"}],
+        },
+        logging_obj=logging_obj,
+        model="gpt-5.4",
+        args=(),
+    )
+
+    assert isinstance(result, CustomStreamWrapper)
+
+
 def test_convert_cached_streaming_reasoning_result_to_iterator():
     caching_handler = LLMCachingHandler(
         original_function=responses, request_kwargs={}, start_time=datetime.now()

diff --git a/tests/test_litellm/completion_extras/test_litellm_responses_transformation_transformation.py b/tests/test_litellm/completion_extras/test_litellm_responses_transformation_transformation.py
--- a/tests/test_litellm/completion_extras/test_litellm_responses_transformation_transformation.py
+++ b/tests/test_litellm/completion_extras/test_litellm_responses_transformation_transformation.py
@@ -230,3 +230,30 @@
 
     assert "metadata" not in result
     assert result["litellm_metadata"]["internal_key"] == "secret"
+
+
+def test_translate_responses_chunk_passthrough_chat_completion_chunk():
+    from litellm.completion_extras.litellm_responses_transformation.transformation import (
+        OpenAiResponsesToChatCompletionStreamIterator,
+    )
+
+    chat_chunk = {
+        "id": "chatcmpl-cache-passthrough",
+        "object": "chat.completion.chunk",
+        "created": 1779104834,
+        "model": "gpt-5.4",
+        "choices": [
+            {
+                "index": 0,
+                "delta": {"role": "assistant", "content": "Hi! How can I help?"},
+                "finish_reason": None,
+            }
+        ],
+    }
+
+    result = OpenAiResponsesToChatCompletionStreamIterator.translate_responses_chunk_to_openai_stream(
+        chat_chunk
+    )
+
+    assert result.choices[0].delta.content == "Hi! How can I help?"
+    assert result.choices[0].finish_reason is None

_{You can send follow-ups to the cloud agent here.}

…ead acompletion check Co-authored-by: Yassin Kortam <[email protected]>

…fallback Prefer chat.completion object type over the broad choices-key heuristic so Responses API cached payloads are not misclassified if their schema changes. Co-authored-by: Cursor <[email protected]>

Sameerlite · 2026-05-18T12:39:00Z

@greptile re review

…est suite The new bridge cache replay logic in caching_handler.py and the preformatted-stream guard in litellm_responses_transformation/handler.py were exercised only by tests under tests/local_testing/, which the responses-caching-types and misc shards do not run. Codecov flagged the patch as 29.72% covered. Add equivalent unit tests under tests/test_litellm/ so the responses, caching, types, and misc shards execute them and ship their coverage data to Codecov: - _is_chat_completion_cached_dict happy/sad paths - aresponses streaming bridge cache hit -> CustomStreamWrapper - responses non-streaming bridge cache hit -> ModelResponse - legacy ResponsesAPIResponse stream + non-stream replay - _is_preformatted_cached_chat_stream true/false - completion/acompletion early return on cached ModelResponse - completion/acompletion skip rewrap on preformatted cached stream

mateo-berri · 2026-05-18T20:54:45Z

@greptileai

cursor

Cursor Bugbot has reviewed your changes using high mode and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

✅ Fixed: Missing negative guard on object field in classifier
- When object is a string, the classifier now returns the result of startswith("chat.completion") directly instead of falling through to the choices in cached_result heuristic, so a Responses payload with object: "response" is no longer misclassified.

Preview (8f867396b6)

diff --git a/litellm/caching/caching_handler.py b/litellm/caching/caching_handler.py
--- a/litellm/caching/caching_handler.py
+++ b/litellm/caching/caching_handler.py
@@ -87,6 +87,16 @@
 in_memory_cache_obj = InMemoryCache()
 
 
+def _is_chat_completion_cached_dict(cached_result: dict) -> bool:
+    cached_id = cached_result.get("id")
+    if isinstance(cached_id, str) and cached_id.startswith("chatcmpl"):
+        return True
+    obj = cached_result.get("object")
+    if isinstance(obj, str):
+        return obj.startswith("chat.completion")
+    return "choices" in cached_result
+
+
 def _should_defer_streaming_cache_hit_callbacks(*, kwargs: Dict[str, Any]) -> bool:
     """
     When stream=True, do not run success callbacks at cache-hit time.
@@ -861,27 +871,47 @@
         elif (call_type == "aresponses" or call_type == "responses") and isinstance(
             cached_result, dict
         ):
-            from litellm.responses.streaming_iterator import (
-                CachedResponsesAPIStreamingIterator,
-            )
+            use_chat_completion_cache = _is_chat_completion_cached_dict(cached_result)
+            if use_chat_completion_cache:
+                if kwargs.get("stream", False) is True:
+                    bridge_call_type = (
+                        CallTypes.acompletion.value
+                        if call_type == "aresponses"
+                        else CallTypes.completion.value
+                    )
+                    cached_result = self._convert_cached_stream_response(
+                        cached_result=cached_result,
+                        call_type=bridge_call_type,
+                        logging_obj=logging_obj,
+                        model=model,
+                    )
+                else:
+                    cached_result = convert_to_model_response_object(
+                        response_object=cached_result,
+                        model_response_object=ModelResponse(),
+                    )
+            else:
+                from litellm.responses.streaming_iterator import (
+                    CachedResponsesAPIStreamingIterator,
+                )
 
-            response_obj = ResponsesAPIResponse(**cached_result)
-            if (
-                hasattr(response_obj, "_hidden_params")
-                and response_obj._hidden_params is not None
-                and isinstance(response_obj._hidden_params, dict)
-            ):
-                response_obj._hidden_params["cache_hit"] = True
+                response_obj = ResponsesAPIResponse(**cached_result)
+                if (
+                    hasattr(response_obj, "_hidden_params")
+                    and response_obj._hidden_params is not None
+                    and isinstance(response_obj._hidden_params, dict)
+                ):
+                    response_obj._hidden_params["cache_hit"] = True
 
-            if kwargs.get("stream", False) is True:
-                cached_result = CachedResponsesAPIStreamingIterator(
-                    response=response_obj,
-                    logging_obj=logging_obj,
-                    request_data=kwargs,
-                    call_type=call_type,
-                )
-            else:
-                cached_result = response_obj
+                if kwargs.get("stream", False) is True:
+                    cached_result = CachedResponsesAPIStreamingIterator(
+                        response=response_obj,
+                        logging_obj=logging_obj,
+                        request_data=kwargs,
+                        call_type=call_type,
+                    )
+                else:
+                    cached_result = response_obj
 
         if (
             hasattr(cached_result, "_hidden_params")

diff --git a/litellm/completion_extras/litellm_responses_transformation/handler.py b/litellm/completion_extras/litellm_responses_transformation/handler.py
--- a/litellm/completion_extras/litellm_responses_transformation/handler.py
+++ b/litellm/completion_extras/litellm_responses_transformation/handler.py
@@ -38,6 +38,15 @@
         return bool(stream)
 
     @staticmethod
+    def _is_preformatted_cached_chat_stream(result: Any) -> bool:
+        from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper
+
+        return (
+            isinstance(result, CustomStreamWrapper)
+            and result.custom_llm_provider == "cached_response"
+        )
+
+    @staticmethod
     def _coerce_response_object(
         response_obj: Any,
         hidden_params: Optional[dict],
@@ -177,6 +186,8 @@
             **request_data,
         )
 
+        from litellm.types.utils import ModelResponse
+
         stream = self._resolve_stream_flag(optional_params, litellm_params)
         if isinstance(result, ResponsesAPIResponse):
             return self.transformation_handler.transform_response(
@@ -192,6 +203,8 @@
                 api_key=kwargs.get("api_key"),
                 json_mode=kwargs.get("json_mode"),
             )
+        elif isinstance(result, ModelResponse):
+            return result
         elif not stream:
             responses_api_response = self._collect_response_from_stream(result)
             return self.transformation_handler.transform_response(
@@ -208,6 +221,10 @@
                 json_mode=kwargs.get("json_mode"),
             )
         else:
+            if self._is_preformatted_cached_chat_stream(result):
+                return self._apply_post_stream_processing(
+                    result, model, custom_llm_provider
+                )
             completion_stream = self.transformation_handler.get_model_response_iterator(
                 streaming_response=result,  # type: ignore
                 sync_stream=True,
@@ -256,6 +273,8 @@
             aresponses=True,
         )
 
+        from litellm.types.utils import ModelResponse
+
         stream = self._resolve_stream_flag(optional_params, litellm_params)
         if isinstance(result, ResponsesAPIResponse):
             return self.transformation_handler.transform_response(
@@ -271,6 +290,8 @@
                 api_key=kwargs.get("api_key"),
                 json_mode=kwargs.get("json_mode"),
             )
+        elif isinstance(result, ModelResponse):
+            return result
         elif not stream:
             responses_api_response = await self._collect_response_from_stream_async(
                 result
@@ -289,6 +310,10 @@
                 json_mode=kwargs.get("json_mode"),
             )
         else:
+            if self._is_preformatted_cached_chat_stream(result):
+                return self._apply_post_stream_processing(
+                    result, model, custom_llm_provider
+                )
             completion_stream = self.transformation_handler.get_model_response_iterator(
                 streaming_response=result,  # type: ignore
                 sync_stream=False,

diff --git a/litellm/completion_extras/litellm_responses_transformation/transformation.py b/litellm/completion_extras/litellm_responses_transformation/transformation.py
--- a/litellm/completion_extras/litellm_responses_transformation/transformation.py
+++ b/litellm/completion_extras/litellm_responses_transformation/transformation.py
@@ -1141,6 +1141,14 @@
         event_type = parsed_chunk.get("type")
         if isinstance(event_type, ResponsesAPIStreamEvents):
             event_type = event_type.value
+
+        if parsed_chunk.get("object") == "chat.completion.chunk" or (
+            event_type is None
+            and isinstance(parsed_chunk.get("choices"), list)
+            and parsed_chunk.get("choices")
+        ):
+            return ModelResponseStream(**parsed_chunk)
+
         verbose_logger.debug(f"Chat provider: Processing event type: {event_type}")
 
         if event_type == "response.created":

diff --git a/tests/local_testing/test_caching_handler.py b/tests/local_testing/test_caching_handler.py
--- a/tests/local_testing/test_caching_handler.py
+++ b/tests/local_testing/test_caching_handler.py
@@ -25,6 +25,7 @@
 from litellm.caching.caching_handler import (
     LLMCachingHandler,
     CachingHandlerResponse,
+    _is_chat_completion_cached_dict,
     _should_defer_streaming_cache_hit_callbacks,
 )
 from litellm.caching.caching import LiteLLMCacheType
@@ -40,6 +41,7 @@
 from litellm.types.llms.openai import ResponsesAPIResponse
 from datetime import timedelta, datetime
 from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLogging
+from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper
 from litellm._logging import verbose_logger
 import logging
 
@@ -1072,6 +1074,70 @@
     )
 
 
+def test_is_chat_completion_cached_dict():
+    assert _is_chat_completion_cached_dict(
+        {"id": "chatcmpl-abc", "object": "chat.completion", "choices": []}
+    )
+    assert _is_chat_completion_cached_dict(
+        {"id": "other", "object": "chat.completion.chunk", "choices": []}
+    )
+    assert not _is_chat_completion_cached_dict(
+        {"id": "resp_abc", "object": "response", "output": []}
+    )
+
+
+def test_convert_cached_aresponses_bridge_chat_completion_stream():
+    """
+    openai/responses chat-completions bridge caches ModelResponse JSON on aresponses
+    cache keys; replay must not call ResponsesAPIResponse(**chatcmpl_dict).
+    """
+    caching_handler = LLMCachingHandler(
+        original_function=aresponses, request_kwargs={}, start_time=datetime.now()
+    )
+    logging_obj = LiteLLMLogging(
+        litellm_call_id=str(datetime.now()),
+        call_type=CallTypes.aresponses.value,
+        model="gpt-5.4",
+        messages=[],
+        function_id=str(uuid.uuid4()),
+        stream=True,
+        start_time=datetime.now(),
+    )
+    cached_result = {
+        "id": "chatcmpl-bridge-cache-test",
+        "object": "chat.completion",
+        "created": int(time.time()),
+        "model": "gpt-5.4",
+        "choices": [
+            {
+                "index": 0,
+                "message": {"role": "assistant", "content": "Hi!"},
+                "finish_reason": "stop",
+            }
+        ],
+        "usage": {
+            "prompt_tokens": 7,
+            "completion_tokens": 11,
+            "total_tokens": 18,
+        },
+    }
+
+    result = caching_handler._convert_cached_result_to_model_response(
+        cached_result=cached_result,
+        call_type=CallTypes.aresponses.value,
+        kwargs={
+            "model": "gpt-5.4",
+            "stream": True,
+            "messages": [{"role": "user", "content": "hi"}],
+        },
+        logging_obj=logging_obj,
+        model="gpt-5.4",
+        args=(),
+    )
+
+    assert isinstance(result, CustomStreamWrapper)
+
+
 def test_convert_cached_streaming_reasoning_result_to_iterator():
     caching_handler = LLMCachingHandler(
         original_function=responses, request_kwargs={}, start_time=datetime.now()

diff --git a/tests/test_litellm/caching/test_caching_handler.py b/tests/test_litellm/caching/test_caching_handler.py
--- a/tests/test_litellm/caching/test_caching_handler.py
+++ b/tests/test_litellm/caching/test_caching_handler.py
@@ -232,3 +232,207 @@
     combined = llm_caching_handler.combine_usage(usage_a, usage_c)
     assert combined.prompt_tokens_details is not None
     assert combined.prompt_tokens_details.image_count == 1
+
+
+def test_is_chat_completion_cached_dict():
+    from litellm.caching.caching_handler import _is_chat_completion_cached_dict
+
+    assert _is_chat_completion_cached_dict(
+        {"id": "chatcmpl-abc", "object": "chat.completion", "choices": []}
+    )
+    assert _is_chat_completion_cached_dict(
+        {"id": "other", "object": "chat.completion.chunk", "choices": []}
+    )
+    assert _is_chat_completion_cached_dict(
+        {"id": "no-object", "choices": [{"index": 0}]}
+    )
+    assert not _is_chat_completion_cached_dict(
+        {"id": "resp_abc", "object": "response", "output": []}
+    )
+
+
+def _build_logging_obj(call_type: str, stream: bool):
+    import uuid as _uuid
+
+    from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLogging
+
+    return LiteLLMLogging(
+        litellm_call_id=str(datetime.now()),
+        call_type=call_type,
+        model="gpt-5.4",
+        messages=[],
+        function_id=str(_uuid.uuid4()),
+        stream=stream,
+        start_time=datetime.now(),
+    )
+
+
+def test_convert_cached_aresponses_bridge_chat_completion_stream():
+    """openai/responses chat-completions bridge: streaming cache hit replays as chat stream."""
+    from litellm import aresponses
+    from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper
+    from litellm.types.utils import CallTypes
+
+    caching_handler = LLMCachingHandler(
+        original_function=aresponses, request_kwargs={}, start_time=datetime.now()
+    )
+    cached_result = {
+        "id": "chatcmpl-bridge-cache-test",
+        "object": "chat.completion",
+        "created": int(time.time()),
+        "model": "gpt-5.4",
+        "choices": [
+            {
+                "index": 0,
+                "message": {"role": "assistant", "content": "Hi!"},
+                "finish_reason": "stop",
+            }
+        ],
+        "usage": {"prompt_tokens": 7, "completion_tokens": 11, "total_tokens": 18},
+    }
+
+    result = caching_handler._convert_cached_result_to_model_response(
+        cached_result=cached_result,
+        call_type=CallTypes.aresponses.value,
+        kwargs={
+            "model": "gpt-5.4",
+            "stream": True,
+            "messages": [{"role": "user", "content": "hi"}],
+        },
+        logging_obj=_build_logging_obj(CallTypes.aresponses.value, stream=True),
+        model="gpt-5.4",
+        args=(),
+    )
+
+    assert isinstance(result, CustomStreamWrapper)
+
+
+def test_convert_cached_responses_bridge_chat_completion_nonstream():
+    """openai/responses chat-completions bridge: non-streaming cache hit replays as ModelResponse."""
+    from litellm import responses
+    from litellm.types.utils import CallTypes, ModelResponse
+
+    caching_handler = LLMCachingHandler(
+        original_function=responses, request_kwargs={}, start_time=datetime.now()
+    )
+    cached_result = {
+        "id": "chatcmpl-bridge-nonstream",
+        "object": "chat.completion",
+        "created": int(time.time()),
+        "model": "gpt-5.4",
+        "choices": [
+            {
+                "index": 0,
+                "message": {"role": "assistant", "content": "Hi!"},
+                "finish_reason": "stop",
+            }
+        ],
+        "usage": {"prompt_tokens": 7, "completion_tokens": 11, "total_tokens": 18},
+    }
+
+    result = caching_handler._convert_cached_result_to_model_response(
+        cached_result=cached_result,
+        call_type=CallTypes.responses.value,
+        kwargs={
+            "model": "gpt-5.4",
+            "stream": False,
+            "messages": [{"role": "user", "content": "hi"}],
+        },
+        logging_obj=_build_logging_obj(CallTypes.responses.value, stream=False),
+        model="gpt-5.4",
+        args=(),
+    )
+
+    assert isinstance(result, ModelResponse)
+    assert result.choices[0].message.content == "Hi!"
+
+
+def test_convert_cached_responses_legacy_nonstream_path():
+    """Genuine ResponsesAPIResponse dict (no chatcmpl/choices) falls through legacy path."""
+    from litellm import responses
+    from litellm.types.llms.openai import ResponsesAPIResponse
+    from litellm.types.utils import CallTypes
+
+    caching_handler = LLMCachingHandler(
+        original_function=responses, request_kwargs={}, start_time=datetime.now()
+    )
+    cached_result = {
+        "id": "resp_legacy_nonstream",
+        "created_at": int(time.time()),
+        "status": "completed",
+        "model": "gpt-4o",
+        "object": "response",
+        "output": [
+            {
+                "type": "message",
+                "id": "msg_legacy",
+                "status": "completed",
+                "role": "assistant",
+                "content": [
+                    {
+                        "type": "output_text",
+                        "text": "legacy response",
+                        "annotations": [],
+                    }
+                ],
+            }
+        ],
+    }
+
+    result = caching_handler._convert_cached_result_to_model_response(
+        cached_result=cached_result,
+        call_type=CallTypes.responses.value,
+        kwargs={"model": "gpt-4o", "input": "hi", "stream": False},
+        logging_obj=_build_logging_obj(CallTypes.responses.value, stream=False),
+        model="gpt-4o",
+        args=(),
+    )
+
+    assert isinstance(result, ResponsesAPIResponse)
+    assert result.id == "resp_legacy_nonstream"
+
+
+def test_convert_cached_responses_legacy_stream_path():
+    """Genuine ResponsesAPIResponse dict (no chatcmpl/choices) on stream falls through legacy path."""
+    from litellm import responses
+    from litellm.responses.streaming_iterator import (
+        CachedResponsesAPIStreamingIterator,
+    )
+    from litellm.types.utils import CallTypes
+
+    caching_handler = LLMCachingHandler(
+        original_function=responses, request_kwargs={}, start_time=datetime.now()
+    )
+    cached_result = {
+        "id": "resp_legacy_stream",
+        "created_at": int(time.time()),
+        "status": "completed",
+        "model": "gpt-4o",
+        "object": "response",
+        "output": [
+            {
+                "type": "message",
+                "id": "msg_legacy_stream",
+                "status": "completed",
+                "role": "assistant",
+                "content": [
+                    {
+                        "type": "output_text",
+                        "text": "legacy stream",
+                        "annotations": [],
+                    }
+                ],
+            }
+        ],
+    }
+
+    result = caching_handler._convert_cached_result_to_model_response(
+        cached_result=cached_result,
+        call_type=CallTypes.responses.value,
+        kwargs={"model": "gpt-4o", "input": "hi", "stream": True},
+        logging_obj=_build_logging_obj(CallTypes.responses.value, stream=True),
+        model="gpt-4o",
+        args=(),
+    )
+
+    assert isinstance(result, CachedResponsesAPIStreamingIterator)

diff --git a/tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_handler.py b/tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_handler.py
new file mode 100644
--- /dev/null
+++ b/tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_handler.py
@@ -1,0 +1,150 @@
+import os
+import sys
+from datetime import datetime
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+
+sys.path.insert(0, os.path.abspath("../../.."))
+
+from litellm.completion_extras.litellm_responses_transformation.handler import (
+    ResponsesToCompletionBridgeHandler,
+)
+from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLogging
+from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper
+from litellm.types.utils import ModelResponse
+
+
+def test_is_preformatted_cached_chat_stream_true():
+    stream = MagicMock(spec=CustomStreamWrapper)
+    stream.custom_llm_provider = "cached_response"
+    assert (
+        ResponsesToCompletionBridgeHandler._is_preformatted_cached_chat_stream(stream)
+        is True
+    )
+
+
+def test_is_preformatted_cached_chat_stream_false_wrong_provider():
+    stream = MagicMock(spec=CustomStreamWrapper)
+    stream.custom_llm_provider = "openai"
+    assert (
+        ResponsesToCompletionBridgeHandler._is_preformatted_cached_chat_stream(stream)
+        is False
+    )
+
+
+def test_is_preformatted_cached_chat_stream_false_wrong_type():
+    assert (
+        ResponsesToCompletionBridgeHandler._is_preformatted_cached_chat_stream(
+            {"object": "chat.completion.chunk"}
+        )
+        is False
+    )
+
+
+def _bridge_kwargs(stream: bool):
+    logging_obj = LiteLLMLogging(
+        litellm_call_id="test-call",
+        call_type="completion",
+        model="gpt-5.4",
+        messages=[{"role": "user", "content": "hi"}],
+        function_id="fn-id",
+        stream=stream,
+        start_time=datetime.now(),
+    )
+    return {
+        "model": "gpt-5.4",
+        "custom_llm_provider": "openai",
+        "messages": [{"role": "user", "content": "hi"}],
+        "optional_params": {"stream": stream},
+        "litellm_params": {},
+        "headers": {},
+        "model_response": ModelResponse(),
+        "logging_obj": logging_obj,
+    }
+
+
+def test_completion_returns_cached_model_response_directly():
+    """Non-streaming bridge cache hit: responses() returns a ModelResponse -> bridge returns it as-is."""
+    cached = ModelResponse(id="chatcmpl-cached-nonstream", model="gpt-5.4")
+    bridge = ResponsesToCompletionBridgeHandler()
+
+    with (
+        patch.object(
+            bridge.transformation_handler,
+            "transform_request",
+            return_value={"model": "gpt-5.4", "input": "hi"},
+        ),
+        patch("litellm.responses", return_value=cached),
+    ):
+        result = bridge.completion(**_bridge_kwargs(stream=False))
+
+    assert result is cached
+
+
+@pytest.mark.asyncio
+async def test_acompletion_returns_cached_model_response_directly():
+    cached = ModelResponse(id="chatcmpl-cached-nonstream-async", model="gpt-5.4")
+    bridge = ResponsesToCompletionBridgeHandler()
+
+    with (
+        patch.object(
+            bridge.transformation_handler,
+            "transform_request",
+            return_value={"model": "gpt-5.4", "input": "hi"},
+        ),
+        patch("litellm.aresponses", new=AsyncMock(return_value=cached)),
+    ):
+        result = await bridge.acompletion(**_bridge_kwargs(stream=False))
+
+    assert result is cached
+
+
+def test_completion_skips_rewrapping_preformatted_cached_chat_stream():
+    """Streaming bridge cache hit returning CustomStreamWrapper(cached_response) -> bridge skips re-wrapping."""
+    stream = MagicMock(spec=CustomStreamWrapper)
+    stream.custom_llm_provider = "cached_response"
+    bridge = ResponsesToCompletionBridgeHandler()
+
+    with (
+        patch.object(
+            bridge.transformation_handler,
+            "transform_request",
+            return_value={"model": "gpt-5.4", "input": "hi"},
+        ),
+        patch("litellm.responses", return_value=stream),
+        patch.object(
+            bridge,
+            "_apply_post_stream_processing",
+            side_effect=lambda s, *a, **kw: s,
+        ) as post,
+    ):
+        result = bridge.completion(**_bridge_kwargs(stream=True))
+
+    post.assert_called_once()
+    assert result is stream
+
+
+@pytest.mark.asyncio
+async def test_acompletion_skips_rewrapping_preformatted_cached_chat_stream():
+    stream = MagicMock(spec=CustomStreamWrapper)
+    stream.custom_llm_provider = "cached_response"
+    bridge = ResponsesToCompletionBridgeHandler()
+
+    with (
+        patch.object(
+            bridge.transformation_handler,
+            "transform_request",
+            return_value={"model": "gpt-5.4", "input": "hi"},
+        ),
+        patch("litellm.aresponses", new=AsyncMock(return_value=stream)),
+        patch.object(
+            bridge,
+            "_apply_post_stream_processing",
+            side_effect=lambda s, *a, **kw: s,
+        ) as post,
+    ):
+        result = await bridge.acompletion(**_bridge_kwargs(stream=True))
+
+    post.assert_called_once()
+    assert result is stream

diff --git a/tests/test_litellm/completion_extras/test_litellm_responses_transformation_transformation.py b/tests/test_litellm/completion_extras/test_litellm_responses_transformation_transformation.py
--- a/tests/test_litellm/completion_extras/test_litellm_responses_transformation_transformation.py
+++ b/tests/test_litellm/completion_extras/test_litellm_responses_transformation_transformation.py
@@ -230,3 +230,30 @@
 
     assert "metadata" not in result
     assert result["litellm_metadata"]["internal_key"] == "secret"
+
+
+def test_translate_responses_chunk_passthrough_chat_completion_chunk():
+    from litellm.completion_extras.litellm_responses_transformation.transformation import (
+        OpenAiResponsesToChatCompletionStreamIterator,
+    )
+
+    chat_chunk = {
+        "id": "chatcmpl-cache-passthrough",
+        "object": "chat.completion.chunk",
+        "created": 1779104834,
+        "model": "gpt-5.4",
+        "choices": [
+            {
+                "index": 0,
+                "delta": {"role": "assistant", "content": "Hi! How can I help?"},
+                "finish_reason": None,
+            }
+        ],
+    }
+
+    result = OpenAiResponsesToChatCompletionStreamIterator.translate_responses_chunk_to_openai_stream(
+        chat_chunk
+    )
+
+    assert result.choices[0].delta.content == "Hi! How can I help?"
+    assert result.choices[0].finish_reason is None

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit 9c1eb56. Configure here.}

…_dict Co-authored-by: Yassin Kortam <[email protected]>

mateo-berri · 2026-05-18T21:03:29Z

@greptileai

mateo-berri · 2026-05-18T21:17:26Z

@greptileai

mateo-berri

LGTM; thanks!

The NVIDIA hosted rerank endpoint for nvidia/llama-3_2-nv-rerankqa-1b-v2 reached end-of-life on 2026-05-18 and now returns HTTP 410 Gone, breaking TestNvidiaNim::test_basic_rerank. Switch to nvidia/nv-rerankqa-mistral-4b-v3, which is still hosted on the NVIDIA API catalog and is already listed in model_prices_and_context_window.json. OpenAI also retired the gpt-4o-realtime-preview-2024-12-17 model used by test_realtime_guardrails_openai (now returns model_not_found). Switch the realtime test URL to the GA gpt-realtime alias. Unrelated to the responses-bridge cache fix in this PR, but committing here to unblock CI per maintainer guidance. Co-authored-by: Mateo Wang <[email protected]>

mateo-berri · 2026-05-18T22:24:38Z

@greptileai

OpenAI removed gpt-4o-realtime-preview and all its date snapshots on 2026-05-18 (every variant now returns model_not_found), breaking the live-WebSocket OpenAI realtime tests in CI: - test_openai_realtime_direct_call_no_intent - test_openai_realtime_direct_call_with_intent - TestOpenAIRealtime.test_realtime_connection - TestOpenAIRealtime.test_realtime_with_query_params Point each of those to the current GA alias gpt-realtime (verified live). Pure unit/mock tests that just assert the string value (e.g. in test_realtime_query_params_construction and the test_realtime_query_params_use_normalized_model_name mock) are left alone since they do not depend on model availability. Also relax the AI-response assertion in test_text_message_blocked_by_guardrail_no_ai_response: gpt-realtime occasionally produces a polite refusal ("I'm sorry, but I can't say that") when the cancel arrives after the model has already started generating, which is the expected outcome (no real AI content) but does not contain the words 'blocked' or 'guardrail'. The primary guardrail behaviour (guardrail_violation error event + transcript_delta block message) is still asserted unchanged. Co-authored-by: Mateo Wang <[email protected]>

mateo-berri · 2026-05-18T22:37:50Z

@greptileai

NVIDIA reached end-of-life for the hosted nvidia/llama-3.2-nv-rerankqa-1b-v2 rerank API on 2026-05-18 (returns HTTP 410 Gone), and the proposed replacement nv-rerankqa-mistral-4b-v3 returns HTTP 404 for the CI account, breaking TestNvidiaNim::test_basic_rerank. Override test_basic_rerank to mock the HTTP transport (same pattern as test_nvidia_nim_rerank_ranking_endpoint above) so the request/response transformation and cost calculation stay covered without depending on NVIDIA's hosted catalog rotation. The model identifier reverts to the original llama-3.2-nv-rerankqa-1b-v2 since the request never leaves the test process.

mateo-berri · 2026-05-18T22:42:16Z

@greptileai

mateo-berri · 2026-05-18T22:46:19Z

Re: Greptile summary — the tests/llm_translation/realtime/test_realtime_guardrails_openai.py refusal markers note (sorry / can't / cannot / unable).

The test already verifies guardrail correctness before it reaches the broadened smell test:

Assertion Give me consistent exceptions #1 (lines 175–177) requires at least one guardrail_violation error event — this is the proof that the guardrail fired against the user input.
Assertion Enable model / call timeouts #2 (lines 189–191) requires a response.output_audio_transcript.delta / response.audio_transcript.delta event — this is the proof that the guardrail's own block message was forwarded as an AI transcript.
Assertion Guarantee format of exceptions #3 (lines 209–221) is a leak-check on response.done payloads, gated on if real_ai_text: — it only fires when the upstream model emitted some text despite the cancel.

The blocked input is the artificial gibberish BLOCKED_PHRASE = "XSECRETBLOCKTESTPHRASEX", which the model has no organic reason to refuse on its own. The only realistic path to real_ai_text being non-empty is the documented race where gpt-realtime has already started emitting tokens before response.cancel lands — exactly the case the comment block on lines 193–199 describes. Real prohibited content cannot match the markers (the BLOCKED_PHRASE itself doesn't contain sorry / can't / cannot / unable / blocked / guardrail), so a regression where the original content leaks would still fail this assertion. Narrowing the markers would just reintroduce flakiness on the late-cancel path without strengthening the guardrail invariant, which is already established by #1 and #2.

…itellm_fix_responses_bridge_cache_stream # Conflicts: # tests/llm_translation/realtime/test_openai_realtime.py # tests/llm_translation/realtime/test_openai_realtime_simple.py # tests/llm_translation/realtime/test_realtime_guardrails_openai.py # tests/llm_translation/test_nvidia_nim.py Co-authored-by: Mateo Wang <[email protected]>

mateo-berri · 2026-05-18T23:02:05Z

@greptileai

…to v1.89.0 (#200) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [https://github.com/BerriAI/litellm.git](https://github.com/BerriAI/litellm) | minor | `v1.85.1` → `v1.89.0` | --- > ⚠️ **Warning** > > Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/155) for more information. --- ### Release Notes <details> <summary>BerriAI/litellm (https://github.com/BerriAI/litellm.git)</summary> ### [`v1.89.0`](https://github.com/BerriAI/litellm/releases/tag/v1.89.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.2...v1.89.0) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview by [@mateo-berri](https://github.com/mateo-berri) in [#29433](https://github.com/BerriAI/litellm/pull/29433) - fix: map mistral/ministral-8b-latest in model price map by [@mateo-berri](https://github.com/mateo-berri) in [#29453](https://github.com/BerriAI/litellm/pull/29453) - fix(datadog): split oversized batches on 413 instead of re-queueing forever by [@yassin-berriai](https://github.com/yassin-berriai) in [#29444](https://github.com/BerriAI/litellm/pull/29444) - feat(otel): allowlist team\_metadata sub-keys promoted to baggage by [@yassin-berriai](https://github.com/yassin-berriai) in [#29442](https://github.com/BerriAI/litellm/pull/29442) - fix: stop use\_chat\_completions\_api flag from leaking into provider request body by [@mateo-berri](https://github.com/mateo-berri) in [#29447](https://github.com/BerriAI/litellm/pull/29447) - fix(anthropic, fireworks): inline legacy $ref defs in tool schemas by [@milan-berri](https://github.com/milan-berri) in [#28646](https://github.com/BerriAI/litellm/pull/28646) - fix(proxy): omit OpenAI \[DONE] on google-genai streamGenerateContent by [@Sameerlite](https://github.com/Sameerlite) in [#29426](https://github.com/BerriAI/litellm/pull/29426) - ci(release): create stable/X.Y.x line branch on X.Y.0 tags by [@yuneng-berri](https://github.com/yuneng-berri) in [#29457](https://github.com/BerriAI/litellm/pull/29457) - fix(vector-stores): support engines URL for Vertex AI Search by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#27885](https://github.com/BerriAI/litellm/pull/27885) - fix(ui): render caller-supplied filter options in caller order by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29462](https://github.com/BerriAI/litellm/pull/29462) - fix(batches): skip unnecessary batch input file reads by [@Sameerlite](https://github.com/Sameerlite) in [#29114](https://github.com/BerriAI/litellm/pull/29114) - docs(agents): clarify when to create new test files by [@Sameerlite](https://github.com/Sameerlite) in [#29472](https://github.com/BerriAI/litellm/pull/29472) - Litellm OSS Staging by [@Sameerlite](https://github.com/Sameerlite) in [#29161](https://github.com/BerriAI/litellm/pull/29161) - fix(mcp): clear allowed\_tools and tool overrides on MCP server edit by [@Sameerlite](https://github.com/Sameerlite) in [#29411](https://github.com/BerriAI/litellm/pull/29411) - Litellm OSS Staging 010626 by [@Sameerlite](https://github.com/Sameerlite) in [#29422](https://github.com/BerriAI/litellm/pull/29422) - fix(ci): make CircleCI rerun-failed-tests collect tests when 2+ test files fail by [@mateo-berri](https://github.com/mateo-berri) in [#29475](https://github.com/BerriAI/litellm/pull/29475) - feat(a2a): watsonx Orchestrate agent provider by [@Sameerlite](https://github.com/Sameerlite) in [#29410](https://github.com/BerriAI/litellm/pull/29410) - fix(azure\_ai): strip tool-level extra fields on 400 and retry by [@Sameerlite](https://github.com/Sameerlite) in [#29479](https://github.com/BerriAI/litellm/pull/29479) - fix(docs): remove fixed dimensions from README hero image by [@mateo-berri](https://github.com/mateo-berri) in [#29496](https://github.com/BerriAI/litellm/pull/29496) - Litellm oss staging by [@Sameerlite](https://github.com/Sameerlite) in [#29492](https://github.com/BerriAI/litellm/pull/29492) - fix: small CLAUDE.md nits by [@mateo-berri](https://github.com/mateo-berri) in [#29504](https://github.com/BerriAI/litellm/pull/29504) - Add MCP semantic conventions to otelv2 by [@yassin-berriai](https://github.com/yassin-berriai) in [#29468](https://github.com/BerriAI/litellm/pull/29468) - fix(passthrough): emit otel guardrail span when a guardrail blocks by [@yassin-berriai](https://github.com/yassin-berriai) in [#29470](https://github.com/BerriAI/litellm/pull/29470) - fix(proxy): strip NUL bytes from spend log payloads to prevent PostgreSQL 22P05 by [@milan-berri](https://github.com/milan-berri) in [#29515](https://github.com/BerriAI/litellm/pull/29515) - \[internal copy of [#28008](https://github.com/BerriAI/litellm/issues/28008)] Support MCP OAuth passthrough and issuer-scoped JWT auth by [@mateo-berri](https://github.com/mateo-berri) in [#28356](https://github.com/BerriAI/litellm/pull/28356) - feat(vector-stores): forward per-request params to Vertex AI Search by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29459](https://github.com/BerriAI/litellm/pull/29459) - feat(proxy): add per-MCP-server RPM rate limiting for keys and teams by [@Sameerlite](https://github.com/Sameerlite) in [#29482](https://github.com/BerriAI/litellm/pull/29482) - fix(tests): drop module-level test calls that break local\_testing collection by [@mateo-berri](https://github.com/mateo-berri) in [#29520](https://github.com/BerriAI/litellm/pull/29520) - feat(agents): add LangFlow agent provider with A2A session bridging by [@Sameerlite](https://github.com/Sameerlite) in [#28963](https://github.com/BerriAI/litellm/pull/28963) - fix(ui/agents): make A2A skill tags enterable and validated by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29512](https://github.com/BerriAI/litellm/pull/29512) - \[internal copy of [#29232](https://github.com/BerriAI/litellm/issues/29232)] feat: route future Claude models to Anthropic provider via pattern matching by [@mateo-berri](https://github.com/mateo-berri) in [#29239](https://github.com/BerriAI/litellm/pull/29239) - fix(tests): drop import-time completion call in test\_register\_model by [@mateo-berri](https://github.com/mateo-berri) in [#29521](https://github.com/BerriAI/litellm/pull/29521) - test: stabilize batch VCR coverage and stop live upload/network leaks by [@mateo-berri](https://github.com/mateo-berri) in [#29477](https://github.com/BerriAI/litellm/pull/29477) - \[internal copy of [#29003](https://github.com/BerriAI/litellm/issues/29003)] fix(vertex\_ai): use user-supplied api\_base as is for Model Garden OpenAI-compat path by [@mateo-berri](https://github.com/mateo-berri) in [#29530](https://github.com/BerriAI/litellm/pull/29530) - feat(proxy): native /health/drain preStop hook for graceful shutdown by [@yassin-berriai](https://github.com/yassin-berriai) in [#29439](https://github.com/BerriAI/litellm/pull/29439) - fix(auth): preserve 401 status for expired JWTs in OTel traces by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29510](https://github.com/BerriAI/litellm/pull/29510) - fix(otel): capture 401 error details in management endpoint spans by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29535](https://github.com/BerriAI/litellm/pull/29535) - test(proxy/utils): pin bottom-of-file helper behavior by [@yuneng-berri](https://github.com/yuneng-berri) in [#29509](https://github.com/BerriAI/litellm/pull/29509) - test(proxy/utils): pin PrismaClient and spend-update behavior by [@yuneng-berri](https://github.com/yuneng-berri) in [#29488](https://github.com/BerriAI/litellm/pull/29488) - test(proxy/utils): pin ProxyLogging behavior by [@yuneng-berri](https://github.com/yuneng-berri) in [#29485](https://github.com/BerriAI/litellm/pull/29485) - fix: missing span for guardrail passthrough by [@yassin-berriai](https://github.com/yassin-berriai) in [#29552](https://github.com/BerriAI/litellm/pull/29552) - fix(auth): let internal users view search tools by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29542](https://github.com/BerriAI/litellm/pull/29542) - fix: missing mcp otel attributes by [@yassin-berriai](https://github.com/yassin-berriai) in [#29554](https://github.com/BerriAI/litellm/pull/29554) - fix(proxy): resolve managed video model ids for auth by [@shivamrawat1](https://github.com/shivamrawat1) in [#29545](https://github.com/BerriAI/litellm/pull/29545) - fix(key\_generate): allow team members to create keys on org-scoped teams by [@milan-berri](https://github.com/milan-berri) in [#29310](https://github.com/BerriAI/litellm/pull/29310) - test(pass-through): move Gemini pass-through tests to gemini-3.1-flash-lite by [@mateo-berri](https://github.com/mateo-berri) in [#29595](https://github.com/BerriAI/litellm/pull/29595) - Litellm oss staging 030626 by [@Sameerlite](https://github.com/Sameerlite) in [#29578](https://github.com/BerriAI/litellm/pull/29578) - Fix : a2a bugs 030626 by [@Sameerlite](https://github.com/Sameerlite) in [#29566](https://github.com/BerriAI/litellm/pull/29566) - \[internal copy of [#29533](https://github.com/BerriAI/litellm/issues/29533)] fix(anthropic/adapter): emit thinking block for reasoning\_content-only streaming chunks by [@mateo-berri](https://github.com/mateo-berri) in [#29600](https://github.com/BerriAI/litellm/pull/29600) - ci: reproduce default-Windows wheel install to guard MAX\_PATH by [@yuneng-berri](https://github.com/yuneng-berri) in [#29597](https://github.com/BerriAI/litellm/pull/29597) - fix(vertex): strip output\_config.effort for Vertex Claude models that reject it (Haiku 4.5) by [@mateo-berri](https://github.com/mateo-berri) in [#29585](https://github.com/BerriAI/litellm/pull/29585) - Litellm websocket improvements by [@Sameerlite](https://github.com/Sameerlite) in [#29563](https://github.com/BerriAI/litellm/pull/29563) - feat(arize/phoenix): OpenInference rendering parity — tool\_calls, cost, passthrough I/O, session/user, multimodal, cache tokens by [@milan-berri](https://github.com/milan-berri) in [#28800](https://github.com/BerriAI/litellm/pull/28800) - \[internal copy of [#29550](https://github.com/BerriAI/litellm/issues/29550)] fix: passthrough endpoints duplicate logs by [@mateo-berri](https://github.com/mateo-berri) in [#29598](https://github.com/BerriAI/litellm/pull/29598) - fix(ci): keep coverage rename green when a parallel node runs no tests by [@mateo-berri](https://github.com/mateo-berri) in [#29608](https://github.com/BerriAI/litellm/pull/29608) - test(vcr): close out the remaining VCR live-call leaks by [@mateo-berri](https://github.com/mateo-berri) in [#29603](https://github.com/BerriAI/litellm/pull/29603) - fix(key\_generate): exempt UI/CLI session tokens from the budget ceiling for team keys by [@yuneng-berri](https://github.com/yuneng-berri) in [#29612](https://github.com/BerriAI/litellm/pull/29612) - fix(realtime): allow null transcripts in stream logging payloads by [@milan-berri](https://github.com/milan-berri) in [#29625](https://github.com/BerriAI/litellm/pull/29625) - build(ui): migrate eslint to flat config + bump eslint-config-next to 16 by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29626](https://github.com/BerriAI/litellm/pull/29626) - fix(key\_generate): scope session-token team-key budget exemption to caller-supplied team\_id by [@yuneng-berri](https://github.com/yuneng-berri) in [#29641](https://github.com/BerriAI/litellm/pull/29641) - fix(proxy): disable proxy buffering on streaming SSE responses by [@mateo-berri](https://github.com/mateo-berri) in [#29557](https://github.com/BerriAI/litellm/pull/29557) - fix(mcp): gate /public/mcp\_hub strictly on litellm.public\_mcp\_servers by [@michelligabriele](https://github.com/michelligabriele) in [#27764](https://github.com/BerriAI/litellm/pull/27764) - ci(ui): frontend-lint job enforcing prettier + eslint on changed files by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29633](https://github.com/BerriAI/litellm/pull/29633) - fix(gemini): googleSearch + server-side tools and googleMaps JSON schema by [@Sameerlite](https://github.com/Sameerlite) in [#29582](https://github.com/BerriAI/litellm/pull/29582) - fix(proxy): passthrough 404 when SERVER\_ROOT\_PATH is set by [@Sameerlite](https://github.com/Sameerlite) in [#29658](https://github.com/BerriAI/litellm/pull/29658) - fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility by [@Sameerlite](https://github.com/Sameerlite) in [#29662](https://github.com/BerriAI/litellm/pull/29662) - Litellm oss staging 040626 by [@Sameerlite](https://github.com/Sameerlite) in [#29671](https://github.com/BerriAI/litellm/pull/29671) - style(ui): prettier formatting pass over the dashboard by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29622](https://github.com/BerriAI/litellm/pull/29622) - chore: ignore prettier dashboard reformat in git blame by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29695](https://github.com/BerriAI/litellm/pull/29695) - fix(helm): Enable Backend Deployment to mount Gateway config.yaml by [@tin-berri](https://github.com/tin-berri) in [#29605](https://github.com/BerriAI/litellm/pull/29605) - \[internal copy of [#29277](https://github.com/BerriAI/litellm/issues/29277)] fix(proxy): add default=None to LiteLLM\_TeamMembership.litellm\_budget\_table by [@mateo-berri](https://github.com/mateo-berri) in [#29684](https://github.com/BerriAI/litellm/pull/29684) - test: make custom\_tokenizer proxy tests hermetic by [@yuneng-berri](https://github.com/yuneng-berri) in [#29643](https://github.com/BerriAI/litellm/pull/29643) - test(proxy): stop running real-DB tests in GitHub Actions unit jobs by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29700](https://github.com/BerriAI/litellm/pull/29700) - chore(ui): remove the bare-fetch lint rule by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29712](https://github.com/BerriAI/litellm/pull/29712) - Litellm jwt mapping virtualkeys by [@shivamrawat1](https://github.com/shivamrawat1) in [#28510](https://github.com/BerriAI/litellm/pull/28510) - refactor(ui): shared HTTP client + location-pinned fetch() lint rule by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29723](https://github.com/BerriAI/litellm/pull/29723) - fix(proxy): stop team BYOK model name corruption on model edit by [@yuneng-berri](https://github.com/yuneng-berri) in [#29731](https://github.com/BerriAI/litellm/pull/29731) - \[internal copy of [#29511](https://github.com/BerriAI/litellm/issues/29511)] feat(guardrails): add sensitive data routing to on-premise models by [@mateo-berri](https://github.com/mateo-berri) in [#29531](https://github.com/BerriAI/litellm/pull/29531) - fix(proxy/hooks): populate llm\_provider on internal rate-limit errors by [@mateo-berri](https://github.com/mateo-berri) in [#27707](https://github.com/BerriAI/litellm/pull/27707) - fix(vertex/anthropic): handle namespace tools and strip client\_metadata for codex compatibility by [@Sameerlite](https://github.com/Sameerlite) in [#29489](https://github.com/BerriAI/litellm/pull/29489) - Support OAuth M2M for Databricks Apps A2A agents by [@mateo-berri](https://github.com/mateo-berri) in [#29586](https://github.com/BerriAI/litellm/pull/29586) - fix: small CLAUDE.md nit by [@mateo-berri](https://github.com/mateo-berri) in [#29749](https://github.com/BerriAI/litellm/pull/29749) - fix(anthropic): route Claude Opus 4.8 through adaptive thinking by [@mateo-berri](https://github.com/mateo-berri) in [#29702](https://github.com/BerriAI/litellm/pull/29702) - fix(proxy): persist oauth2\_flow on MCP server registration by [@michelligabriele](https://github.com/michelligabriele) in [#29690](https://github.com/BerriAI/litellm/pull/29690) - \[internal copy of [#27491](https://github.com/BerriAI/litellm/issues/27491)] fix(realtime): Fix Realtime Audio Token Cost Tracking by [@mateo-berri](https://github.com/mateo-berri) in [#29722](https://github.com/BerriAI/litellm/pull/29722) - fix(galileo): use ingest traces API and standard logging payload by [@Sameerlite](https://github.com/Sameerlite) in [#29651](https://github.com/BerriAI/litellm/pull/29651) - fix(auth): expand all-team-models sentinel in can\_key\_call\_model for batch validation by [@Sameerlite](https://github.com/Sameerlite) in [#29746](https://github.com/BerriAI/litellm/pull/29746) - test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h by [@mateo-berri](https://github.com/mateo-berri) in [#29784](https://github.com/BerriAI/litellm/pull/29784) - test(ci): record/replay OpenAI image gen so the spend E2E isn't outage-bound by [@mateo-berri](https://github.com/mateo-berri) in [#29787](https://github.com/BerriAI/litellm/pull/29787) - fix(ui): route MCP playground auth by oauth2 mode instead of token\_url by [@tin-berri](https://github.com/tin-berri) in [#29714](https://github.com/BerriAI/litellm/pull/29714) - refactor(ui): centralize proxy base URL resolution into tested resolver by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29793](https://github.com/BerriAI/litellm/pull/29793) - Litellm oss staging 050626 by [@Sameerlite](https://github.com/Sameerlite) in [#29774](https://github.com/BerriAI/litellm/pull/29774) - test(google): add google-genai SDK proxy integration tests by [@Sameerlite](https://github.com/Sameerlite) in [#29781](https://github.com/BerriAI/litellm/pull/29781) - fix(jwt): use resolved DB user\_id for spend on legacy email match by [@milan-berri](https://github.com/milan-berri) in [#29217](https://github.com/BerriAI/litellm/pull/29217) - feat(ui): generate dashboard API types from the proxy OpenAPI spec by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29816](https://github.com/BerriAI/litellm/pull/29816) - fix(proxy): drop deleted team BYOK model name from team.models by [@yuneng-berri](https://github.com/yuneng-berri) in [#29820](https://github.com/BerriAI/litellm/pull/29820) - feat(mcp): per-server env vars with global + per-user scopes by [@mateo-berri](https://github.com/mateo-berri) in [#28917](https://github.com/BerriAI/litellm/pull/28917) - refactor(ui): route behavior-preserving networking calls through apiClient by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29806](https://github.com/BerriAI/litellm/pull/29806) - fix(mcp): persist Tools-tab MCP OAuth token to DB by [@tin-berri](https://github.com/tin-berri) in [#29809](https://github.com/BerriAI/litellm/pull/29809) - fix(ui): require new expiration when regenerating an expired key by [@milan-berri](https://github.com/milan-berri) in [#29838](https://github.com/BerriAI/litellm/pull/29838) - refactor(ui): route query-building networking calls through apiClient by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29815](https://github.com/BerriAI/litellm/pull/29815) - Make the image-gen record/replay proxy report cache mode and per-request HIT/MISS by [@mateo-berri](https://github.com/mateo-berri) in [#29802](https://github.com/BerriAI/litellm/pull/29802) - feat(proxy): hot-reload .env in dev when running with --reload by [@mateo-berri](https://github.com/mateo-berri) in [#29783](https://github.com/BerriAI/litellm/pull/29783) - fix(ui): stop MCP playground tool calls from sending twice by [@tin-berri](https://github.com/tin-berri) in [#29821](https://github.com/BerriAI/litellm/pull/29821) - feat(fal\_ai): add Nano Banana / Gemini 2.5 Flash Image generation support by [@mateo-berri](https://github.com/mateo-berri) in [#29798](https://github.com/BerriAI/litellm/pull/29798) - Title: Fix managed batch cancel credential resolution by [@shivamrawat1](https://github.com/shivamrawat1) in [#29734](https://github.com/BerriAI/litellm/pull/29734) - Title: fix(proxy): resolve vector store file list credentials from team deployments by [@shivamrawat1](https://github.com/shivamrawat1) in [#29739](https://github.com/BerriAI/litellm/pull/29739) - refactor: convert AWS and GCP Terraform stacks into reusable modules … by [@yassin-berriai](https://github.com/yassin-berriai) in [#28103](https://github.com/BerriAI/litellm/pull/28103) - chore(ui): build ui for release by [@yuneng-berri](https://github.com/yuneng-berri) in [#29853](https://github.com/BerriAI/litellm/pull/29853) - fix(terraform/gcp): prompt for image\_registry in DeployStack one-click by [@yassin-berriai](https://github.com/yassin-berriai) in [#29852](https://github.com/BerriAI/litellm/pull/29852) - fix(terraform/gcp): abandon SQL user on destroy by [@yassin-berriai](https://github.com/yassin-berriai) in [#29855](https://github.com/BerriAI/litellm/pull/29855) - Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic by [@mateo-berri](https://github.com/mateo-berri) in [#29847](https://github.com/BerriAI/litellm/pull/29847) - chore(deps): bump deps by [@yuneng-berri](https://github.com/yuneng-berri) in [#29860](https://github.com/BerriAI/litellm/pull/29860) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#29861](https://github.com/BerriAI/litellm/pull/29861) - fix: 400 on Anthropic context overflow; seed identity on failed auth by [@yassin-berriai](https://github.com/yassin-berriai) in [#29848](https://github.com/BerriAI/litellm/pull/29848) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#29862](https://github.com/BerriAI/litellm/pull/29862) - chore(release): patch v1.89.0-rc.1 with [#30064](https://github.com/BerriAI/litellm/issues/30064) (Claude Fable 5) for v1.89.0-rc.2 by [@mateo-berri](https://github.com/mateo-berri) in [#30143](https://github.com/BerriAI/litellm/pull/30143) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.89.0> ### [`v1.88.2`](https://github.com/BerriAI/litellm/releases/tag/v1.88.2) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.88.2 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.2/cosign.pub \ ghcr.io/berriai/litellm:v1.88.2 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - chore(release): backport Fable 5, batch-file auth, CrowdStrike AIDR, Mantle Responses SigV4, and NetApp streaming-cost fix to stable/1.88.x and cut 1.88.2 by [@mateo-berri](https://github.com/mateo-berri) in [#30144](https://github.com/BerriAI/litellm/pull/30144) - chore(release): backport DB-resilience, passthrough, model-info, budget, and deps fixes to stable/1.88.x by [@yuneng-berri](https://github.com/yuneng-berri) in [#30408](https://github.com/BerriAI/litellm/pull/30408) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2> ### [`v1.88.1`](https://github.com/BerriAI/litellm/releases/tag/v1.88.1) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.0...v1.88.1) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.88.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.1/cosign.pub \ ghcr.io/berriai/litellm:v1.88.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - build(deps): bump pyjwt to 2.13.0 and ws override to 8.20.1 (1.88.x) by [@yuneng-berri](https://github.com/yuneng-berri) in [#29987](https://github.com/BerriAI/litellm/pull/29987) - chore(release): bump version to 1.88.1 by [@yuneng-berri](https://github.com/yuneng-berri) in [#29989](https://github.com/BerriAI/litellm/pull/29989) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.88.1> ### [`v1.88.0`](https://github.com/BerriAI/litellm/releases/tag/v1.88.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.87.3...v1.88.0) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.88.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.0/cosign.pub \ ghcr.io/berriai/litellm:v1.88.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - fix(proxy): gate team allowed\_passthrough\_routes to proxy admins by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28097](https://github.com/BerriAI/litellm/pull/28097) - fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend by [@mateo-berri](https://github.com/mateo-berri) in [#28110](https://github.com/BerriAI/litellm/pull/28110) - fix(bedrock/cohere): send embedding\_types as JSON array, not string by [@ishaan-berri](https://github.com/ishaan-berri) in [#28172](https://github.com/BerriAI/litellm/pull/28172) - fix(tests): migrate realtime + rerank tests off shut-down upstream models by [@yuneng-berri](https://github.com/yuneng-berri) in [#28191](https://github.com/BerriAI/litellm/pull/28191) - fix(caching): replay openai/responses bridge cache hits as chat streams by [@Sameerlite](https://github.com/Sameerlite) in [#28158](https://github.com/BerriAI/litellm/pull/28158) - Litellm oss staging by [@Sameerlite](https://github.com/Sameerlite) in [#28161](https://github.com/BerriAI/litellm/pull/28161) - feat(prometheus): add user\_email and user\_alias to user budget metrics by [@Sameerlite](https://github.com/Sameerlite) in [#28155](https://github.com/BerriAI/litellm/pull/28155) - test(callbacks): harden flaky proxy callback-leak detector by [@yuneng-berri](https://github.com/yuneng-berri) in [#28195](https://github.com/BerriAI/litellm/pull/28195) - fix(bedrock): sanitize batch metadata to prevent Pydantic ValidationError by [@mateo-berri](https://github.com/mateo-berri) in [#28202](https://github.com/BerriAI/litellm/pull/28202) - fix(deepseek): use native /anthropic/v1/messages endpoint and sanitize tools by [@mateo-berri](https://github.com/mateo-berri) in [#28200](https://github.com/BerriAI/litellm/pull/28200) - feat(ui): add Interactions API endpoint to playground with SSE streaming by [@Sameerlite](https://github.com/Sameerlite) in [#28156](https://github.com/BerriAI/litellm/pull/28156) - fix(proxy): decode bytes and pass-through SSE for Google-native streamGenerateContent ([#27444](https://github.com/BerriAI/litellm/issues/27444)) by [@Sameerlite](https://github.com/Sameerlite) in [#28213](https://github.com/BerriAI/litellm/pull/28213) - refactor(bedrock/sagemaker): switch to lazy loading for response stre… by [@harish-berri](https://github.com/harish-berri) in [#28189](https://github.com/BerriAI/litellm/pull/28189) - \[Refactor] UI - Spend Logs: consolidate filter state and extract components by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#25847](https://github.com/BerriAI/litellm/pull/25847) - fix(tests): replace shut-down gpt-4o-audio-preview with gpt-audio-1.5 by [@yuneng-berri](https://github.com/yuneng-berri) in [#28281](https://github.com/BerriAI/litellm/pull/28281) - chore(ci): bump versions by [@yuneng-berri](https://github.com/yuneng-berri) in [#28287](https://github.com/BerriAI/litellm/pull/28287) - feat: propagate team\_id and team\_alias to all child OTEL spans by [@yassin-berriai](https://github.com/yassin-berriai) in [#28273](https://github.com/BerriAI/litellm/pull/28273) - Day 0 support : Gemini 3.5 Flash by [@Sameerlite](https://github.com/Sameerlite) in [#28268](https://github.com/BerriAI/litellm/pull/28268) - Gemini managed agents support by [@Sameerlite](https://github.com/Sameerlite) in [#28270](https://github.com/BerriAI/litellm/pull/28270) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#28292](https://github.com/BerriAI/litellm/pull/28292) - feat(gemini): add gemini-3.1-flash-lite model cost map by [@Sameerlite](https://github.com/Sameerlite) in [#28320](https://github.com/BerriAI/litellm/pull/28320) - fix(spend\_counter): seed Redis counter via SET NX to prevent cross-pod double-seed by [@milan-berri](https://github.com/milan-berri) in [#27854](https://github.com/BerriAI/litellm/pull/27854) - fix(proxy): normalize batch file IDs before ManagedObjectTable write by [@Sameerlite](https://github.com/Sameerlite) in [#28339](https://github.com/BerriAI/litellm/pull/28339) - fix(router): use forwarded model\_id for native Azure container IDs by [@Sameerlite](https://github.com/Sameerlite) in [#27921](https://github.com/BerriAI/litellm/pull/27921) - fix(ui): restore log filter loading indicator by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28282](https://github.com/BerriAI/litellm/pull/28282) - test(e2e): migrate runner to uv, add All Proxy Models key test by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28313](https://github.com/BerriAI/litellm/pull/28313) - feat(ui): team passthrough routes create parity + edit load fix by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28098](https://github.com/BerriAI/litellm/pull/28098) - fix(mcp): JWT on tools/list and REST tools/call server resolution by [@Sameerlite](https://github.com/Sameerlite) in [#28227](https://github.com/BerriAI/litellm/pull/28227) - feat(interactions): migrate to Google Interactions API steps schema (May 2026) by [@Sameerlite](https://github.com/Sameerlite) in [#28153](https://github.com/BerriAI/litellm/pull/28153) - test(ui-e2e): admin key creation with a specific proxy model by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28365](https://github.com/BerriAI/litellm/pull/28365) - fix(vertex\_ai): omit function\_call id on Vertex Gemini 3.5+ tool turns by [@Sameerlite](https://github.com/Sameerlite) in [#28324](https://github.com/BerriAI/litellm/pull/28324) - feat(mcp): allow native MCP OAuth support for cursor by [@Sameerlite](https://github.com/Sameerlite) in [#28327](https://github.com/BerriAI/litellm/pull/28327) - fix(interactions): never drop streamed text deltas; always emit terminal completion by [@mateo-berri](https://github.com/mateo-berri) in [#28394](https://github.com/BerriAI/litellm/pull/28394) - fix(proxy): expose Prisma idle/connect timeout + extra DB URL params by [@yassin-berriai](https://github.com/yassin-berriai) in [#28395](https://github.com/BerriAI/litellm/pull/28395) - Litellm oss staging 1 by [@Sameerlite](https://github.com/Sameerlite) in [#28337](https://github.com/BerriAI/litellm/pull/28337) - fix: serialize guardrail\_response to JSON in OTEL traces by [@yassin-berriai](https://github.com/yassin-berriai) in [#28362](https://github.com/BerriAI/litellm/pull/28362) - chore(ci): merge dev branch by [@yuneng-berri](https://github.com/yuneng-berri) in [#28314](https://github.com/BerriAI/litellm/pull/28314) - test(realtime): expect session.created as xAI realtime initial event by [@yuneng-berri](https://github.com/yuneng-berri) in [#28424](https://github.com/BerriAI/litellm/pull/28424) - feat(tests): behavior-pinning harness + Key Tier-1 matrix by [@yuneng-berri](https://github.com/yuneng-berri) in [#28321](https://github.com/BerriAI/litellm/pull/28321) - fix(proxy): hydrate wildcard discovery credentials ([#28284](https://github.com/BerriAI/litellm/issues/28284)) - CCI Run by [@yuneng-berri](https://github.com/yuneng-berri) in [#28419](https://github.com/BerriAI/litellm/pull/28419) - Litellm oss staging 04 21 2026 2 by [@Sameerlite](https://github.com/Sameerlite) in [#26569](https://github.com/BerriAI/litellm/pull/26569) - chore(ci): merge dev branch by [@yuneng-berri](https://github.com/yuneng-berri) in [#28290](https://github.com/BerriAI/litellm/pull/28290) - fix(vertex\_gemma): strip `context_management` from request body by [@mateo-berri](https://github.com/mateo-berri) in [#28438](https://github.com/BerriAI/litellm/pull/28438) - fix(logging): recalculate cost after router retry failures by [@milan-berri](https://github.com/milan-berri) in [#28476](https://github.com/BerriAI/litellm/pull/28476) - fix(otel): emit guardrail span on violation, surface status + categories by [@yassin-berriai](https://github.com/yassin-berriai) in [#28364](https://github.com/BerriAI/litellm/pull/28364) - test(proxy): behavior-pinning matrix for team management endpoints by [@yuneng-berri](https://github.com/yuneng-berri) in [#28441](https://github.com/BerriAI/litellm/pull/28441) - test(vertex\_ai): tolerate transient 500 in google maps grounding test by [@yuneng-berri](https://github.com/yuneng-berri) in [#28503](https://github.com/BerriAI/litellm/pull/28503) - fix(docker): restore npm to non\_root builder image by [@yuneng-berri](https://github.com/yuneng-berri) in [#28519](https://github.com/BerriAI/litellm/pull/28519) - chore(ci): bump deps by [@yuneng-berri](https://github.com/yuneng-berri) in [#28524](https://github.com/BerriAI/litellm/pull/28524) - build(deps-dev): bump black to 26.3.1 and apply formatting by [@yuneng-berri](https://github.com/yuneng-berri) in [#28525](https://github.com/BerriAI/litellm/pull/28525) - chore(deps): bump deps by [@yuneng-berri](https://github.com/yuneng-berri) in [#28528](https://github.com/BerriAI/litellm/pull/28528) - test(e2e): forward LITELLM\_LICENSE to UI e2e proxy by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28398](https://github.com/BerriAI/litellm/pull/28398) - Add granian as a ASGI compliant web server. Provider better throughput stability, by [@harish-berri](https://github.com/harish-berri) in [#26027](https://github.com/BerriAI/litellm/pull/26027) - Fix conflicts and UI by [@Sameerlite](https://github.com/Sameerlite) in [#28477](https://github.com/BerriAI/litellm/pull/28477) - Add error\_description and hint for oauth flows by [@Sameerlite](https://github.com/Sameerlite) in [#28471](https://github.com/BerriAI/litellm/pull/28471) - feat(mcp): Add tool call and tool list support via UI for Oauth mcps by [@Sameerlite](https://github.com/Sameerlite) in [#28454](https://github.com/BerriAI/litellm/pull/28454) - feat(proxy): persist allowlisted OIDC claims in CLI SSO poll by [@Sameerlite](https://github.com/Sameerlite) in [#28463](https://github.com/BerriAI/litellm/pull/28463) - fix(responses): use OpenAI SSEDecoder for Responses API streaming by [@Sameerlite](https://github.com/Sameerlite) in [#28566](https://github.com/BerriAI/litellm/pull/28566) - Litellm oss staging 2 by [@Sameerlite](https://github.com/Sameerlite) in [#28582](https://github.com/BerriAI/litellm/pull/28582) - \[internal copy of [#28269](https://github.com/BerriAI/litellm/issues/28269)] Codex cli jwt team alias by [@mateo-berri](https://github.com/mateo-berri) in [#28621](https://github.com/BerriAI/litellm/pull/28621) - fix(check\_licenses): read PEP 639 license-expression metadata by [@yuneng-berri](https://github.com/yuneng-berri) in [#28529](https://github.com/BerriAI/litellm/pull/28529) - test(proxy): behavior-pinning matrix for tier-2/3 key + team management endpoints by [@yuneng-berri](https://github.com/yuneng-berri) in [#28620](https://github.com/BerriAI/litellm/pull/28620) - chore(test): remove dead old Playwright e2e suite by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28632](https://github.com/BerriAI/litellm/pull/28632) - fix(sagemaker): send native Cohere embed payload to Cohere SageMaker endpoints by [@milan-berri](https://github.com/milan-berri) in [#28613](https://github.com/BerriAI/litellm/pull/28613) - style: apply black formatting to fix lint CI (LIT-3274) ([#28639](https://github.com/BerriAI/litellm/issues/28639)) by [@krrish-berri-2](https://github.com/krrish-berri-2) in [#28641](https://github.com/BerriAI/litellm/pull/28641) - fix(bedrock): decouple STS region from Bedrock aws\_region\_name by [@milan-berri](https://github.com/milan-berri) in [#28245](https://github.com/BerriAI/litellm/pull/28245) - test(streaming): tolerate Vertex 429 wrapped in MidStreamFallbackError by [@yuneng-berri](https://github.com/yuneng-berri) in [#28669](https://github.com/BerriAI/litellm/pull/28669) - feat(guardrails): add Microsoft Purview DLP guardrail by [@Sameerlite](https://github.com/Sameerlite) in [#24966](https://github.com/BerriAI/litellm/pull/24966) - fix(mcp): forward upstream initialize instructions on cold gateway init by [@milan-berri](https://github.com/milan-berri) in [#28231](https://github.com/BerriAI/litellm/pull/28231) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#28680](https://github.com/BerriAI/litellm/pull/28680) - CI: copy of [#25177](https://github.com/BerriAI/litellm/issues/25177) (OCI GenAI: embeddings, streaming/reasoning fixes, model catalog) by [@mateo-berri](https://github.com/mateo-berri) in [#28223](https://github.com/BerriAI/litellm/pull/28223) - Encrypt callback\_vars in key/team metadata in DB by [@Michael-RZ-Berri](https://github.com/Michael-RZ-Berri) in [#27141](https://github.com/BerriAI/litellm/pull/27141) - perf: reduce per-request and per-chunk overhead across Anthropic streaming hot paths by [@yassin-berriai](https://github.com/yassin-berriai) in [#28289](https://github.com/BerriAI/litellm/pull/28289) - feat(azure): add Speech STT config support by [@ishaan-berri](https://github.com/ishaan-berri) in [#27482](https://github.com/BerriAI/litellm/pull/27482) - test(proxy): phase-4 payload behavior pinning for tier-2/3 key + team management endpoints by [@yuneng-berri](https://github.com/yuneng-berri) in [#28681](https://github.com/BerriAI/litellm/pull/28681) - feat(prometheus): emit per-token-type detail metrics (LIT-3220) ([#28372](https://github.com/BerriAI/litellm/issues/28372)) by [@ishaan-berri](https://github.com/ishaan-berri) in [#28378](https://github.com/BerriAI/litellm/pull/28378) - fix(otel): stamp http.response.status\_code on all error responses by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28405](https://github.com/BerriAI/litellm/pull/28405) - chore(ui): build ui by [@yuneng-berri](https://github.com/yuneng-berri) in [#28707](https://github.com/BerriAI/litellm/pull/28707) - fix(helm): drop main- prefix from default image tag by [@yuneng-berri](https://github.com/yuneng-berri) in [#28710](https://github.com/BerriAI/litellm/pull/28710) - test(model\_prices): allow audio\_transcription\_config in schema by [@yuneng-berri](https://github.com/yuneng-berri) in [#28708](https://github.com/BerriAI/litellm/pull/28708) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#28709](https://github.com/BerriAI/litellm/pull/28709) - fix(team): refresh team cache on team\_model\_add/delete (LIT-3244) by [@yuneng-berri](https://github.com/yuneng-berri) in [#28683](https://github.com/BerriAI/litellm/pull/28683) - fix(ui/add-model): stop vertex\_ai-anthropic\_models from leaking into Anthropic dropdown by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28723](https://github.com/BerriAI/litellm/pull/28723) - Fix spend logs v2 route permissions by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28705](https://github.com/BerriAI/litellm/pull/28705) - fix(proxy): Bedrock Knowledge Base pass-through: preserve SigV4 headers and signed request body by [@milan-berri](https://github.com/milan-berri) in [#27526](https://github.com/BerriAI/litellm/pull/27526) - chore(tests): migrate Bedrock CI to AWS account [`9412775`](https://github.com/BerriAI/litellm/commit/941277531214) by [@mateo-berri](https://github.com/mateo-berri) in [#28728](https://github.com/BerriAI/litellm/pull/28728) - fix(otel): export SERVER span on management-endpoint success without http\_request by [@yassin-berriai](https://github.com/yassin-berriai) in [#28794](https://github.com/BerriAI/litellm/pull/28794) - chore(ci): merge dev branch by [@yuneng-berri](https://github.com/yuneng-berri) in [#28801](https://github.com/BerriAI/litellm/pull/28801) - chore(ci): merge dev branch by [@yuneng-berri](https://github.com/yuneng-berri) in [#28657](https://github.com/BerriAI/litellm/pull/28657) - fix(ui): show 2-decimal precision for max\_budget on key overview by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28809](https://github.com/BerriAI/litellm/pull/28809) - feat(proxy): allow `llm_api_routes` virtual keys to list MCP servers by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28442](https://github.com/BerriAI/litellm/pull/28442) - chore(ci): merge dev branch by [@yuneng-berri](https://github.com/yuneng-berri) in [#28807](https://github.com/BerriAI/litellm/pull/28807) - fix(team): keep team\_alias cache in sync on \_cache\_team\_object writes by [@yuneng-berri](https://github.com/yuneng-berri) in [#28737](https://github.com/BerriAI/litellm/pull/28737) - chore(ci): merge dev branch by [@yuneng-berri](https://github.com/yuneng-berri) in [#28822](https://github.com/BerriAI/litellm/pull/28822) - ci: daily oss-agent-shin canonical branch by [@ishaan-berri](https://github.com/ishaan-berri) in [#28829](https://github.com/BerriAI/litellm/pull/28829) - test(proxy): add harness for proxy\_server.py behavior-pinning by [@yuneng-berri](https://github.com/yuneng-berri) in [#28827](https://github.com/BerriAI/litellm/pull/28827) - feat(openai): apply regional-processing cost uplift for EU/US data residency by [@mateo-berri](https://github.com/mateo-berri) in [#28626](https://github.com/BerriAI/litellm/pull/28626) - chore(admin-ui): regenerate static export with trailingSlash: true by [@mateo-berri](https://github.com/mateo-berri) in [#28112](https://github.com/BerriAI/litellm/pull/28112) - fix(azure): preserve AD token refresh in v1 OpenAI client path by [@mateo-berri](https://github.com/mateo-berri) in [#28627](https://github.com/BerriAI/litellm/pull/28627) - fix(ui): route API Reference back to query-param page by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28726](https://github.com/BerriAI/litellm/pull/28726) - fix(model-edit): allow clearing custom pricing on wildcard models by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28719](https://github.com/BerriAI/litellm/pull/28719) - fix(tests/vcr): make Redis cassette cache replay deterministically (zero VCR misses on consecutive runs) by [@mateo-berri](https://github.com/mateo-berri) in [#28826](https://github.com/BerriAI/litellm/pull/28826) - fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata by [@shivamrawat1](https://github.com/shivamrawat1) in [#28425](https://github.com/BerriAI/litellm/pull/28425) - Litellm OpenAI double prefix bug by [@shivamrawat1](https://github.com/shivamrawat1) in [#28661](https://github.com/BerriAI/litellm/pull/28661) - Litellm oss staging 250526 by [@Sameerlite](https://github.com/Sameerlite) in [#28770](https://github.com/BerriAI/litellm/pull/28770) - fix(bedrock): align toolUse/toolSpec names and allow hyphens by [@Sameerlite](https://github.com/Sameerlite) in [#28874](https://github.com/BerriAI/litellm/pull/28874) - fix(realtime): send TEXT frames and valid guardrail session.update by [@Sameerlite](https://github.com/Sameerlite) in [#28848](https://github.com/BerriAI/litellm/pull/28848) - fix(mcp): extend key access-group union to MCP servers by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28890](https://github.com/BerriAI/litellm/pull/28890) - fix(galileo): support hosted v2 spans API and string output extraction by [@Sameerlite](https://github.com/Sameerlite) in [#28771](https://github.com/BerriAI/litellm/pull/28771) - fix(proxy): exclude proxy\_server\_request from its own body snapshot by [@michelligabriele](https://github.com/michelligabriele) in [#28618](https://github.com/BerriAI/litellm/pull/28618) - \[Feat] Add tool calling support for gemini and vertex ai live api by [@Sameerlite](https://github.com/Sameerlite) in [#26590](https://github.com/BerriAI/litellm/pull/26590) - refactor(ui): remove dead App Router scaffolding in (dashboard)/\* by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28891](https://github.com/BerriAI/litellm/pull/28891) - fix(docker): use system Node in componentized builders + retry apk add by [@yassin-berriai](https://github.com/yassin-berriai) in [#28888](https://github.com/BerriAI/litellm/pull/28888) - docs(agents): require consent before writing new third-party names by [@yuneng-berri](https://github.com/yuneng-berri) in [#28908](https://github.com/BerriAI/litellm/pull/28908) - refactor(ui): extract auth state into AuthContext by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28910](https://github.com/BerriAI/litellm/pull/28910) - fix(mcp): resolve team.access\_group\_ids → MCP servers by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28997](https://github.com/BerriAI/litellm/pull/28997) - test(ui): e2e cover team model edit + admin identity in navbar by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#28652](https://github.com/BerriAI/litellm/pull/28652) - test(e2e): cover add-fallback flow in Router Settings by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29069](https://github.com/BerriAI/litellm/pull/29069) - test(e2e): cover Team-BYOK add-model flow as proxy admin by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29068](https://github.com/BerriAI/litellm/pull/29068) - fix(containers): record ownership for service-account keys + fix Prisma Json serialization by [@Sameerlite](https://github.com/Sameerlite) in [#28990](https://github.com/BerriAI/litellm/pull/28990) - test(e2e): cover add-MCP-server flow via discovery → custom form by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29070](https://github.com/BerriAI/litellm/pull/29070) - test(e2e): cover AI Hub make-public flow and public model\_hub\_table by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29071](https://github.com/BerriAI/litellm/pull/29071) - \[internal copy of [#28877](https://github.com/BerriAI/litellm/issues/28877)] feat: add support for claude code goal mode for bedrock opus output config by [@mateo-berri](https://github.com/mateo-berri) in [#28898](https://github.com/BerriAI/litellm/pull/28898) - feat(guardrails): wire apply\_guardrail into proxy logging callbacks by [@Sameerlite](https://github.com/Sameerlite) in [#28970](https://github.com/BerriAI/litellm/pull/28970) - chore(ci): merge dev brach by [@yuneng-berri](https://github.com/yuneng-berri) in [#29192](https://github.com/BerriAI/litellm/pull/29192) - perf(streaming): cut per-chunk overhead \~30% on Anthropic + Bedrock hot path by [@yassin-berriai](https://github.com/yassin-berriai) in [#28720](https://github.com/BerriAI/litellm/pull/28720) - fix(proxy): enforce tag budgets for key-level tags by [@Sameerlite](https://github.com/Sameerlite) in [#29108](https://github.com/BerriAI/litellm/pull/29108) - fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit by [@Sameerlite](https://github.com/Sameerlite) in [#29098](https://github.com/BerriAI/litellm/pull/29098) - fix(datadog): drain cost-management queue + opt-in FinOps tag allowlist by [@michelligabriele](https://github.com/michelligabriele) in [#28487](https://github.com/BerriAI/litellm/pull/28487) - feat(helm): split per-component ServiceAccounts for gateway, backend, and UI by [@yassin-berriai](https://github.com/yassin-berriai) in [#28712](https://github.com/BerriAI/litellm/pull/28712) - chore(ci): bump deps ([#29208](https://github.com/BerriAI/litellm/issues/29208)) by [@yuneng-berri](https://github.com/yuneng-berri) in [#29226](https://github.com/BerriAI/litellm/pull/29226) - fix(tests/vcr): mint Google OAuth tokens live to prevent stale-token replay by [@yuneng-berri](https://github.com/yuneng-berri) in [#29229](https://github.com/BerriAI/litellm/pull/29229) - chore(cookbook): bump Go directive to 1.26.3 in gollem example by [@yuneng-berri](https://github.com/yuneng-berri) in [#29234](https://github.com/BerriAI/litellm/pull/29234) - chore(ci): bump version by [@yuneng-berri](https://github.com/yuneng-berri) in [#29242](https://github.com/BerriAI/litellm/pull/29242) - feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags by [@mateo-berri](https://github.com/mateo-berri) in [#29238](https://github.com/BerriAI/litellm/pull/29238) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#29243](https://github.com/BerriAI/litellm/pull/29243) - fix(ci): restore real Bedrock batch S3 bucket/role in oai\_misc\_config by [@mateo-berri](https://github.com/mateo-berri) in [#29245](https://github.com/BerriAI/litellm/pull/29245) - fix(guardrails): persist disable\_global\_guardrails on keys by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29233](https://github.com/BerriAI/litellm/pull/29233) - test(e2e): cover Team Admin view + member + key flows by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29072](https://github.com/BerriAI/litellm/pull/29072) - docs: hand-written CLAUDE.md; remove AGENTS.md, point GEMINI.md at it by [@mateo-berri](https://github.com/mateo-berri) in [#29252](https://github.com/BerriAI/litellm/pull/29252) - fix(teams): expose keys\_count on /v2/team/list and wire UI Resources badge by [@michelligabriele](https://github.com/michelligabriele) in [#28502](https://github.com/BerriAI/litellm/pull/28502) - fix(anthropic): stop injecting unsupported output\_config.effort=xhigh for Claude Code on Sonnet/Opus 4.6 by [@mateo-berri](https://github.com/mateo-berri) in [#29304](https://github.com/BerriAI/litellm/pull/29304) - test(e2e): cover Internal Viewer nav, key, and team-info gating by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29075](https://github.com/BerriAI/litellm/pull/29075) - test(e2e): cover Internal User key modal, team info, key page by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29074](https://github.com/BerriAI/litellm/pull/29074) - test(e2e): cover navbar Logout flow as proxy admin by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29076](https://github.com/BerriAI/litellm/pull/29076) - fix(mcp): resolve key.access\_group\_ids → MCP servers (ungated) by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29195](https://github.com/BerriAI/litellm/pull/29195) - fix(router): enforce deployment budgets for dynamically added models by [@Sameerlite](https://github.com/Sameerlite) in [#29273](https://github.com/BerriAI/litellm/pull/29273) - fix(proxy): map stripped batch body.model to proxy alias for auth by [@Sameerlite](https://github.com/Sameerlite) in [#29264](https://github.com/BerriAI/litellm/pull/29264) - feat(mcp): support stateless and stateful clients via session-id routing by [@Sameerlite](https://github.com/Sameerlite) in [#26857](https://github.com/BerriAI/litellm/pull/26857) - fix(bedrock): support tool search results + chat annotations by [@Sameerlite](https://github.com/Sameerlite) in [#29120](https://github.com/BerriAI/litellm/pull/29120) - fix(mcp): ignore stale ids on key save by [@Sameerlite](https://github.com/Sameerlite) in [#29128](https://github.com/BerriAI/litellm/pull/29128) - feat(a2a): well-known agent-card discovery + LangGraph Platform mode by [@Sameerlite](https://github.com/Sameerlite) in [#28860](https://github.com/BerriAI/litellm/pull/28860) - fix(proxy): link passthrough success spans to the SERVER root OTEL span by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29315](https://github.com/BerriAI/litellm/pull/29315) - \[internal copy of [#29089](https://github.com/BerriAI/litellm/issues/29089)] fix: duplicate claude code traces by [@mateo-berri](https://github.com/mateo-berri) in [#29311](https://github.com/BerriAI/litellm/pull/29311) - feat(otel): typed semconv-aligned OpenTelemetry instrumentation by [@yassin-berriai](https://github.com/yassin-berriai) in [#28909](https://github.com/BerriAI/litellm/pull/28909) - tests(proxy\_server): surface current behavior in tests by [@yuneng-berri](https://github.com/yuneng-berri) in [#29309](https://github.com/BerriAI/litellm/pull/29309) - test(e2e): cover Internal User create-key flow when in no teams by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29083](https://github.com/BerriAI/litellm/pull/29083) - test(e2e): assert internal-user navbar identity is scoped to that user by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29077](https://github.com/BerriAI/litellm/pull/29077) - feat(otel): add team\_metadata, http.route, and model names to inference spans by [@yassin-berriai](https://github.com/yassin-berriai) in [#29319](https://github.com/BerriAI/litellm/pull/29319) - feat(context\_management): compact\_20260112 polyfill for non-Anthropic providers by [@Sameerlite](https://github.com/Sameerlite) in [#28868](https://github.com/BerriAI/litellm/pull/28868) - feat(enterprise): add RESEND\_FROM\_EMAIL for self-hosted Resend sends by [@shivamrawat1](https://github.com/shivamrawat1) in [#28830](https://github.com/BerriAI/litellm/pull/28830) - Revert Bedrock CI back to the reactivated AWS account ([`8886022`](https://github.com/BerriAI/litellm/commit/888602223428)) by [@mateo-berri](https://github.com/mateo-berri) in [#29326](https://github.com/BerriAI/litellm/pull/29326) - fix(mcp): preserve source\_url in GET /v1/mcp/server list responses by [@shivamrawat1](https://github.com/shivamrawat1) in [#29249](https://github.com/BerriAI/litellm/pull/29249) - fix(mcp): preserve omitted fields on PUT /v1/mcp/server partial updates by [@shivamrawat1](https://github.com/shivamrawat1) in [#29253](https://github.com/BerriAI/litellm/pull/29253) - fix(ci): make litellm\_internal\_staging green (logging test + Bedrock Opus 4.7 self-heal) by [@mateo-berri](https://github.com/mateo-berri) in [#29344](https://github.com/BerriAI/litellm/pull/29344) - refactor(proxy/auth): normalize Bearer prefix in safe-hash helper by [@yuneng-berri](https://github.com/yuneng-berri) in [#29343](https://github.com/BerriAI/litellm/pull/29343) - test(reasoning-effort-grid): cover Claude Opus 4.8 across provider routes by [@mateo-berri](https://github.com/mateo-berri) in [#29327](https://github.com/BerriAI/litellm/pull/29327) - fix(guardrails): return HTTP 400 for litellm content filter blocks by [@shivamrawat1](https://github.com/shivamrawat1) in [#28418](https://github.com/BerriAI/litellm/pull/28418) - fix(proxy): restrict vector store index create/delete to proxy admins by [@shivamrawat1](https://github.com/shivamrawat1) in [#29202](https://github.com/BerriAI/litellm/pull/29202) - feat(pass\_through): extend passthrough\_managed\_object\_ids to Azure by [@Sameerlite](https://github.com/Sameerlite) in [#29160](https://github.com/BerriAI/litellm/pull/29160) - fix(proxy): enforce allowed\_passthrough\_routes for auth=true pass-thr… by [@shivamrawat1](https://github.com/shivamrawat1) in [#29256](https://github.com/BerriAI/litellm/pull/29256) - feat(mcp/auth): additive key access-group grants + opt-in member assignment by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29313](https://github.com/BerriAI/litellm/pull/29313) - fix(reset\_budget): write only {spend, budget\_reset\_at} and stop pre-zeroing counter by [@yuneng-berri](https://github.com/yuneng-berri) in [#29358](https://github.com/BerriAI/litellm/pull/29358) - test(e2e): cover PROXY\_LOGOUT\_URL redirect on Logout by [@ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#29080](https://github.com/BerriAI/litellm/pull/29080) - fix(ui): break logout redirect loop across dev and proxy origins by [@yuneng-berri](https://github.com/yuneng-berri) in [#29360](https://github.com/BerriAI/litellm/pull/29360) - fix(openai-moderation): wire streaming flags through to unified dispatcher by [@michelligabriele](https://github.com/michelligabriele) in [#27324](https://github.com/BerriAI/litellm/pull/27324) - chore(ci): build ui by [@yuneng-berri](https://github.com/yuneng-berri) in [#29366](https://github.com/BerriAI/litellm/pull/29366) - fix(v3 limiter): cap no-max\_tokens TPM floor at smallest configured limit by [@michelligabriele](https://github.com/michelligabriele) in [#28805](https://github.com/BerriAI/litellm/pull/28805) - fix(e2e): tolerate trailing slash in SERVER\_ROOT\_PATH login redirect by [@yuneng-berri](https://github.com/yuneng-berri) in [#29369](https://github.com/BerriAI/litellm/pull/29369) - chore(deps): bump deps by [@yuneng-berri](https://github.com/yuneng-berri) in [#29373](https://github.com/BerriAI/litellm/pull/29373) - chore(ci): promote internal staging to main by [@yuneng-berri](https://github.com/yuneng-berri) in [#29372](https://github.com/BerriAI/litellm/pull/29372) - chore(release): patch v1.88.0-rc.1 with four staged fixes by [@mateo-berri](https://github.com/mateo-berri) in [#29632](https://github.com/BerriAI/litellm/pull/29632) - chore(release): patch v1.88.0-rc.1 with [#29612](https://github.com/BerriAI/litellm/issues/29612) (session-token budget-ceiling exemption) by [@mateo-berri](https://github.com/mateo-berri) in [#29637](https://github.com/BerriAI/litellm/pull/29637) - fix(key\_generate): harden GHSA-q775 …

…ms (BerriAI#28158) * fix(caching): replay openai/responses bridge cache hits as chat streams When chat completions route through openai/responses, cached ModelResponse payloads under aresponses keys were deserialized as ResponsesAPIResponse (500) or re-translated as responses events (empty streaming deltas). Deserialize chat-shaped cache entries as acompletion and bypass the responses stream iterator for cached CustomStreamWrapper replay. Co-authored-by: Cursor <[email protected]> * fix(caching): map responses bridge call_type for sync vs async stream replay Co-authored-by: Yassin Kortam <[email protected]> * fix: handle ModelResponse cache return in responses bridge and drop dead acompletion check Co-authored-by: Yassin Kortam <[email protected]> * fix(caching): detect chat cache hits via object field before choices fallback Prefer chat.completion object type over the broad choices-key heuristic so Responses API cached payloads are not misclassified if their schema changes. Co-authored-by: Cursor <[email protected]> * test(caching): cover responses bridge cache-hit paths in CI-tracked test suite The new bridge cache replay logic in caching_handler.py and the preformatted-stream guard in litellm_responses_transformation/handler.py were exercised only by tests under tests/local_testing/, which the responses-caching-types and misc shards do not run. Codecov flagged the patch as 29.72% covered. Add equivalent unit tests under tests/test_litellm/ so the responses, caching, types, and misc shards execute them and ship their coverage data to Codecov: - _is_chat_completion_cached_dict happy/sad paths - aresponses streaming bridge cache hit -> CustomStreamWrapper - responses non-streaming bridge cache hit -> ModelResponse - legacy ResponsesAPIResponse stream + non-stream replay - _is_preformatted_cached_chat_stream true/false - completion/acompletion early return on cached ModelResponse - completion/acompletion skip rewrap on preformatted cached stream * fix: add negative guard on object field in _is_chat_completion_cached_dict Co-authored-by: Yassin Kortam <[email protected]> * fix(vcr): treat corrupt cassette payloads as cache miss * test: bump EOL'd NVIDIA rerank and OpenAI realtime models in CI The NVIDIA hosted rerank endpoint for nvidia/llama-3_2-nv-rerankqa-1b-v2 reached end-of-life on 2026-05-18 and now returns HTTP 410 Gone, breaking TestNvidiaNim::test_basic_rerank. Switch to nvidia/nv-rerankqa-mistral-4b-v3, which is still hosted on the NVIDIA API catalog and is already listed in model_prices_and_context_window.json. OpenAI also retired the gpt-4o-realtime-preview-2024-12-17 model used by test_realtime_guardrails_openai (now returns model_not_found). Switch the realtime test URL to the GA gpt-realtime alias. Unrelated to the responses-bridge cache fix in this PR, but committing here to unblock CI per maintainer guidance. Co-authored-by: Mateo Wang <[email protected]> * test(realtime): switch retired gpt-4o-realtime-preview to gpt-realtime OpenAI removed gpt-4o-realtime-preview and all its date snapshots on 2026-05-18 (every variant now returns model_not_found), breaking the live-WebSocket OpenAI realtime tests in CI: - test_openai_realtime_direct_call_no_intent - test_openai_realtime_direct_call_with_intent - TestOpenAIRealtime.test_realtime_connection - TestOpenAIRealtime.test_realtime_with_query_params Point each of those to the current GA alias gpt-realtime (verified live). Pure unit/mock tests that just assert the string value (e.g. in test_realtime_query_params_construction and the test_realtime_query_params_use_normalized_model_name mock) are left alone since they do not depend on model availability. Also relax the AI-response assertion in test_text_message_blocked_by_guardrail_no_ai_response: gpt-realtime occasionally produces a polite refusal ("I'm sorry, but I can't say that") when the cancel arrives after the model has already started generating, which is the expected outcome (no real AI content) but does not contain the words 'blocked' or 'guardrail'. The primary guardrail behaviour (guardrail_violation error event + transcript_delta block message) is still asserted unchanged. Co-authored-by: Mateo Wang <[email protected]> * test(nvidia_nim): mock rerank live API instead of hitting EOL'd endpoint NVIDIA reached end-of-life for the hosted nvidia/llama-3.2-nv-rerankqa-1b-v2 rerank API on 2026-05-18 (returns HTTP 410 Gone), and the proposed replacement nv-rerankqa-mistral-4b-v3 returns HTTP 404 for the CI account, breaking TestNvidiaNim::test_basic_rerank. Override test_basic_rerank to mock the HTTP transport (same pattern as test_nvidia_nim_rerank_ranking_endpoint above) so the request/response transformation and cost calculation stay covered without depending on NVIDIA's hosted catalog rotation. The model identifier reverts to the original llama-3.2-nv-rerankqa-1b-v2 since the request never leaves the test process. --------- Co-authored-by: Cursor <[email protected]> Co-authored-by: Yassin Kortam <[email protected]> Co-authored-by: mateo-berri <[email protected]> Co-authored-by: Mateo Wang <[email protected]>

greptile-apps Bot reviewed May 18, 2026

View reviewed changes

Comment thread litellm/caching/caching_handler.py

cursor Bot reviewed May 18, 2026

View reviewed changes

Comment thread litellm/caching/caching_handler.py

fix(caching): map responses bridge call_type for sync vs async stream…

9e32844

… replay Co-authored-by: Yassin Kortam <[email protected]>

cursor Bot reviewed May 18, 2026

View reviewed changes

Comment thread litellm/caching/caching_handler.py

Comment thread litellm/caching/caching_handler.py Outdated

cursoragent and others added 2 commits May 18, 2026 12:16

fix: handle ModelResponse cache return in responses bridge and drop d…

bda3ee6

…ead acompletion check Co-authored-by: Yassin Kortam <[email protected]>

fix(caching): detect chat cache hits via object field before choices …

fb8bee7

…fallback Prefer chat.completion object type over the broad choices-key heuristic so Responses API cached payloads are not misclassified if their schema changes. Co-authored-by: Cursor <[email protected]>

cursor Bot reviewed May 18, 2026

View reviewed changes

Comment thread litellm/caching/caching_handler.py

fix: add negative guard on object field in _is_chat_completion_cached…

8f86739

…_dict Co-authored-by: Yassin Kortam <[email protected]>

fix(vcr): treat corrupt cassette payloads as cache miss

ee892dc

mateo-berri approved these changes May 18, 2026

View reviewed changes

mateo-berri merged commit 477b63c into litellm_internal_staging May 18, 2026
124 of 126 checks passed

mateo-berri deleted the litellm_fix_responses_bridge_cache_stream branch May 18, 2026 23:27

Uh oh!

Uh oh!

Conversation

Sameerlite commented May 18, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

codecov Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

greptile-apps Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

CLAassistant commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Sameerlite commented May 18, 2026

Uh oh!

mateo-berri commented May 18, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mateo-berri commented May 18, 2026

Uh oh!

mateo-berri commented May 18, 2026

Uh oh!

mateo-berri left a comment

Choose a reason for hiding this comment

Uh oh!

mateo-berri commented May 18, 2026

Uh oh!

mateo-berri commented May 18, 2026

Uh oh!

mateo-berri commented May 18, 2026

Uh oh!

mateo-berri commented May 18, 2026

Uh oh!

mateo-berri commented May 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Sameerlite commented May 18, 2026 •

edited by cursor Bot

Loading

codecov Bot commented May 18, 2026 •

edited

Loading

greptile-apps Bot commented May 18, 2026 •

edited

Loading

CLAassistant commented May 18, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading