fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit#29098
Merged
mateo-berri merged 8 commits intoMay 28, 2026
Merged
Conversation
…handlers and implement video edit for Veo - Pass litellm_params to validate_environment in 11 video handler call sites (remix, create_character, get_character, edit, extension, delete) so DB-stored Vertex AI credentials are used instead of falling back to ADC - Implement transform_video_edit_request/response for VertexAI: fetches source video via fetchPredictOperation then submits a new predictLongRunning request with the video bytes/gcsUri + edit prompt Co-authored-by: Cursor <[email protected]>
Contributor
Greptile SummaryThis PR fixes two bugs in Vertex AI video endpoints: it passes
Confidence Score: 4/5Safe to merge; the credential fix is straightforward and the Vertex edit implementation is well-structured, though a minor exception-handling gap in the prefetch path leaves one narrow failure mode untranslated. The overall change is well-reasoned and the previous review concerns (blocking sync client in async path, extra_body mutation, try/except coverage) are all addressed. The remaining gap is that prefetch_resp.json() sits just outside the try/except in both the sync and async handlers, so a malformed-but-2xx prefetch response would surface as a raw JSONDecodeError instead of a typed LiteLLM exception — a narrow but genuine inconsistency with the rest of the handler. The sync and async prefetch blocks in
|
| Filename | Overview |
|---|---|
| litellm/llms/custom_httpx/llm_http_handler.py | Adds litellm_params to 11 validate_environment calls (credential-fix) and restructures both sync/async video-edit handlers to use a shared httpx client for the prefetch step; prefetch_resp.json() sits just outside the try/except in both paths. |
| litellm/llms/vertex_ai/videos/transformation.py | Implements get_video_edit_prefetch_params, transform_video_edit_request, and transform_video_edit_response for Vertex AI Veo; extracts _build_vertex_video_usage_from_request_data as a shared helper; logic is clean. |
| litellm/llms/base_llm/videos/transformation.py | Adds get_video_edit_prefetch_params hook (returns None by default) and updates signatures of transform_video_edit_request/transform_video_edit_response to accept the new optional parameters. |
| litellm/cost_calculator.py | Adds video_edit and avideo_edit call types to _VIDEO_CALL_TYPES, routing cost calculation for edit operations through the same video-billing path as generation. |
| tests/test_litellm/llms/vertex_ai/videos/test_vertex_video_transformation.py | Adds 6 unit tests for the new Vertex AI edit flow using mocks only; no real network calls; covers bytes, GCS URI, not-done error, response shape, and cost usage. |
Reviews (6): Last reviewed commit: "fix(video_edit): route transform errors ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
…ocking event loop - Add get_video_edit_prefetch_params() to BaseVideoConfig (returns None) - VertexAI overrides it to return the fetchPredictOperation URL/body - Both sync and async video_edit handlers call this and use their shared httpx client for the fetch, passing the result as prefetched_source_data - transform_video_edit_request is now a pure transform with no HTTP calls - Fix extra_body.pop() mutation by working on a shallow copy Co-authored-by: Cursor <[email protected]>
Collaborator
Author
… block Co-authored-by: Cursor <[email protected]>
Collaborator
Author
…edit_request overrides Co-authored-by: Cursor <[email protected]>
Collaborator
Author
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
…rors propagate Move transform_video_edit_request and logging_obj.pre_call outside the try/except that wraps HTTP calls in (async_)video_edit_handler so that ValueError validation errors (e.g. 'source video not complete yet') are not silently wrapped as 500s by _handle_error. The prefetch HTTP call keeps its own try/except so its errors are still mapped through the provider's error handler. Matches the pattern used by video_extension_handler and video_remix_handler. Co-authored-by: Yassin Kortam <[email protected]>
|
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: New prefetch method duplicates existing status retrieve logic
- Refactored get_video_edit_prefetch_params to delegate to transform_video_status_retrieve_request, removing the duplicated URL/body construction logic.
Preview (7543fe0c03)
diff --git a/litellm/llms/base_llm/videos/transformation.py b/litellm/llms/base_llm/videos/transformation.py
--- a/litellm/llms/base_llm/videos/transformation.py
+++ b/litellm/llms/base_llm/videos/transformation.py
@@ -321,6 +321,23 @@
"video get character is not supported for this provider"
)
+ def get_video_edit_prefetch_params(
+ self,
+ video_id: str,
+ api_base: str,
+ litellm_params: GenericLiteLLMParams,
+ headers: dict,
+ ) -> Optional[Tuple[str, Dict]]:
+ """
+ Return (url, body) for a pre-fetch HTTP call that must be made before
+ transform_video_edit_request, or None if no pre-fetch is required.
+
+ Providers that need to retrieve the source video before constructing the
+ edit request (e.g. Vertex AI) should override this method. The handler
+ uses the existing shared httpx client so the call is properly async.
+ """
+ return None
+
def transform_video_edit_request(
self,
prompt: str,
@@ -329,6 +346,7 @@
litellm_params: GenericLiteLLMParams,
headers: dict,
extra_body: Optional[Dict[str, Any]] = None,
+ prefetched_source_data: Optional[Dict[str, Any]] = None,
) -> Tuple[str, Dict]:
"""
Transform the video edit request into a URL and JSON data.
diff --git a/litellm/llms/custom_httpx/llm_http_handler.py b/litellm/llms/custom_httpx/llm_http_handler.py
--- a/litellm/llms/custom_httpx/llm_http_handler.py
+++ b/litellm/llms/custom_httpx/llm_http_handler.py
@@ -6560,6 +6560,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
@@ -6642,6 +6643,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
@@ -6734,6 +6736,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
headers.update(extra_headers)
@@ -6805,6 +6808,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
headers.update(extra_headers)
@@ -6888,6 +6892,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
headers.update(extra_headers)
@@ -6945,6 +6950,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
headers.update(extra_headers)
@@ -7021,6 +7027,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
headers.update(extra_headers)
@@ -7031,6 +7038,27 @@
litellm_params=dict(litellm_params),
)
+ prefetched_source_data = None
+ prefetch_params = video_provider_config.get_video_edit_prefetch_params(
+ video_id=video_id,
+ api_base=api_base,
+ litellm_params=litellm_params,
+ headers=headers,
+ )
+ if prefetch_params is not None:
+ prefetch_url, prefetch_body = prefetch_params
+ try:
+ prefetch_resp = sync_httpx_client.post(
+ url=prefetch_url,
+ headers=headers,
+ json=prefetch_body,
+ timeout=timeout,
+ )
+ prefetch_resp.raise_for_status()
+ except Exception as e:
+ raise self._handle_error(e=e, provider_config=video_provider_config)
+ prefetched_source_data = prefetch_resp.json()
+
url, data = video_provider_config.transform_video_edit_request(
prompt=prompt,
video_id=video_id,
@@ -7038,6 +7066,7 @@
litellm_params=litellm_params,
headers=headers,
extra_body=extra_body,
+ prefetched_source_data=prefetched_source_data,
)
logging_obj.pre_call(
@@ -7093,6 +7122,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
headers.update(extra_headers)
@@ -7103,6 +7133,27 @@
litellm_params=dict(litellm_params),
)
+ prefetched_source_data = None
+ prefetch_params = video_provider_config.get_video_edit_prefetch_params(
+ video_id=video_id,
+ api_base=api_base,
+ litellm_params=litellm_params,
+ headers=headers,
+ )
+ if prefetch_params is not None:
+ prefetch_url, prefetch_body = prefetch_params
+ try:
+ prefetch_resp = await async_httpx_client.post(
+ url=prefetch_url,
+ headers=headers,
+ json=prefetch_body,
+ timeout=timeout,
+ )
+ prefetch_resp.raise_for_status()
+ except Exception as e:
+ raise self._handle_error(e=e, provider_config=video_provider_config)
+ prefetched_source_data = prefetch_resp.json()
+
url, data = video_provider_config.transform_video_edit_request(
prompt=prompt,
video_id=video_id,
@@ -7110,6 +7161,7 @@
litellm_params=litellm_params,
headers=headers,
extra_body=extra_body,
+ prefetched_source_data=prefetched_source_data,
)
logging_obj.pre_call(
@@ -7182,6 +7234,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
headers.update(extra_headers)
@@ -7256,6 +7309,7 @@
api_key=api_key or litellm_params.get("api_key", None),
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
headers.update(extra_headers)
@@ -7467,6 +7521,7 @@
api_key=api_key,
headers=extra_headers or {},
model="",
+ litellm_params=litellm_params,
)
if extra_headers:
diff --git a/litellm/llms/gemini/videos/transformation.py b/litellm/llms/gemini/videos/transformation.py
--- a/litellm/llms/gemini/videos/transformation.py
+++ b/litellm/llms/gemini/videos/transformation.py
@@ -581,7 +581,14 @@
raise NotImplementedError("video get character is not supported for Gemini")
def transform_video_edit_request(
- self, prompt, video_id, api_base, litellm_params, headers, extra_body=None
+ self,
+ prompt,
+ video_id,
+ api_base,
+ litellm_params,
+ headers,
+ extra_body=None,
+ prefetched_source_data=None,
):
raise NotImplementedError("video edit is not supported for Gemini")
diff --git a/litellm/llms/openai/videos/transformation.py b/litellm/llms/openai/videos/transformation.py
--- a/litellm/llms/openai/videos/transformation.py
+++ b/litellm/llms/openai/videos/transformation.py
@@ -534,6 +534,7 @@
litellm_params: GenericLiteLLMParams,
headers: dict,
extra_body: Optional[Dict[str, Any]] = None,
+ prefetched_source_data: Optional[Dict[str, Any]] = None,
) -> Tuple[str, Dict]:
original_video_id = extract_original_video_id(video_id)
url = f"{api_base.rstrip('/')}/edits"
diff --git a/litellm/llms/runwayml/videos/transformation.py b/litellm/llms/runwayml/videos/transformation.py
--- a/litellm/llms/runwayml/videos/transformation.py
+++ b/litellm/llms/runwayml/videos/transformation.py
@@ -623,7 +623,14 @@
raise NotImplementedError("video get character is not supported for RunwayML")
def transform_video_edit_request(
- self, prompt, video_id, api_base, litellm_params, headers, extra_body=None
+ self,
+ prompt,
+ video_id,
+ api_base,
+ litellm_params,
+ headers,
+ extra_body=None,
+ prefetched_source_data=None,
):
raise NotImplementedError("video edit is not supported for RunwayML")
diff --git a/litellm/llms/vertex_ai/videos/transformation.py b/litellm/llms/vertex_ai/videos/transformation.py
--- a/litellm/llms/vertex_ai/videos/transformation.py
+++ b/litellm/llms/vertex_ai/videos/transformation.py
@@ -647,16 +647,118 @@
def transform_video_get_character_response(self, raw_response, logging_obj):
raise NotImplementedError("video get character is not supported for Vertex AI")
+ def get_video_edit_prefetch_params(
+ self,
+ video_id: str,
+ api_base: str,
+ litellm_params: GenericLiteLLMParams,
+ headers: dict,
+ ) -> Tuple[str, Dict]:
+ """Return the fetchPredictOperation URL and body needed to retrieve the source video."""
+ return self.transform_video_status_retrieve_request(
+ video_id=video_id,
+ api_base=api_base,
+ litellm_params=litellm_params,
+ headers=headers,
+ )
+
def transform_video_edit_request(
- self, prompt, video_id, api_base, litellm_params, headers, extra_body=None
- ):
- raise NotImplementedError("video edit is not supported for Vertex AI")
+ self,
+ prompt: str,
+ video_id: str,
+ api_base: str,
+ litellm_params: GenericLiteLLMParams,
+ headers: dict,
+ extra_body: Optional[Dict[str, Any]] = None,
+ prefetched_source_data: Optional[Dict[str, Any]] = None,
+ ) -> Tuple[str, Dict]:
+ """
+ Build a predictLongRunning edit request from the pre-fetched source video.
+ The actual fetchPredictOperation HTTP call is hoisted into the handler so
+ it can use the shared async/sync httpx client instead of blocking the loop.
+ """
+ if prefetched_source_data is None:
+ raise ValueError(
+ "prefetched_source_data is required for Vertex AI video edit. "
+ "Ensure get_video_edit_prefetch_params is called by the handler."
+ )
+
+ if not prefetched_source_data.get("done", False):
+ raise ValueError(
+ "Source video generation is not complete yet. "
+ "Check the video status before editing."
+ )
+
+ videos = prefetched_source_data.get("response", {}).get("videos", [])
+ if not videos:
+ raise ValueError("No videos found in the completed operation. Cannot edit.")
+
+ source_video = videos[0]
+ video_input: Dict[str, Any] = {}
+ if "gcsUri" in source_video:
+ video_input["gcsUri"] = source_video["gcsUri"]
+ elif "bytesBase64Encoded" in source_video:
+ video_input["bytesBase64Encoded"] = source_video["bytesBase64Encoded"]
+ video_input["mimeType"] = source_video.get("mimeType", "video/mp4")
+ else:
+ raise ValueError(
+ "Source video has neither gcsUri nor bytesBase64Encoded. Cannot edit."
+ )
+
+ operation_name = extract_original_video_id(video_id)
+ model = self.extract_model_from_operation_name(operation_name) or ""
+
+ instance_dict: Dict[str, Any] = {"prompt": prompt, "video": video_input}
+ request_data: Dict[str, Any] = {"instances": [instance_dict]}
+
+ if extra_body:
+ extra_body_copy = dict(extra_body)
+ nested_params = extra_body_copy.pop("parameters", None)
+ vertex_params: Dict[str, Any] = {}
+ if isinstance(nested_params, dict):
+ vertex_params.update(nested_params)
+ vertex_params.update(extra_body_copy)
+ if vertex_params:
+ request_data["parameters"] = vertex_params
+
+ edit_url = f"{api_base.rstrip('/')}/{model}:predictLongRunning"
+ return edit_url, request_data
+
def transform_video_edit_response(
- self, raw_response, logging_obj, custom_llm_provider=None
- ):
- raise NotImplementedError("video edit is not supported for Vertex AI")
+ self,
+ raw_response: httpx.Response,
+ logging_obj: LiteLLMLoggingObj,
+ custom_llm_provider: Optional[str] = None,
+ ) -> VideoObject:
+ """
+ Transform the Veo video edit response.
+ Veo returns the same operation response as video generation:
+ {"name": "projects/.../operations/OPERATION_ID"}
+ """
+ response_data = raw_response.json()
+
+ operation_name = response_data.get("name")
+ if not operation_name:
+ raise ValueError(f"No operation name in Veo edit response: {response_data}")
+
+ model = self.extract_model_from_operation_name(operation_name) or ""
+
+ if custom_llm_provider:
+ video_id = encode_video_id_with_provider(
+ operation_name, custom_llm_provider, model
+ )
+ else:
+ video_id = operation_name
+
+ return VideoObject(
+ id=video_id,
+ object="video",
+ status="processing",
+ model=model,
+ )
+
def transform_video_extension_request(
self,
prompt,
diff --git a/tests/test_litellm/llms/vertex_ai/videos/test_vertex_video_transformation.py b/tests/test_litellm/llms/vertex_ai/videos/test_vertex_video_transformation.py
--- a/tests/test_litellm/llms/vertex_ai/videos/test_vertex_video_transformation.py
+++ b/tests/test_litellm/llms/vertex_ai/videos/test_vertex_video_transformation.py
@@ -456,6 +456,106 @@
raw_response=mock_response, logging_obj=self.mock_logging_obj
)
+ def test_get_video_edit_prefetch_params(self):
+ """Test that prefetch params returns the fetchPredictOperation URL and body."""
+ operation_name = "projects/test-project/locations/us-central1/publishers/google/models/veo-3.1-generate-001/operations/op-123"
+ api_base = "https://us-central1-aiplatform.googleapis.com/v1/projects/test-project/locations/us-central1/publishers/google/models"
+
+ fetch_url, fetch_body = self.config.get_video_edit_prefetch_params(
+ video_id=operation_name,
+ api_base=api_base,
+ litellm_params=GenericLiteLLMParams(),
+ headers={},
+ )
+
+ assert "fetchPredictOperation" in fetch_url
+ assert "veo-3.1-generate-001" in fetch_url
+ assert fetch_body == {"operationName": operation_name}
+
+ def test_transform_video_edit_request_with_bytes(self):
+ """Test video edit request builds predictLongRunning body from pre-fetched bytes."""
+ operation_name = "projects/test-project/locations/us-central1/publishers/google/models/veo-3.1-generate-001/operations/op-123"
+ api_base = "https://us-central1-aiplatform.googleapis.com/v1/projects/test-project/locations/us-central1/publishers/google/models"
+ fake_bytes = base64.b64encode(b"fake_video").decode()
+
+ prefetched = {
+ "done": True,
+ "response": {
+ "videos": [{"bytesBase64Encoded": fake_bytes, "mimeType": "video/mp4"}]
+ },
+ }
+
+ url, data = self.config.transform_video_edit_request(
+ prompt="Make it brighter",
+ video_id=operation_name,
+ api_base=api_base,
+ litellm_params=GenericLiteLLMParams(),
+ headers={"Authorization": "Bearer token"},
+ prefetched_source_data=prefetched,
+ )
+
+ assert url.endswith(":predictLongRunning")
+ assert "veo-3.1-generate-001" in url
+ instance = data["instances"][0]
+ assert instance["prompt"] == "Make it brighter"
+ assert instance["video"]["bytesBase64Encoded"] == fake_bytes
+ assert instance["video"]["mimeType"] == "video/mp4"
+
+ def test_transform_video_edit_request_with_gcs_uri(self):
+ """Test that gcsUri is used when present in source video."""
+ operation_name = "projects/test-project/locations/us-central1/publishers/google/models/veo-3.1-generate-001/operations/op-456"
+ api_base = "https://us-central1-aiplatform.googleapis.com/v1/projects/test-project/locations/us-central1/publishers/google/models"
+
+ prefetched = {
+ "done": True,
+ "response": {
+ "videos": [{"gcsUri": "gs://bucket/video.mp4", "mimeType": "video/mp4"}]
+ },
+ }
+
+ _, data = self.config.transform_video_edit_request(
+ prompt="Make it darker",
+ video_id=operation_name,
+ api_base=api_base,
+ litellm_params=GenericLiteLLMParams(),
+ headers={},
+ prefetched_source_data=prefetched,
+ )
+
+ assert data["instances"][0]["video"] == {"gcsUri": "gs://bucket/video.mp4"}
+
+ def test_transform_video_edit_request_source_not_done_raises(self):
+ """Test that editing an in-progress video raises a clear error."""
+ operation_name = "projects/test-project/locations/us-central1/publishers/google/models/veo-3.1-generate-001/operations/op-789"
+ api_base = "https://us-central1-aiplatform.googleapis.com/v1/projects/test-project/locations/us-central1/publishers/google/models"
+
+ with pytest.raises(ValueError, match="not complete yet"):
+ self.config.transform_video_edit_request(
+ prompt="Make it brighter",
+ video_id=operation_name,
+ api_base=api_base,
+ litellm_params=GenericLiteLLMParams(),
+ headers={},
+ prefetched_source_data={"done": False},
+ )
+
+ def test_transform_video_edit_response(self):
+ """Test that edit response returns a processing VideoObject with encoded ID."""
+ operation_name = "projects/test-project/locations/us-central1/publishers/google/models/veo-3.1-generate-001/operations/new-op-123"
+ mock_response = Mock(spec=httpx.Response)
+ mock_response.json.return_value = {"name": operation_name}
+
+ video_obj = self.config.transform_video_edit_response(
+ raw_response=mock_response,
+ logging_obj=self.mock_logging_obj,
+ custom_llm_provider="vertex_ai",
+ )
+
+ assert isinstance(video_obj, VideoObject)
+ assert video_obj.status == "processing"
+ assert video_obj.id
+ assert video_obj.model == "veo-3.1-generate-001"
+
def test_transform_video_remix_request_not_supported(self):
"""Test that video remix raises NotImplementedError."""
with pytest.raises(NotImplementedError, match="Video remix is not supported"):You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 77c6257. Configure here.
…s retrieve Co-authored-by: Yassin Kortam <[email protected]>
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
Contributor
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 2 · PR risk: 0/10 |
Collaborator
Author
Wrap transform_video_edit_request and pre_call in the same try/except as the HTTP call in sync and async handlers so validation failures (e.g. source video not complete) return typed LiteLLM exceptions. Co-authored-by: Cursor <[email protected]>
Collaborator
Author
mateo-berri
merged commit 69afcd0
into
litellm_internal_staging
103 of 118 checks passed
shudonglin
added a commit
to rayward-external/litellm
that referenced
this pull request
Jun 1, 2026
* feat: add support for claude code goal mode for bedrock opus output config (BerriAI#28898) * feat: support goal mode for claude on bedrock * fix failing lint test * addressing greptile comments * fixing failed test * address greptile: copy output_config and warn on dropped converse format * fix(bedrock): skip redundant output_config normalization on Converse reasoning_effort path When reasoning_effort is mapped via _handle_reasoning_effort_parameter, the resulting output_config is already normalized via normalize_bedrock_opus_output_config_effort. Mark it as normalized so _prepare_request_params can skip the redundant call (and the associated get_model_info lookup) on every request. Co-authored-by: Yassin Kortam <[email protected]> * test(reasoning-effort-grid): reflect Bedrock opus-4-6 xhigh→max clamping * fix(bedrock): stop leaking output_config marker and message-content mutation * fix(bedrock): guard effort key access in normalize_bedrock_opus_output_config_effort Defensively check that 'effort' is a valid key in _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER before indexing, to prevent a KeyError if the hardcoded guard tuple ever drifts from the order dict's keys. Co-authored-by: Yassin Kortam <[email protected]> * fix(bedrock): drop dead second clause in effort normalization guard The 'effort not in _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER' check is unreachable once 'effort not in ("xhigh", "max")' has been ruled out, since both literals are present in the order dict. Keep the literal membership check and let the dict lookups below speak for themselves. * fix(bedrock): clamp output_config.effort against ceiling for any known value The early return when effort was not 'xhigh'/'max' meant a ceiling of 'low' or 'medium' would silently forward an out-of-range value. Gate on the known effort ordering instead so the ceiling comparison runs for every recognized effort. * test(grid_spec): use _CAPS_OPUS_4_7 for non-Bedrock opus-4-6 entries claude-opus-4-6 now declares supports_xhigh_reasoning_effort in the model map, so production accepts xhigh on Azure AI and Vertex AI routes. Update those grid_spec entries to match production capabilities so expected() predicts 200 for xhigh instead of 400. Co-authored-by: Yassin Kortam <[email protected]> * test(grid_spec): revert xhigh caps for non-Bedrock opus-4-6 azure_ai/claude-opus-4-6 and vertex_ai/claude-opus-4-6 do not declare supports_xhigh_reasoning_effort in model_prices_and_context_window.json. Azure AI upstream rejects xhigh with HTTP 400 ("Supported levels: high, low, max, medium"). Restore _CAPS_4_6 so the grid predicts 400 for xhigh, matching production capabilities. * fix: stop advertising xhigh effort on Opus 4.5/4.6 Only Opus 4.7 supports the xhigh reasoning effort level. Remove the supports_xhigh_reasoning_effort flag from every Opus 4.5 and Opus 4.6 entry (direct Anthropic, Bedrock, and regional variants) in both model catalog files. On the direct Anthropic path there is no effort clamp, so flagging 4.5/4.6 as xhigh-capable caused litellm to forward xhigh to a model that rejects it (and made get_model_info misreport the capability). xhigh now correctly degrades to high / raises on those models. Bedrock graceful degradation for Claude Code goal mode is unaffected: it relies solely on the bedrock_output_config_effort_ceiling clamp (4.5->high, 4.6->max, 4.7->xhigh), which runs before validation, so xhigh requests to older Bedrock Opus models are still silently lowered rather than rejected. Update effort-gating tests to reflect that 4.5/4.6 no longer accept xhigh. * fix: clamp xhigh effort on Bedrock Invoke /v1/messages instead of rejecting Claude Code "goal mode" sends output_config.effort=xhigh over the Anthropic /v1/messages API, which routes Bedrock models through AmazonAnthropicClaudeMessagesConfig. That path validated effort against the model's native capability and raised 400 for xhigh on Opus 4.6, while the chat-completions paths (Converse + Invoke) already clamp xhigh to the model's bedrock_output_config_effort_ceiling. That asymmetry broke goal mode on the exact API surface Claude Code uses. Apply the same ceiling clamp on the messages path before the shared effort gate runs, so xhigh degrades to max on Opus 4.6 (and stays xhigh on 4.7). Scoped to adaptive-thinking models and to models that declare a ceiling, so Sonnet 4.6 (no ceiling) and Opus 4.5 (budget mode) are unaffected and still reject xhigh. * fix(bedrock): preserve user output_config when applying reasoning_effort - Converse path: merge mapped effort into existing output_config via setdefault instead of overwriting it, matching the Anthropic Messages path. Prevents user-supplied output_config.format from being silently dropped when reasoning_effort is also provided. - tests: clear _get_local_model_cost_map lru_cache in the autouse fixture alongside get_bedrock_response_stream_shape to avoid stale cache leakage between tests. Co-authored-by: Yassin Kortam <[email protected]> * fix(bedrock): pre-clamp reasoning_effort for chat invoke; correct test caps - Add _clamp_adaptive_reasoning_effort_for_bedrock to AmazonAnthropicClaudeConfig so raw reasoning_effort=xhigh degrades to the model's bedrock effort ceiling before AnthropicConfig.map_openai_params converts it to output_config. Mirrors converse path (_handle_reasoning_effort_parameter) and messages path (_clamp_adaptive_reasoning_effort_for_bedrock) so the three Bedrock paths are consistent. - grid_spec: restore caps=_CAPS_4_6 for Bedrock converse/invoke Opus 4.6 entries so the test reflects the model's actual JSON capabilities. Teach expected() to bypass the xhigh/max cap check when bedrock_effort_ceiling will clamp the wire effort, so the test still passes for Bedrock's graceful degradation contract without lying about native model caps. Co-authored-by: Yassin Kortam <[email protected]> --------- Co-authored-by: Dennis Henry <[email protected]> Co-authored-by: Cursor Agent <[email protected]> Co-authored-by: Yassin Kortam <[email protected]> * feat(guardrails): wire apply_guardrail into proxy logging callbacks (BerriAI#28970) * feat(guardrails): wire apply_guardrail into proxy logging callbacks Route /apply_guardrail through pre/post proxy hooks and LiteLLM success/failure handlers so Langfuse and OTEL integrations receive input/output on guardrail-only requests. Co-authored-by: Cursor <[email protected]> * fix(guardrails): fix Greptile review comments on apply_guardrail logging Co-authored-by: Cursor <[email protected]> * fix(apply_guardrail): preserve original exception and capture modified response - Capture return value from post_call_success_hook so callback-modified responses propagate to the caller. - Wrap success/failure logging calls in defensive try/except so logging infrastructure failures don't replace the user-visible response or mask the original guardrail exception. Co-authored-by: Yassin Kortam <[email protected]> * Fix mypy * fix(apply_guardrail): isolate failure logging and use post-hook response for logging - Split async_failure_handler and post_call_failure_hook into independent try/except blocks so a callback bug in one does not silently skip the other. - Build response_for_logging inside _emit_guardrail_success_logs after post_call_success_hook runs, so logged data matches the response the caller actually receives when the hook modifies the response. Co-authored-by: Yassin Kortam <[email protected]> * fix(apply_guardrail): fix black formatting and update tests for fastapi_request param - Run black on guardrail_endpoints.py to fix CI formatting check - Add _mock_proxy_logging() helper to enterprise guardrail tests to patch proxy-server globals imported at call time - Pass fastapi_request=Mock() in all direct apply_guardrail test calls to match updated function signature Co-authored-by: Cursor <[email protected]> * fix(guardrails): use transformed exception from post_call_failure_hook in apply_guardrail Co-authored-by: Yassin Kortam <[email protected]> * fix(guardrails): isolate sync/async logging handlers in apply_guardrail Separate each logging handler call into its own try/except so a failure in the async handler does not silently skip the sync handler submission (and vice versa). Matches the docstring's defensive intent. Co-authored-by: Yassin Kortam <[email protected]> * fix(apply_guardrail): guard transformed_exception with isinstance check Co-authored-by: Cursor <[email protected]> * test(guardrails): mock proxy globals in not_found test and share apply_guardrail logging fixture - Add proxy-server global mocks to test_apply_guardrail_not_found so the failure-path post_call_failure_hook call doesn't touch the real proxy logging singleton. - Extract the duplicated _mock_proxy_logging context manager out of the two enterprise apply_guardrail test files into a shared conftest fixture so the helper stays in one place. * fix(guardrails): use update_messages to keep logging obj in sync Co-authored-by: Yassin Kortam <[email protected]> --------- Co-authored-by: Cursor <[email protected]> Co-authored-by: Yassin Kortam <[email protected]> Co-authored-by: mateo-berri <[email protected]> * chore(ci): merge dev brach (BerriAI#29192) * build(deps): bump next from 16.2.4 to 16.2.6 in /ui/litellm-dashboard (BerriAI#27665) Bumps [next](https://github.com/vercel/next.js) from 16.2.4 to 16.2.6. - [Release notes](https://github.com/vercel/next.js/releases) - [Changelog](https://github.com/vercel/next.js/blob/canary/release.js) - [Commits](vercel/next.js@v16.2.4...v16.2.6) --- updated-dependencies: - dependency-name: next dependency-version: 16.2.6 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump protobufjs in /tests/pass_through_tests (BerriAI#28296) Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) from 7.5.6 to 7.6.0. - [Release notes](https://github.com/protobufjs/protobuf.js/releases) - [Changelog](https://github.com/protobufjs/protobuf.js/blob/protobufjs-v7.6.0/CHANGELOG.md) - [Commits](protobufjs/protobuf.js@protobufjs-v7.5.6...protobufjs-v7.6.0) --- updated-dependencies: - dependency-name: protobufjs dependency-version: 7.6.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump ws from 8.20.0 to 8.20.1 in /tests/pass_through_tests (BerriAI#28303) Bumps [ws](https://github.com/websockets/ws) from 8.20.0 to 8.20.1. - [Release notes](https://github.com/websockets/ws/releases) - [Commits](websockets/ws@8.20.0...8.20.1) --- updated-dependencies: - dependency-name: ws dependency-version: 8.20.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: improve bedrock streaming hot path perf (BerriAI#28720) * fix(proxy): enforce tag budgets for key-level tags (BerriAI#29108) * fix(proxy): enforce tag budgets for key-level tags Merge API key metadata.tags into request_data before _tag_max_budget_check so per-tag budgets apply when tags are set on the key at creation time. Co-authored-by: Cursor <[email protected]> * fix(auth): avoid false reject for key-inherited tags Run reject_clientside_metadata_tags before key-tag injection, then inject key metadata tags immediately before tag budget checks so key tags still enforce budgets without being treated as client-supplied tags. Co-authored-by: Cursor <[email protected]> --------- Co-authored-by: Cursor <[email protected]> * fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit (BerriAI#29098) * fix(vertex-ai): pass litellm_params to validate_environment in video handlers and implement video edit for Veo - Pass litellm_params to validate_environment in 11 video handler call sites (remix, create_character, get_character, edit, extension, delete) so DB-stored Vertex AI credentials are used instead of falling back to ADC - Implement transform_video_edit_request/response for VertexAI: fetches source video via fetchPredictOperation then submits a new predictLongRunning request with the video bytes/gcsUri + edit prompt Co-authored-by: Cursor <[email protected]> * fix(vertex-ai): hoist fetchPredictOperation into handlers to avoid blocking event loop - Add get_video_edit_prefetch_params() to BaseVideoConfig (returns None) - VertexAI overrides it to return the fetchPredictOperation URL/body - Both sync and async video_edit handlers call this and use their shared httpx client for the fetch, passing the result as prefetched_source_data - transform_video_edit_request is now a pure transform with no HTTP calls - Fix extra_body.pop() mutation by working on a shallow copy Co-authored-by: Cursor <[email protected]> * fix(vertex-ai): include prefetch call inside _handle_error try/except block Co-authored-by: Cursor <[email protected]> * fix(videos): add prefetched_source_data param to all transform_video_edit_request overrides Co-authored-by: Cursor <[email protected]> * fix(video_edit): keep transform/pre_call outside try so validation errors propagate Move transform_video_edit_request and logging_obj.pre_call outside the try/except that wraps HTTP calls in (async_)video_edit_handler so that ValueError validation errors (e.g. 'source video not complete yet') are not silently wrapped as 500s by _handle_error. The prefetch HTTP call keeps its own try/except so its errors are still mapped through the provider's error handler. Matches the pattern used by video_extension_handler and video_remix_handler. Co-authored-by: Yassin Kortam <[email protected]> * refactor(vertex_ai): delegate get_video_edit_prefetch_params to status retrieve Co-authored-by: Yassin Kortam <[email protected]> * Fix varia review * fix(video_edit): route transform errors through _handle_error Wrap transform_video_edit_request and pre_call in the same try/except as the HTTP call in sync and async handlers so validation failures (e.g. source video not complete) return typed LiteLLM exceptions. Co-authored-by: Cursor <[email protected]> --------- Co-authored-by: Cursor <[email protected]> Co-authored-by: Yassin Kortam <[email protected]> * fix(datadog): drain cost-management queue + opt-in FinOps tag allowlist (BerriAI#28487) * fix(datadog): drain cost-management queue + opt-in FinOps tag allowlist * fix(datadog): guard non-dict callback_specific_params + log empty aggregation * fix(datadog): block user-controlled tags from overwriting reserved cost-attribution dimensions * fix(datadog): cast metadata to dict[str, Any] to satisfy mypy * feat(helm): split per-component ServiceAccounts for gateway, backend, and UI (BerriAI#28712) * feat(helm): split per-component ServiceAccounts for gateway, backend, and UI Replace the single shared serviceAccount with three separate serviceAccounts (gateway, backend, ui) so operators can attach different IRSA / Workload Identity annotations per component without granting data-plane credentials to the UI pod. Key changes: - values.yaml: rename serviceAccount → serviceAccounts with gateway/backend/ui sub-keys; UI defaults to automount: false - _helpers.tpl: replace litellm.serviceAccountName with three component-scoped helpers (litellm.gateway/backend/ui.serviceAccountName) - serviceaccount.yaml: create up to three separate ServiceAccount objects with component labels and per-SA automountServiceAccountToken - gateway/backend deployments: use their respective SA helpers - ui deployment: use litellm.ui.serviceAccountName + explicit automountServiceAccountToken: false on the pod spec so the projected token is absent even when the SA itself allows it - migrations-job: share the backend SA (both need DB write access) Resolves LIT-3171 https://claude.ai/code/session_01QPy362WnjmEpeNuJaPUqmF * fix(helm): enforce automountServiceAccountToken on all pod specs; fix leading --- in serviceaccount.yaml - gateway/backend deployments: add explicit automountServiceAccountToken on the pod spec so serviceAccounts.*.automount is honoured regardless of whether the SA is chart-created or operator-supplied (previously the flag only took effect on the SA object when create: true, creating an asymmetry with the UI which already enforced it at pod-spec level) - serviceaccount.yaml: use a $prev sentinel to emit --- only between documents, preventing a leading --- when gateway SA is skipped but backend or ui SA is created (avoids lint/GitOps warnings from strict YAML parsers and tools like ArgoCD) https://claude.ai/code/session_01QPy362WnjmEpeNuJaPUqmF --------- Co-authored-by: Claude <[email protected]> * bump deps (BerriAI#29208) (BerriAI#29226) * fix(deps): bump vulnerable proxy dependencies (starlette/fastapi, granian, pyarrow, semantic-router) Resolve known CVEs flagged by osv-scanner/grype against uv.lock. All bumped versions verified to resolve, install, and pass the proxy auth/route/middleware unit suites (717 tests) plus an import smoke on the new stack. - starlette 0.50.0 -> 1.1.0 (CVE-2026-48710 "BadHost", GHSA-86qp-5c8j-p5mr): versions <1.0.1 reconstruct request.url from the unvalidated Host header, poisoning request.url.path. Required raising fastapi 0.124.4 -> 0.136.3, which dropped fastapi's starlette<0.51.0 cap; an explicit starlette>=1.0.1 floor blocks regression to a vulnerable transitive resolution. The proxy's own auth already reads scope["path"] via get_request_route, but the locked starlette still flagged in container scanners and left other request.url consumers exposed. - granian 2.5.7 -> 2.7.4 (CVE-2026-42544, unauthenticated DoS via WebSocket subprotocol header panic; CVE-2026-42545, WSGI response-header-panic DoS). granian is a selectable proxy server (proxy_cli). - pyarrow 22.0.0 -> 23.0.1 (CVE-2026-25087 / PYSEC-2026-113). - semantic-router 0.1.12 -> 0.1.15: 0.1.12 was yanked (CVE-2026-42208 — its unbounded litellm pin could resolve a credential-exfiltrating litellm==1.82.8 wheel). Not fixable by bump: diskcache 5.6.3 (CVE-2025-69872, unsafe pickle deserialization) has no upstream fix and is left pinned; exploiting it requires write access to the local cache directory. Relock side effect: sse-starlette 3.4.2 -> 3.4.4. * deps: relax exact pins in optional extras to compatible ranges The proxy/optional extras exact-pinned every dependency, which (1) forces downstream `pip install litellm[proxy]` consumers into version lockstep and (2) blocks them from pulling transitive security patches without forking — the structural cause behind needing a litellm release to clear the starlette CVE in the previous commit. Convert the ordinary extras deps to `>=current,<next_major` ranges, mirroring the core [project].dependencies style. Reproducibility for litellm's own Docker/CI is unaffected: images install via `uv sync --frozen`, and the lock re-resolves to the identical versions (no locked version changed). Kept exact-pinned: - litellm-proxy-extras, litellm-enterprise — litellm's own sub-packages, versioned in lockstep with the release. - opentelemetry-api/sdk/exporter-otlp — must resolve to matching versions. - grpcio — supply-chain-pinned to a vetted, aged release. Also corrects the stale comment claiming the extras are exact-pinned for Docker reproducibility (the images use the lock, not these pins). * fix(ci): resolve license-check lookup version from the floor for ranged deps check_licenses.py derived the PyPI lookup version with `next(iter(req.specifier))`, which returns an arbitrary specifier clause. For a range like `>=0.12.1,<1.0` it picked the upper bound (`1.0`) — a version that doesn't exist on PyPI — so the license lookup 404'd and the package was flagged as having an unknown license. The previous commit's switch from exact pins to ranges exposed this for soundfile, pyroscope-io, redisvl, diskcache, and mlflow (the ranged deps not already in liccheck.ini's allowlist). Prefer a lower-bound/exact version (a real released version) for the lookup. * fix(proxy): set strict_content_type=False on the FastAPI app Starlette 1.0 / FastAPI 0.13x flipped the default to strict_content_type=True, which refuses to parse a JSON request body when the client omits the Content-Type header. The proxy previously accepted those requests, so the fastapi/starlette bump in this PR would silently break clients that don't send a Content-Type. Restore the prior lenient behavior explicitly. Co-authored-by: stuxf <[email protected]> * fix(tests/vcr): mint Google OAuth tokens live to prevent stale-token replay (BerriAI#29229) The Redis-backed VCR layer was recording and replaying the Google OAuth2/STS token-mint call. The replayed ya29.* access token is long-expired, but its recorded expires_in keeps credentials.expired False, so litellm never refreshes it and sends the stale token to a live Vertex/Gemini endpoint, which returns 401 ACCESS_TOKEN_EXPIRED. This broke live partner-model tests whose completion call is not itself cassette-backed (e.g. test_vertex_ai_llama_tool_calling). Force credential-exchange hosts to pass through live (never recorded, never replayed) by returning None from before_record_request, mirroring the existing telemetry passthrough, so a fresh token is minted each run. Regression from BerriAI#28826, which added OAuth-token matcher tolerance plus TTL-refresh-on-read so a stale token episode matched and never expired. * chore(cookbook): bump Go directive to 1.26.3 in gollem example (BerriAI#29234) Updates the gollem_go_agent_framework example to the current Go release. Clears stale Go stdlib advisories reported by osv-scanner against the older 1.25.1 directive. No source changes; the single pinned dependency (gollem v0.1.0) is backward compatible. * chore(ci): bump version (BerriAI#29242) * bump: version 1.87.0 → 1.88.0 * uv lock * feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (BerriAI#29238) * feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags Register claude-opus-4-8 across the anthropic/bedrock/vertex/azure cost-map entries, BEDROCK_CONVERSE_MODELS, and the setup-wizard provider list. Prune two reasoning-effort fields from the cost map: - Drop supports_minimal_reasoning_effort from the Claude fleet (58 entries). "minimal" is not a real Anthropic effort level (the API accepts only low/medium/high/xhigh/max), so LiteLLM degrades it to "low" regardless; the flag was inert and misleading on Anthropic. - Remove tool_use_system_prompt_tokens everywhere (103 entries). It is not in the ModelInfo type and is read by no production code. Update the affected config/schema tests; the reasoning-effort registry tests now assert the Claude fleet omits supports_minimal. * fix(anthropic): recognize output_config effort after minimal-flag prune Pruning supports_minimal_reasoning_effort from the Claude fleet removed the only "supports effort param" marker from 11 Opus 4.5 / mythos-preview map entries that lack supports_output_config. _model_supports_effort_param then returned False for them, so output_config was wrongly dropped under drop_params=True -- regressing test_anthropic_model_supports_effort_param_recognizes_supporting_models for claude-opus-4-5-20251101 and the mythos preview. - _model_supports_effort_param now treats supports_output_config as a sufficient signal, matching the bedrock-invoke call sites that already check supports_output_config OR a reasoning-effort flag. Shared map lookup extracted into _supports_model_capability. - Add supports_output_config: true to the 11 Opus 4.5 / mythos entries that lost their only marker, restoring prior effort-forwarding behavior without re-adding the inert minimal flag. * fix(ci): restore real Bedrock batch S3 bucket and role in oai_misc_config (BerriAI#29245) The OSS-staging sync (d52fbfb) overwrote the Bedrock batch model's s3_bucket_name and aws_batch_role_arn with public-safe placeholders (account 123456789012 / *_EXAMPLE role). The e2e_openai_endpoints CI job runs the proxy with AWS account 941277531214 credentials, so on file upload test_bedrock_batches_api failed with: NoSuchBucket: The specified bucket does not exist <BucketName>litellm-proxy-123456789012</BucketName> Restore the real resources that live in account 941277531214 (verified to exist) — the same values tests/batches_tests/test_bedrock_files_and_batches.py already references. Co-authored-by: Claude Opus 4.8 <[email protected]> * fix(guardrails): persist disable_global_guardrails on keys (BerriAI#29233) * fix(guardrails): restore disable_global_guardrails persistence for keys The per-key/team "Disable Global Guardrails" toggle silently stopped working after BerriAI#17042, which removed `disable_global_guardrails` from the key/team request models and from the premium metadata allowlist. Without those, the UI's top-level field was dropped by pydantic and never folded into key `metadata`, so the runtime gate always read False and global default_on guardrails kept running. Restore the request-model fields (KeyRequestBase, NewTeamRequest, UpdateTeamRequest) and the `LiteLLM_ManagementEndpoint_MetadataFields_Premium` entry so the flag is promoted into metadata again. Because the key edit form always submits the flag (false by default), guard the UI so it is only sent when it actually changed (edit) or is enabled (create) — this keeps the premium gate on enabling intact while not 403-ing non-premium users who edit unrelated key fields, mirroring how guardrails/tags are already stripped. * test(guardrails): cover disable_global_guardrails toggle-off + clarify premium field comment Add a prepare_metadata_fields case asserting `disable_global_guardrails: False` overwrites an existing `True`, and rewrite the PREMIUM_METADATA_FIELDS comment to explain why boolean premium fields are excluded from the empty-value strip loop. * test(e2e): cover Team Admin view + member + key flows (BerriAI#29072) * test(e2e): cover Team Admin view + member + key flows Adds a new spec exercising the previously-uncovered team-admin manual-QA items: viewing all team keys (including other members'), adding a member, removing a member, and creating a team key with All Team Models. Also seeds a dedicated invitee user so the add-member test can run in parallel with the proxy-admin invite test without colliding on the team roster. * test(e2e): harden team-admin member specs per review feedback Address Greptile feedback on the Team Admin spec: - locate the delete action via getByTestId("delete-member") instead of the fragile svg/img .last() selector - match the seeded removable member by user_id (members_with_roles stores no email, so the roster renders user_id) - assert exact success-toast strings rather than broad regexes that could match unrelated "success" text * docs: hand-written CLAUDE.md; point GEMINI.md and AGENTS.md at it (BerriAI#29252) * docs: replace generated CLAUDE.md with hand-written guidance, remove AGENTS.md Swap the auto-generated CLAUDE.md for a concise hand-written version that captures how we actually want agents to work in this repo: minimal comments, simplicity first, meaningful tests with a high mutation kill rate, PRs based off litellm_internal_staging rather than main, and curl against a live proxy as proof of fix instead of pasted pytest output. Remove AGENTS.md so there is one source of truth for agent guidance. The customer and company name confidentiality policy, along with the MCP available_on_public_internet note, are carried over from the previous CLAUDE.md. * fix: further clarify communication guidelines * docs: point GEMINI.md at CLAUDE.md instead of duplicating guidance Replace the standalone GEMINI.md copy, which had already drifted from the new CLAUDE.md, with a one-line pointer so Gemini reads the same single source of truth. * docs: simplify PR template test checklist item Replace the rigid "at least 1 test is a hard requirement" checklist line with "I have added meaningful tests", which matches the testing guidance in CLAUDE.md, and tidy a comma into a semicolon in the scope-isolation item. * docs: point AGENTS.md at CLAUDE.md instead of deleting it Keep AGENTS.md so tools that read it still resolve guidance, but collapse it to the same one-line pointer to CLAUDE.md used by GEMINI.md, keeping a single source of truth. * fix: make AI-generated rules more concise * fix: spelling Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix: make the .env usage more careful * docs: restore MCP available_on_public_internet note to CLAUDE.md The PR description states this note was carried over verbatim from the previous CLAUDE.md, but it was dropped in the rewrite. Restore it so the file matches the description and the team guidance is not lost. * docs: restore browser storage and CI supply-chain safety notes to CLAUDE.md These security-relevant rules were dropped in the rewrite. Restore the sessionStorage-over-localStorage (XSS) guidance and the CI supply-chain rules (no curl|bash, pin versions, verify checksums) so agents editing UI or CI code are still steered away from those pitfalls. * docs: move area-specific guidance into nested CLAUDE.md files The MCP, browser-storage, and CI supply-chain notes are scoped to particular parts of the tree, so move each into a nested CLAUDE.md that Claude Code loads on demand when those files are touched: the MCP note under the mcp_server gateway, the browser-storage rule under the UI dashboard, and the CI supply-chain rules under .circleci. Keeps the root CLAUDE.md focused on general guidance while the area notes surface where they are relevant. * docs: keep CI supply-chain note in root CLAUDE.md CI guidance applies beyond .circleci (it also covers downloads in GitHub workflows and any CI script), and CI work does not reliably touch a single subtree, so a nested file under .circleci would not surface it dependably. Keep it in the always-loaded root instead. The MCP and browser-storage notes stay nested where they map cleanly to one area of the tree. * fix: make it clear we prefer httpOnly * chore: make ci rule more concise * chore: make concise Fix formatting and punctuation in MCP note. * fix: don't include Claude attribution --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix: regenerate uv.lock to sync with pyproject.toml Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Mateo Wang <[email protected]> Co-authored-by: Dennis Henry <[email protected]> Co-authored-by: Cursor Agent <[email protected]> Co-authored-by: Yassin Kortam <[email protected]> Co-authored-by: Sameer Kankute <[email protected]> Co-authored-by: yuneng-jiang <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: michelligabriele <[email protected]> Co-authored-by: Claude <[email protected]> Co-authored-by: stuxf <[email protected]> Co-authored-by: ryan-crabbe-berri <[email protected]> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
…video edit (BerriAI#29098) * fix(vertex-ai): pass litellm_params to validate_environment in video handlers and implement video edit for Veo - Pass litellm_params to validate_environment in 11 video handler call sites (remix, create_character, get_character, edit, extension, delete) so DB-stored Vertex AI credentials are used instead of falling back to ADC - Implement transform_video_edit_request/response for VertexAI: fetches source video via fetchPredictOperation then submits a new predictLongRunning request with the video bytes/gcsUri + edit prompt Co-authored-by: Cursor <[email protected]> * fix(vertex-ai): hoist fetchPredictOperation into handlers to avoid blocking event loop - Add get_video_edit_prefetch_params() to BaseVideoConfig (returns None) - VertexAI overrides it to return the fetchPredictOperation URL/body - Both sync and async video_edit handlers call this and use their shared httpx client for the fetch, passing the result as prefetched_source_data - transform_video_edit_request is now a pure transform with no HTTP calls - Fix extra_body.pop() mutation by working on a shallow copy Co-authored-by: Cursor <[email protected]> * fix(vertex-ai): include prefetch call inside _handle_error try/except block Co-authored-by: Cursor <[email protected]> * fix(videos): add prefetched_source_data param to all transform_video_edit_request overrides Co-authored-by: Cursor <[email protected]> * fix(video_edit): keep transform/pre_call outside try so validation errors propagate Move transform_video_edit_request and logging_obj.pre_call outside the try/except that wraps HTTP calls in (async_)video_edit_handler so that ValueError validation errors (e.g. 'source video not complete yet') are not silently wrapped as 500s by _handle_error. The prefetch HTTP call keeps its own try/except so its errors are still mapped through the provider's error handler. Matches the pattern used by video_extension_handler and video_remix_handler. Co-authored-by: Yassin Kortam <[email protected]> * refactor(vertex_ai): delegate get_video_edit_prefetch_params to status retrieve Co-authored-by: Yassin Kortam <[email protected]> * Fix varia review * fix(video_edit): route transform errors through _handle_error Wrap transform_video_edit_request and pre_call in the same try/except as the HTTP call in sync and async handlers so validation failures (e.g. source video not complete) return typed LiteLLM exceptions. Co-authored-by: Cursor <[email protected]> --------- Co-authored-by: Cursor <[email protected]> Co-authored-by: Yassin Kortam <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two bugs in Vertex AI video endpoints:
Wrong credentials: 10 video handler call sites (
remix,create_character,get_character,edit,extension,delete) inllm_http_handler.pycalledvalidate_environmentwithout passinglitellm_params. This caused Vertex AI to ignore DB-stored credentials and fall back to application default credentials (ADC), resulting inRefreshErrorfailures when ADC was not configured.NotImplementedError on video edit:
transform_video_edit_requestfor Vertex AI raisedNotImplementedError, so/v1/videos/editsalways failed for Vertex AI models.Fix
litellm_params=litellm_paramsto all 11 affectedvalidate_environmentcall sites so DB-storedvertex_project/vertex_credentialsare used correctly.transform_video_edit_requestandtransform_video_edit_responsefor Vertex AI: the handler first callsfetchPredictOperationto retrieve the completed source video (bytes or GCS URI), then submits a newpredictLongRunningrequest with the source video + edit prompt, returning a new operation ID that can be polled for status.Tests
Added 4 unit tests to
tests/test_litellm/llms/vertex_ai/videos/test_vertex_video_transformation.py:test_transform_video_edit_request_with_bytes— edit using base64 video bytestest_transform_video_edit_request_with_gcs_uri— edit using GCS URItest_transform_video_edit_request_source_not_done_raises— clear error when source video is still processingtest_transform_video_edit_response— response encodes new operation ID correctlyFixed it here
Note
Medium Risk
Touches shared video HTTP paths and credential resolution for multiple operations; Vertex edit adds a two-step API flow and validates completed source video data before submit.
Overview
Video HTTP handlers now pass
litellm_paramsintovalidate_environmentacross remix, character, edit, extension, delete, and related paths so Vertex (and other providers) can use DB-stored project/credentials instead of defaulting to ADC.Vertex AI Veo video edit is implemented end-to-end: a new optional
get_video_edit_prefetch_paramshook lets the shared sync/async httpx handler runfetchPredictOperationbeforetransform_video_edit_request, which builds apredictLongRunningedit from the completed source video (GCS URI or base64 bytes) and returns a processingVideoObjectfrom the new operation name. Other providers only accept the newprefetched_source_dataparameter on edit transforms.Unit tests cover prefetch params, edit request/response shapes, and errors when the source operation is not done.
Reviewed by Cursor Bugbot for commit 7543fe0. Bugbot is set up for automated code reviews on this repo. Configure here.