fix(proxy): enforce allowed_passthrough_routes for auth=true pass-thr…#29256
Merged
mateo-berri merged 19 commits intoMay 31, 2026
Merged
Conversation
…ough Pass-through endpoints with auth=true were injected into openai_routes, so teams with openai_routes access bypassed per-team allowed_passthrough_routes. Gate auth-enforced pass-through at JWT, virtual-key, and non-admin route checks. Co-authored-by: Cursor <[email protected]>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Contributor
Greptile SummaryThis PR enforces
Confidence Score: 4/5The access-control logic is sound and well-tested, but a pre-existing defect in The new enforcement logic across all three auth paths (virtual-key, non-proxy-admin, JWT) is correct and tests cover the critical scenarios including the RBAC role-claim edge case. The remaining concern is that litellm/proxy/pass_through_endpoints/pass_through_endpoints.py — specifically the
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/route_checks.py | Adds is_auth_enforced_pass_through_route, _require_auth_pass_through_access, and related helpers; threads the passthrough allowlist check before the blanket is_llm_api_route allow in both non_proxy_admin_allowed_routes_check and is_virtual_key_allowed_to_call_route. |
| litellm/proxy/auth/handle_jwt.py | Adds passthrough access enforcement to find_team_with_model_access and a post-resolution check in auth_builder; includes targeted RBAC team-object fetch so teams selected by role-claim now have their metadata evaluated. Propagates request_method through the JWT auth stack. |
| litellm/proxy/pass_through_endpoints/pass_through_endpoints.py | Stores auth flag in route registry entries, propagates it through create/update paths, and adds a 405 guard in the handler for stale method-keyed routes. The update_pass_through_endpoints path uses model_dump(exclude_none=True) which promotes silent auth=True on partial updates (flagged in an existing comment). |
| litellm/proxy/auth/user_api_key_auth.py | Passes request_method from the incoming request to auth_builder via RouteChecks._get_request_method; minimal change with no logic impact. |
| tests/test_litellm/proxy/auth/test_route_checks.py | Adds comprehensive tests for auth-enforced passthrough gating via openai_routes/llm_api_routes; replaces the old single-scenario llm_api_routes test with multiple focused cases. Registry mock keys updated to include methods and auth flag. |
| tests/test_litellm/proxy/auth/test_handle_jwt.py | New tests cover: find_team_with_model_access raising the correct 403, method-aware passthrough gating in JWT team selection, header/field/RBAC paths in auth_builder, and the RBAC team-fetch-then-check sequence. |
| tests/local_testing/test_pass_through_endpoints.py | Adds allowed_passthrough_routes: ["/v1/rerank"] to mock key cache values in the two RPM-limit tests so they continue to reach the rate-limiting logic now that auth-enforced passthrough is checked first; does not weaken RPM coverage. |
| tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py | New tests cover: 405 handler for stale-method routes, create/update endpoints with auth=True enforcing the allowlist correctly. |
Reviews (12): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Co-authored-by: Cursor <[email protected]>
Collaborator
Author
|
@greptile review again with new commit that fixes. P2 No test coverage for the JWT find_team_with_model_access passthrough path, P2 JWT passthrough check only inspects team metadata, not key-level metadata, P2 Misleading error message when JWT team fails passthrough check. Also note the backwards incompatibility concern is valid but that is also the expected default behavior so update score taking this into consideration. |
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
Contributor
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 2 · PR risk: 0/10 |
Prevent allowlist bypass when the same path is registered with different auth settings per HTTP method. Co-authored-by: Cursor <[email protected]>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: JWT passthrough check omits HTTP method, risking bypass
- JWT auth now propagates the request method into team passthrough route checks so method-specific auth settings are enforced.
- ✅ Fixed: Auth-enforced passthrough denial gives misleading generic error
- Virtual key passthrough denials now raise the passthrough-specific 403 guidance when allowed_passthrough_routes is missing.
Preview (bf95c60b65)
diff --git a/litellm/proxy/auth/handle_jwt.py b/litellm/proxy/auth/handle_jwt.py
--- a/litellm/proxy/auth/handle_jwt.py
+++ b/litellm/proxy/auth/handle_jwt.py
@@ -1120,10 +1120,16 @@
user_api_key_cache: UserApiKeyCache,
parent_otel_span: Optional[Span],
proxy_logging_obj: ProxyLogging,
+ request_method: Optional[str] = None,
) -> Tuple[Optional[str], Optional[LiteLLM_TeamTable]]:
"""Find first team with access to the requested model"""
from litellm.proxy.proxy_server import llm_router
+ normalized_request_method = (
+ request_method.upper() if isinstance(request_method, str) else None
+ )
+ denied_auth_enforced_pass_through_route = False
+
if not team_ids:
if jwt_handler.litellm_jwtauth.enforce_team_based_model_access:
raise HTTPException(
@@ -1158,6 +1164,23 @@
user_route=route,
litellm_proxy_roles=jwt_handler.litellm_jwtauth,
)
+ if (
+ is_allowed
+ and RouteChecks.is_auth_enforced_pass_through_route(
+ route=route,
+ method=normalized_request_method,
+ )
+ ):
+ # JWT team selection is team-scoped; key metadata is not
+ # available here, so passthrough access is granted only by
+ # the selected team's metadata.
+ is_allowed = RouteChecks.check_passthrough_route_access(
+ route=route,
+ user_api_key_dict=UserAPIKeyAuth(
+ team_metadata=team_object.metadata or {}
+ ),
+ )
+ denied_auth_enforced_pass_through_route = not is_allowed
verbose_proxy_logger.debug(
f"JWT team route check: team_id={team_id}, route={route}, is_allowed={is_allowed}"
)
@@ -1166,6 +1189,15 @@
except Exception:
continue
+ if denied_auth_enforced_pass_through_route:
+ raise HTTPException(
+ status_code=403,
+ detail=(
+ f"Team not allowed to access passthrough route {route}. "
+ "Configure `allowed_passthrough_routes` on the team."
+ ),
+ )
+
if requested_model:
raise HTTPException(
status_code=403,
@@ -1592,6 +1624,7 @@
parent_otel_span: Optional[Span],
proxy_logging_obj: ProxyLogging,
request_headers: Optional[dict] = None,
+ request_method: Optional[str] = None,
) -> JWTAuthBuilderResult:
"""Main authentication and authorization builder"""
# Check if OIDC UserInfo endpoint is enabled, but fall back to standard
@@ -1728,6 +1761,7 @@
user_api_key_cache=user_api_key_cache,
parent_otel_span=parent_otel_span,
proxy_logging_obj=proxy_logging_obj,
+ request_method=request_method,
)
# Extract alias fields for resolution (if configured)
org_alias = jwt_handler.get_org_alias(token=jwt_valid_token, default_value=None)
diff --git a/litellm/proxy/auth/route_checks.py b/litellm/proxy/auth/route_checks.py
--- a/litellm/proxy/auth/route_checks.py
+++ b/litellm/proxy/auth/route_checks.py
@@ -103,6 +103,8 @@
if len(valid_token.allowed_routes) == 0:
return True
+ denied_auth_enforced_pass_through_route = False
+
# explicit check for allowed routes (exact match or prefix match)
for allowed_route in valid_token.allowed_routes:
if RouteChecks._route_matches_allowed_route(
@@ -121,7 +123,17 @@
route=route,
allowed_routes=LiteLLMRoutes._member_map_[allowed_route].value,
):
- return True
+ if RouteChecks.is_auth_enforced_pass_through_route(
+ route=route,
+ method=RouteChecks._get_request_method(request=request),
+ ):
+ if RouteChecks.check_passthrough_route_access(
+ route=route, user_api_key_dict=valid_token
+ ):
+ return True
+ denied_auth_enforced_pass_through_route = True
+ else:
+ return True
################################################
# For llm_api_routes, also check registered pass-through endpoints
@@ -134,7 +146,17 @@
if InitPassThroughEndpointHelpers.is_registered_pass_through_route(
route=route
):
- return True
+ if RouteChecks.is_auth_enforced_pass_through_route(
+ route=route,
+ method=RouteChecks._get_request_method(request=request),
+ ):
+ if RouteChecks.check_passthrough_route_access(
+ route=route, user_api_key_dict=valid_token
+ ):
+ return True
+ denied_auth_enforced_pass_through_route = True
+ else:
+ return True
# Method-aware carve-out: allow GET on the two
# read-only MCP-server discovery endpoints
@@ -158,6 +180,11 @@
):
return True
+ if denied_auth_enforced_pass_through_route:
+ RouteChecks._require_auth_pass_through_access(
+ route=route, valid_token=valid_token
+ )
+
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=f"Virtual key is not allowed to call this route. Only allowed to call routes: {valid_token.allowed_routes}. Tried to call route: {route}",
@@ -228,7 +255,14 @@
route=route,
)
- if RouteChecks.is_llm_api_route(route=route):
+ if RouteChecks.is_auth_enforced_pass_through_route(
+ route=route,
+ method=RouteChecks._get_request_method(request=request),
+ ):
+ RouteChecks._require_auth_pass_through_access(
+ route=route, valid_token=valid_token
+ )
+ elif RouteChecks.is_llm_api_route(route=route):
pass
elif RouteChecks.is_info_route(route=route):
# check if user allowed to call an info route
@@ -625,6 +659,60 @@
return False
@staticmethod
+ def _get_request_method(request: Optional[Request]) -> Optional[str]:
+ if request is None:
+ return None
+
+ method = getattr(request, "method", None)
+ if not isinstance(method, str):
+ return None
+
+ return method.upper()
+
+ @staticmethod
+ def is_auth_enforced_pass_through_route(
+ route: str, method: Optional[str] = None
+ ) -> bool:
+ """
+ True for config/DB pass-through endpoints registered with auth=true.
+
+ These routes are injected into ``openai_routes`` for spend/budget hooks but
+ must not inherit blanket ``openai_routes`` RBAC; access is gated by
+ ``allowed_passthrough_routes`` on the key or team.
+ """
+ from litellm.proxy.pass_through_endpoints.pass_through_endpoints import (
+ InitPassThroughEndpointHelpers,
+ )
+
+ route_info = InitPassThroughEndpointHelpers.get_registered_pass_through_route(
+ route=route, method=method
+ )
+ if route_info is None:
+ return False
+ dependencies = route_info.get("passthrough_params", {}).get("dependencies")
+ return dependencies is not None
+
+ @staticmethod
+ def _require_auth_pass_through_access(
+ route: str,
+ valid_token: UserAPIKeyAuth,
+ ) -> None:
+ """
+ Require an explicit ``allowed_passthrough_routes`` match for auth=true pass-through.
+ """
+ if RouteChecks.check_passthrough_route_access(
+ route=route, user_api_key_dict=valid_token
+ ):
+ return
+ raise HTTPException(
+ status_code=status.HTTP_403_FORBIDDEN,
+ detail=(
+ f"Key/team not allowed to access passthrough route {route}. "
+ "Configure `allowed_passthrough_routes` on the team or key."
+ ),
+ )
+
+ @staticmethod
def check_passthrough_route_access(
route: str, user_api_key_dict: UserAPIKeyAuth
) -> bool:
diff --git a/litellm/proxy/auth/user_api_key_auth.py b/litellm/proxy/auth/user_api_key_auth.py
--- a/litellm/proxy/auth/user_api_key_auth.py
+++ b/litellm/proxy/auth/user_api_key_auth.py
@@ -917,6 +917,9 @@
proxy_logging_obj=proxy_logging_obj,
parent_otel_span=parent_otel_span,
request_headers=_safe_get_request_headers(request),
+ request_method=RouteChecks._get_request_method(
+ request=request
+ ),
)
is_proxy_admin = result["is_proxy_admin"]
diff --git a/litellm/proxy/pass_through_endpoints/pass_through_endpoints.py b/litellm/proxy/pass_through_endpoints/pass_through_endpoints.py
--- a/litellm/proxy/pass_through_endpoints/pass_through_endpoints.py
+++ b/litellm/proxy/pass_through_endpoints/pass_through_endpoints.py
@@ -2312,8 +2312,12 @@
InitPassThroughEndpointHelpers._build_full_path_with_root(parts[2])
)
- # Get the methods for this route
+ # Get the methods for this route. Prefer the registered metadata,
+ # but keep supporting test fixtures / older registry entries that
+ # only encoded methods in the route key.
route_methods = _registered_pass_through_routes[key].get("methods", [])
+ if not route_methods and len(parts) == 4:
+ route_methods = parts[3].split(",")
# Check if path matches
path_matches = False
diff --git a/tests/test_litellm/proxy/auth/test_handle_jwt.py b/tests/test_litellm/proxy/auth/test_handle_jwt.py
--- a/tests/test_litellm/proxy/auth/test_handle_jwt.py
+++ b/tests/test_litellm/proxy/auth/test_handle_jwt.py
@@ -1,6 +1,7 @@
from typing import Optional
from unittest.mock import AsyncMock, MagicMock, patch
+from fastapi import HTTPException
import pytest
from litellm.proxy._types import (
@@ -133,6 +134,137 @@
@pytest.mark.asyncio
+async def test_find_team_with_model_access_reports_passthrough_allowlist_denial():
+ jwt_handler = JWTHandler()
+ jwt_handler.litellm_jwtauth = LiteLLM_JWTAuth()
+ team = LiteLLM_TeamTable(
+ team_id="team-a",
+ models=["gpt-4"],
+ metadata={},
+ )
+
+ with (
+ patch(
+ "litellm.proxy.auth.handle_jwt.get_team_object",
+ new_callable=AsyncMock,
+ return_value=team,
+ ),
+ patch(
+ "litellm.proxy.auth.handle_jwt.can_team_access_model",
+ new_callable=AsyncMock,
+ return_value=True,
+ ),
+ patch(
+ "litellm.proxy.auth.handle_jwt.allowed_routes_check",
+ return_value=True,
+ ),
+ patch(
+ "litellm.proxy.auth.handle_jwt.RouteChecks.is_auth_enforced_pass_through_route",
+ return_value=True,
+ ),
+ patch(
+ "litellm.proxy.auth.handle_jwt.RouteChecks.check_passthrough_route_access",
+ return_value=False,
+ ) as mock_passthrough_check,
+ ):
+ with pytest.raises(HTTPException) as exc_info:
+ await JWTAuthManager.find_team_with_model_access(
+ team_ids={"team-a"},
+ requested_model="gpt-4",
+ route="/my-pass-through",
+ jwt_handler=jwt_handler,
+ prisma_client=None,
+ user_api_key_cache=MagicMock(),
+ parent_otel_span=None,
+ proxy_logging_obj=MagicMock(),
+ )
+
+ assert exc_info.value.status_code == 403
+ assert "allowed_passthrough_routes" in exc_info.value.detail
+ assert "requested model" not in exc_info.value.detail
+
+ user_api_key_dict = mock_passthrough_check.call_args.kwargs["user_api_key_dict"]
+ assert user_api_key_dict.metadata == {}
+ assert user_api_key_dict.team_metadata == {}
+
+
+@pytest.mark.asyncio
+async def test_find_team_with_model_access_uses_request_method_for_passthrough_auth():
+ jwt_handler = JWTHandler()
+ jwt_handler.litellm_jwtauth = LiteLLM_JWTAuth()
+ team = LiteLLM_TeamTable(
+ team_id="team-a",
+ models=["gpt-4"],
+ metadata={},
+ )
+ mock_registered_routes = {
+ "test-uuid-1:exact:/custom:GET": {
+ "endpoint_id": "test-uuid-1",
+ "path": "/custom",
+ "type": "exact",
+ "methods": ["GET"],
+ "passthrough_params": {"dependencies": None},
+ },
+ "test-uuid-2:exact:/custom:POST": {
+ "endpoint_id": "test-uuid-2",
+ "path": "/custom",
+ "type": "exact",
+ "methods": ["POST"],
+ "passthrough_params": {"dependencies": [object()]},
+ },
+ }
+
+ with (
+ patch(
+ "litellm.proxy.auth.handle_jwt.get_team_object",
+ new_callable=AsyncMock,
+ return_value=team,
+ ),
+ patch(
+ "litellm.proxy.auth.handle_jwt.allowed_routes_check",
+ return_value=True,
+ ),
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints._registered_pass_through_routes",
+ mock_registered_routes,
+ ),
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path",
+ return_value="/",
+ ),
+ ):
+ team_id, team_obj = await JWTAuthManager.find_team_with_model_access(
+ team_ids={"team-a"},
+ requested_model=None,
+ route="/custom",
+ jwt_handler=jwt_handler,
+ prisma_client=None,
+ user_api_key_cache=MagicMock(),
+ parent_otel_span=None,
+ proxy_logging_obj=MagicMock(),
+ request_method="GET",
+ )
+ assert team_id == "team-a"
+ assert team_obj == team
+
+ with pytest.raises(HTTPException) as exc_info:
+ await JWTAuthManager.find_team_with_model_access(
+ team_ids={"team-a"},
+ requested_model=None,
+ route="/custom",
+ jwt_handler=jwt_handler,
+ prisma_client=None,
+ user_api_key_cache=MagicMock(),
+ parent_otel_span=None,
+ proxy_logging_obj=MagicMock(),
+ request_method="POST",
+ )
+
+ assert exc_info.value.status_code == 403
+ assert "allowed_passthrough_routes" in exc_info.value.detail
+
+
+@pytest.mark.asyncio
async def test_auth_builder_proxy_admin_user_role():
"""Test that is_proxy_admin is True when user_object.user_role is PROXY_ADMIN"""
# Setup test data
diff --git a/tests/test_litellm/proxy/auth/test_route_checks.py b/tests/test_litellm/proxy/auth/test_route_checks.py
--- a/tests/test_litellm/proxy/auth/test_route_checks.py
+++ b/tests/test_litellm/proxy/auth/test_route_checks.py
@@ -686,24 +686,23 @@
def test_virtual_key_llm_api_routes_allows_registered_pass_through_endpoints():
"""
- Test that virtual keys with llm_api_routes permission can access registered pass-through endpoints.
-
- This tests the scenario where a pass-through endpoint is registered from the DB
- (e.g., /azure-assistant) and a virtual key with llm_api_routes permission should be able to access
- both the exact path and subpaths (e.g., /azure-assistant/openai/assistants).
+ Virtual keys with llm_api_routes can access auth=true pass-through endpoints only when
+ allowed_passthrough_routes is configured on the key or team.
"""
- # Mock the registered pass-through routes
+ mock_dependencies = [object()]
mock_registered_routes = {
- "test-uuid-1:exact:/azure-assistant": {
+ "test-uuid-1:exact:/azure-assistant:DELETE,GET,PATCH,POST,PUT": {
"endpoint_id": "test-uuid-1",
"path": "/azure-assistant",
"type": "exact",
+ "passthrough_params": {"dependencies": mock_dependencies},
},
- "test-uuid-2:subpath:/custom-endpoint": {
+ "test-uuid-2:subpath:/custom-endpoint:DELETE,GET,PATCH,POST,PUT": {
"endpoint_id": "test-uuid-2",
"path": "/custom-endpoint",
"type": "subpath",
+ "passthrough_params": {"dependencies": mock_dependencies},
},
}
@@ -717,32 +716,208 @@
return_value="/",
),
):
- # Create a virtual key with llm_api_routes permission
valid_token = UserAPIKeyAuth(
user_id="test_user",
allowed_routes=["llm_api_routes"],
+ metadata={
+ "allowed_passthrough_routes": [
+ "/azure-assistant",
+ "/custom-endpoint",
+ ]
+ },
)
- # Test exact match for registered pass-through endpoint
- result1 = RouteChecks.is_virtual_key_allowed_to_call_route(
- route="/azure-assistant",
- valid_token=valid_token,
+ assert (
+ RouteChecks.is_virtual_key_allowed_to_call_route(
+ route="/azure-assistant",
+ valid_token=valid_token,
+ )
+ is True
)
- assert result1 is True
+ assert (
+ RouteChecks.is_virtual_key_allowed_to_call_route(
+ route="/custom-endpoint/openai/assistants",
+ valid_token=valid_token,
+ )
+ is True
+ )
+ assert (
+ RouteChecks.is_virtual_key_allowed_to_call_route(
+ route="/custom-endpoint",
+ valid_token=valid_token,
+ )
+ is True
+ )
- # Test subpath for registered pass-through endpoint with subpath type
- result2 = RouteChecks.is_virtual_key_allowed_to_call_route(
- route="/custom-endpoint/openai/assistants",
- valid_token=valid_token,
+
+def test_virtual_key_llm_api_routes_denies_auth_pass_through_without_allowlist():
+ """auth=true pass-through must not be reachable via llm_api_routes alone."""
+
+ mock_registered_routes = {
+ "test-uuid-1:exact:/azure-assistant:GET,POST": {
+ "endpoint_id": "test-uuid-1",
+ "path": "/azure-assistant",
+ "type": "exact",
+ "passthrough_params": {"dependencies": [object()]},
+ },
+ }
+
+ with (
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints._registered_pass_through_routes",
+ mock_registered_routes,
+ ),
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path",
+ return_value="/",
+ ),
+ ):
+ valid_token = UserAPIKeyAuth(
+ user_id="test_user",
+ allowed_routes=["llm_api_routes"],
)
- assert result2 is True
- # Test exact match for subpath type
- result3 = RouteChecks.is_virtual_key_allowed_to_call_route(
- route="/custom-endpoint",
+ with pytest.raises(HTTPException) as exc_info:
+ RouteChecks.is_virtual_key_allowed_to_call_route(
+ route="/azure-assistant",
+ valid_token=valid_token,
+ )
+ assert exc_info.value.status_code == 403
+ assert "allowed_passthrough_routes" in exc_info.value.detail
+
+
+def test_virtual_key_llm_api_routes_uses_method_specific_auth_setting():
+ """Same-path pass-through routes must be checked against the request method."""
+
+ mock_registered_routes = {
+ "test-uuid-1:exact:/custom:GET": {
+ "endpoint_id": "test-uuid-1",
+ "path": "/custom",
+ "type": "exact",
+ "methods": ["GET"],
+ "passthrough_params": {"dependencies": None},
+ },
+ "test-uuid-2:exact:/custom:POST": {
+ "endpoint_id": "test-uuid-2",
+ "path": "/custom",
+ "type": "exact",
+ "methods": ["POST"],
+ "passthrough_params": {"dependencies": [object()]},
+ },
+ }
+
+ with (
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints._registered_pass_through_routes",
+ mock_registered_routes,
+ ),
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path",
+ return_value="/",
+ ),
+ ):
+ valid_token = UserAPIKeyAuth(
+ user_id="test_user",
+ allowed_routes=["llm_api_routes"],
+ )
+
+ get_request = MagicMock(spec=Request)
+ get_request.method = "GET"
+ assert (
+ RouteChecks.is_virtual_key_allowed_to_call_route(
+ route="/custom",
+ valid_token=valid_token,
+ request=get_request,
+ )
+ is True
+ )
+
+ post_request = MagicMock(spec=Request)
+ post_request.method = "POST"
+ with pytest.raises(HTTPException) as exc_info:
+ RouteChecks.is_virtual_key_allowed_to_call_route(
+ route="/custom",
+ valid_token=valid_token,
+ request=post_request,
+ )
+
+ assert exc_info.value.status_code == 403
+
+
+def test_non_proxy_admin_denies_auth_pass_through_without_allowlist():
+ """Internal users must not bypass allowed_passthrough_routes via openai_routes."""
+
+ mock_registered_routes = {
+ "test-uuid-1:exact:/my-pass-through:GET,POST": {
+ "endpoint_id": "test-uuid-1",
+ "path": "/my-pass-through",
+ "type": "exact",
+ "passthrough_params": {"dependencies": [object()]},
+ },
+ }
+
+ valid_token = UserAPIKeyAuth(
+ user_id="test_user",
+ user_role=LitellmUserRoles.INTERNAL_USER.value,
+ )
+
+ with (
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints._registered_pass_through_routes",
+ mock_registered_routes,
+ ),
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path",
+ return_value="/",
+ ),
+ ):
+ with pytest.raises(HTTPException) as exc_info:
+ RouteChecks.non_proxy_admin_allowed_routes_check(
+ user_obj=None,
+ _user_role=LitellmUserRoles.INTERNAL_USER.value,
+ route="/my-pass-through",
+ request=MagicMock(spec=Request),
+ valid_token=valid_token,
+ request_data={},
+ )
+ assert exc_info.value.status_code == 403
+ assert "allowed_passthrough_routes" in exc_info.value.detail
+
+
+def test_non_proxy_admin_allows_auth_pass_through_with_team_allowlist():
+ mock_registered_routes = {
+ "test-uuid-1:exact:/my-pass-through:GET,POST": {
+ "endpoint_id": "test-uuid-1",
+ "path": "/my-pass-through",
+ "type": "exact",
+ "passthrough_params": {"dependencies": [object()]},
+ },
+ }
+
+ valid_token = UserAPIKeyAuth(
+ user_id="test_user",
+ user_role=LitellmUserRoles.INTERNAL_USER.value,
+ team_metadata={"allowed_passthrough_routes": ["/my-pass-through"]},
+ )
+
+ with (
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints._registered_pass_through_routes",
+ mock_registered_routes,
+ ),
+ patch(
+ "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path",
+ return_value="/",
+ ),
+ ):
+ RouteChecks.non_proxy_admin_allowed_routes_check(
+ user_obj=None,
+ _user_role=LitellmUserRoles.INTERNAL_USER.value,
+ route="/my-pass-through",
+ request=MagicMock(spec=Request),
valid_token=valid_token,
+ request_data={},
)
- assert result3 is True
def test_virtual_key_without_llm_api_routes_cannot_access_pass_through():
diff --git a/tests/test_litellm/proxy/auth/test_user_api_key_auth.py b/tests/test_litellm/proxy/auth/test_user_api_key_auth.py
--- a/tests/test_litellm/proxy/auth/test_user_api_key_auth.py
+++ b/tests/test_litellm/proxy/auth/test_user_api_key_auth.py
@@ -1513,6 +1513,7 @@
mock_request = MagicMock()
mock_request.url.path = "/v1/chat/completions"
+ mock_request.method = "POST"
mock_request.headers = {"authorization": f"Bearer {jwt_token}"}
mock_request.query_params = {}
@@ -1546,6 +1547,7 @@
mock_oauth2.assert_not_called()
# JWT auth SHOULD be called
mock_jwt_auth.assert_called_once()
+ assert mock_jwt_auth.call_args.kwargs["request_method"] == "POST"
assert result.user_id == "jwt-human-user"
@pytest.mark.asyncioYou can send follow-ups to the cloud agent here.
|
|
Enforce method-aware JWT checks and return 405 when stale FastAPI routes accept requests outside the current pass-through registry. Co-authored-by: Cursor <[email protected]>
Fixes SyntaxError on proxy startup caused by passing request_method twice to find_team_with_model_access. Co-authored-by: Cursor <[email protected]>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Auth-enforced check applies broadly to all route groups
- Scoped auth-enforced passthrough gating to only openai_routes and llm_api_routes so other explicitly granted route groups remain allowed.
- ✅ Fixed: JWT header-selected teams skip passthrough enforcement entirely
- Added shared team passthrough enforcement for JWT-selected teams, including header and team-id claim paths, with regression coverage.
You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 36f43c0. Configure here.
Collaborator
Collaborator
Collaborator
…28728)" (#29326) This reverts the Bedrock CI account migration (#28728). The original account (888602223428) was put under an AWS security restriction after a leaked key and has since been reactivated, while the replacement account (941277531214) lacks access to several models the suites exercise (legacy Bedrock Claude 3 models, Cohere, Nova Canvas image gen, Bedrock batch inference, and flagship Opus). Pointing CI back at the reactivated account restores that coverage. This is the exact inverse of #28728: all hardcoded 941277531214 references go back to 888602223428 (provisioned/imported-model ARNs, AgentCore runtime ARNs and their suffixes, batch execution role ARN, and the example proxy config), the S3 buckets revert to litellm-proxy and load-testing-oct, the guardrail IDs revert to wf0hkdb5x07f and ff6ujrregl1q, the SageMaker endpoint and Knowledge Base revert to their original ids, and the live-call tests go back to the legacy model strings. The grid_spec fail_reason workaround for the unentitled Opus cells is dropped while keeping the unrelated bedrock_effort_ceiling field added after the migration. The CircleCI AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env vars still point at 941277531214 and must be set to the reactivated account's fresh credentials separately via the CircleCI API; AWS_REGION_NAME stays us-west-2. (cherry picked from commit f11c12d)
…assthrough in rpm tests The auth=true pass-through 405 guard fired for mapped provider routes (e.g. /assemblyai/*) that are not in the in-memory registry, since get_registered_pass_through_route returns None for them while is_registered_pass_through_route matches via mapped_pass_through_routes. Only raise 405 when the path is registered but the request method is not allowed, so mapped provider pass-throughs fall through to the default target params as before. The rpm-limit pass-through tests register /v1/rerank with auth=true but gave their keys no allowed_passthrough_routes, so the new default-deny returned 403 before the rate limiter ran (non-deterministically, depending on registry insertion order). Grant the keys explicit passthrough access so the tests exercise rate limiting under the new auth model.
…itellm_passthrough_fix_allowed_routes # Conflicts: # tests/llm_translation/reasoning_effort_grid/grid_spec.py
Collaborator
Starlette's Request.method property reads scope["method"] and raises KeyError when the scope omits it (e.g. minimally-constructed test requests). getattr only swallows AttributeError, so the new _get_request_method helper propagated the KeyError up through user_api_key_auth and surfaced as a ProxyException. Catch KeyError (and AttributeError) and fall back to None.
Collaborator
test_custom_proxy.py sets os.environ['SERVER_ROOT_PATH'] = '/my-custom-path' at module import with no cleanup. When that module is collected into the same xdist worker as this test, the leaked root path is prepended to registered pass-through paths, so is_registered_pass_through_route misses '/test/path' and the handler returns 404 instead of the expected 405 (order-dependent). Pin SERVER_ROOT_PATH to '' so the test is deterministic.
Collaborator
…pass-through via llm_api_routes
Collaborator
…nforcement Auth-enforced pass-through detection inferred enforcement from the FastAPI dependency stored at registration time. The management create and update endpoints register routes with dependencies=None even though auth defaults to true, so is_auth_enforced_pass_through_route treated those DB-created routes as unenforced. A key allowed for llm_api_routes could then call a management-created auth-enabled pass-through route without matching allowed_passthrough_routes. Store the auth setting on each registry entry and read it directly when deciding whether the allowlist applies, instead of deriving it from dependency metadata.
Collaborator
… flag The auth flag stored in _registered_pass_through_routes is a bool, which was not part of the registry value Union, so mypy rejected the dict literal. Add bool to the Union and narrow route_methods to a list before the membership check so the in-operator stays valid.
Collaborator
model_dump(exclude_none=True) re-included the auth=True default whenever a partial update omitted auth, silently flipping an existing auth=false pass-through to auth-enforced and 403ing every team/key without allowed_passthrough_routes. Merge only explicitly set fields via exclude_unset so omitted fields keep their stored value.
…itellm_passthrough_fix_allowed_routes # Conflicts: # litellm/proxy/pass_through_endpoints/pass_through_endpoints.py
Collaborator
7 tasks
milan-berri
added a commit
that referenced
this pull request
Jun 5, 2026
Staging #29256 added a test that still mocked get_objects with a 4-tuple; our PR expanded the return to 5 values (effective_user_id). Co-authored-by: Cursor <[email protected]>
ryan-crabbe-berri
pushed a commit
that referenced
this pull request
Jun 5, 2026
…9217) * fix(jwt): attribute spend to resolved DB user_id on email/sso fuzzy match When user_id_upsert is enabled with JWT auth and a pre-migration user row exists whose user_email matches the JWT email but whose user_id is a UUID, get_user_object resolves the legacy row via fuzzy lookup, but the JWT-claim user_id (the email) still flowed into team-membership lookup, JWTAuthBuilderResult.user_id, UserAPIKeyAuth and the spend tables. Spend was orphaned under a phantom email id; /user/info and the Usage page showed $0 for the legacy user (GH #26789). Treat the resolved user_object as the source of truth: add _canonical_user_id_from_db, rebind inside get_objects, and return effective_user_id so auth_builder unpacks it without adding statements. Fixes #26789 Co-authored-by: Cursor <[email protected]> * fix(jwt): log user_id rebind at DEBUG to avoid email PII in INFO streams Greptile review on #29217: rebinding often logs JWT email claims at INFO. Co-authored-by: Cursor <[email protected]> * test(jwt): update passthrough allowlist mock for 5-tuple get_objects Staging #29256 added a test that still mocked get_objects with a 4-tuple; our PR expanded the return to 5 values (effective_user_id). Co-authored-by: Cursor <[email protected]> --------- Co-authored-by: Cursor <[email protected]>
1 task
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
BerriAI#29256) * fix(proxy): enforce allowed_passthrough_routes for auth=true pass-through Pass-through endpoints with auth=true were injected into openai_routes, so teams with openai_routes access bypassed per-team allowed_passthrough_routes. Gate auth-enforced pass-through at JWT, virtual-key, and non-admin route checks. Co-authored-by: Cursor <[email protected]> * fix(proxy): clarify JWT passthrough denial Co-authored-by: Cursor <[email protected]> * fix(proxy): make pass-through auth checks method-aware Prevent allowlist bypass when the same path is registered with different auth settings per HTTP method. Co-authored-by: Cursor <[email protected]> * Fix passthrough route auth checks * fix(proxy): reject unregistered pass-through HTTP methods Enforce method-aware JWT checks and return 405 when stale FastAPI routes accept requests outside the current pass-through registry. Co-authored-by: Cursor <[email protected]> * fix(proxy): remove duplicate request_method in JWT team lookup Fixes SyntaxError on proxy startup caused by passing request_method twice to find_team_with_model_access. Co-authored-by: Cursor <[email protected]> * Fix passthrough route auth enforcement * fix(proxy): raise passthrough-specific 403 directly in virtual-key path * fix(proxy): load team for RBAC role-claim JWT passthrough gating * Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (BerriAI#28728)" (BerriAI#29326) This reverts the Bedrock CI account migration (BerriAI#28728). The original account (888602223428) was put under an AWS security restriction after a leaked key and has since been reactivated, while the replacement account (941277531214) lacks access to several models the suites exercise (legacy Bedrock Claude 3 models, Cohere, Nova Canvas image gen, Bedrock batch inference, and flagship Opus). Pointing CI back at the reactivated account restores that coverage. This is the exact inverse of BerriAI#28728: all hardcoded 941277531214 references go back to 888602223428 (provisioned/imported-model ARNs, AgentCore runtime ARNs and their suffixes, batch execution role ARN, and the example proxy config), the S3 buckets revert to litellm-proxy and load-testing-oct, the guardrail IDs revert to wf0hkdb5x07f and ff6ujrregl1q, the SageMaker endpoint and Knowledge Base revert to their original ids, and the live-call tests go back to the legacy model strings. The grid_spec fail_reason workaround for the unentitled Opus cells is dropped while keeping the unrelated bedrock_effort_ceiling field added after the migration. The CircleCI AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env vars still point at 941277531214 and must be set to the reactivated account's fresh credentials separately via the CircleCI API; AWS_REGION_NAME stays us-west-2. (cherry picked from commit f11c12d) * fix(proxy): scope pass-through 405 to registry routes; grant rerank passthrough in rpm tests The auth=true pass-through 405 guard fired for mapped provider routes (e.g. /assemblyai/*) that are not in the in-memory registry, since get_registered_pass_through_route returns None for them while is_registered_pass_through_route matches via mapped_pass_through_routes. Only raise 405 when the path is registered but the request method is not allowed, so mapped provider pass-throughs fall through to the default target params as before. The rpm-limit pass-through tests register /v1/rerank with auth=true but gave their keys no allowed_passthrough_routes, so the new default-deny returned 403 before the rate limiter ran (non-deterministically, depending on registry insertion order). Grant the keys explicit passthrough access so the tests exercise rate limiting under the new auth model. * fix(proxy): guard request method lookup against scopes without a method Starlette's Request.method property reads scope["method"] and raises KeyError when the scope omits it (e.g. minimally-constructed test requests). getattr only swallows AttributeError, so the new _get_request_method helper propagated the KeyError up through user_api_key_auth and surfaced as a ProxyException. Catch KeyError (and AttributeError) and fall back to None. * test(passthrough): pin SERVER_ROOT_PATH in unregistered-method test test_custom_proxy.py sets os.environ['SERVER_ROOT_PATH'] = '/my-custom-path' at module import with no cleanup. When that module is collected into the same xdist worker as this test, the leaked root path is prepended to registered pass-through paths, so is_registered_pass_through_route misses '/test/path' and the handler returns 404 instead of the expected 405 (order-dependent). Pin SERVER_ROOT_PATH to '' so the test is deterministic. * test(passthrough): restore regression coverage for non-auth-enforced pass-through via llm_api_routes * fix(proxy): record auth flag in pass-through registry for allowlist enforcement Auth-enforced pass-through detection inferred enforcement from the FastAPI dependency stored at registration time. The management create and update endpoints register routes with dependencies=None even though auth defaults to true, so is_auth_enforced_pass_through_route treated those DB-created routes as unenforced. A key allowed for llm_api_routes could then call a management-created auth-enabled pass-through route without matching allowed_passthrough_routes. Store the auth setting on each registry entry and read it directly when deciding whether the allowlist applies, instead of deriving it from dependency metadata. * fix(proxy): include bool in pass-through registry value type for auth flag The auth flag stored in _registered_pass_through_routes is a bool, which was not part of the registry value Union, so mypy rejected the dict literal. Add bool to the Union and narrow route_methods to a list before the membership check so the in-operator stays valid. * fix(proxy): preserve stored auth flag on pass-through endpoint update model_dump(exclude_none=True) re-included the auth=True default whenever a partial update omitted auth, silently flipping an existing auth=false pass-through to auth-enforced and 403ing every team/key without allowed_passthrough_routes. Merge only explicitly set fields via exclude_unset so omitted fields keep their stored value. --------- Co-authored-by: Cursor <[email protected]> Co-authored-by: mateo-berri <[email protected]>
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
…rriAI#29217) * fix(jwt): attribute spend to resolved DB user_id on email/sso fuzzy match When user_id_upsert is enabled with JWT auth and a pre-migration user row exists whose user_email matches the JWT email but whose user_id is a UUID, get_user_object resolves the legacy row via fuzzy lookup, but the JWT-claim user_id (the email) still flowed into team-membership lookup, JWTAuthBuilderResult.user_id, UserAPIKeyAuth and the spend tables. Spend was orphaned under a phantom email id; /user/info and the Usage page showed $0 for the legacy user (GH BerriAI#26789). Treat the resolved user_object as the source of truth: add _canonical_user_id_from_db, rebind inside get_objects, and return effective_user_id so auth_builder unpacks it without adding statements. Fixes BerriAI#26789 Co-authored-by: Cursor <[email protected]> * fix(jwt): log user_id rebind at DEBUG to avoid email PII in INFO streams Greptile review on BerriAI#29217: rebinding often logs JWT email claims at INFO. Co-authored-by: Cursor <[email protected]> * test(jwt): update passthrough allowlist mock for 5-tuple get_objects Staging BerriAI#29256 added a test that still mocked get_objects with a 4-tuple; our PR expanded the return to 5 values (effective_user_id). Co-authored-by: Cursor <[email protected]> --------- Co-authored-by: Cursor <[email protected]>
This was referenced Jun 26, 2026
Closed
timothybrush
pushed a commit
to timothybrush/litellm
that referenced
this pull request
Jun 26, 2026
…=true test reaches rpm path BerriAI#29256 made auth=true pass-through routes deny-by-default unless the key/team has allowed_passthrough_routes configured, but this integration test was not updated. The test key had no allowlist, so the auth=true parametrizations (rpm_limit=0 -> expect 429, rpm_limit=2 -> expect 207) now hit the 403 gate in auth before reaching the rpm/forwarding logic they mean to exercise. Grant the test key allowed_passthrough_routes for /api/public/ingestion so it clears the gate. Also removes a latent order-dependency: the case only passed locally when an earlier (auth=false) parametrization registered the route first; under worker isolation (CI xdist) it failed with 403.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves LIT-3255
Description
Pass-through endpoints configured with auth: true are registered in LiteLLMRoutes.openai_routes so they participate in spend/budget hooks. That registration also made them look like standard OpenAI/LLM API routes, so any team or key with openai_routes / llm_api_routes access could call them without matching allowed_passthrough_routes.
This change keeps those routes on openai_routes for budgeting, but enforces per-team/key allowed_passthrough_routes at runtime for auth-enforced pass-through only.
Cause
On registration, auth: true appends the pass-through path to LiteLLMRoutes.openai_routes.
Auth then short-circuits in non_proxy_admin_allowed_routes_check at is_llm_api_route(), which returns true for those paths. check_passthrough_route_access() is never reached.
With default JWT team_allowed_routes: ["openai_routes", "info_routes", "mcp_routes"], every team with a valid token effectively gained access to all auth: true pass-through endpoints, regardless of allowed_passthrough_routes metadata.
Fix
Add is_auth_enforced_pass_through_route() — detects config/DB pass-through routes registered with auth: true (via non-null FastAPI dependencies).
Add _require_auth_pass_through_access() — requires an explicit allowed_passthrough_routes match on key or team metadata; otherwise returns 403.
non_proxy_admin_allowed_routes_check: run auth-enforced pass-through check before the blanket is_llm_api_route() allow.
is_virtual_key_allowed_to_call_route: when openai_routes / llm_api_routes would allow a registered pass-through, require allowlist match if auth: true.
JWT find_team_with_model_access: after allowed_routes_check passes for openai_routes, re-check check_passthrough_route_access against team metadata for auth-enforced pass-through routes.
Breaking change
Default-deny for auth: true pass-through: teams/keys must have allowed_passthrough_routes configured (admin-set) to call those endpoints. Existing deployments relying on blanket openai_routes access need to add explicit allowlists.
Before:
Team key with with or without passthrough access via team. All requests succeed.
After:
Team key with no passthrough access via team.
Team key with passthrough access via team.
Note
High Risk
Changes authentication/authorization for pass-through routes with a breaking default-deny; misconfigured allowlists will block production traffic until team/key metadata is updated.
Overview
Auth-enforced pass-through endpoints (
auth: true) still register onopenai_routesfor spend/budget hooks, but no longer inherit blanketopenai_routes/llm_api_routesaccess. Runtime checks now require an explicitallowed_passthrough_routesmatch on the key or team metadata.RouteChecksaddsis_auth_enforced_pass_through_route(registered pass-through with non-null FastAPIdependencies) and_require_auth_pass_through_access. Non–proxy-admin and virtual-key paths run this check before the usualis_llm_api_routeallow. JWT team selection infind_team_with_model_accessre-validates passthrough using team metadata and the HTTP method, with a dedicated 403 when the allowlist fails (instead of a generic model error).Method-aware registry lookup distinguishes same-path routes with different verbs; the pass-through handler returns 405 when the method is not in the current registry (e.g. stale routes after config changes). JWT auth now receives
request_methodfrom the incoming request.Breaking: teams/keys that relied on broad
openai_routesaccess must configureallowed_passthrough_routesforauth: truepass-through calls.Reviewed by Cursor Bugbot for commit 36f43c0. Bugbot is set up for automated code reviews on this repo. Configure here.