* fix(guardrails): restore disable_global_guardrails persistence for keys

The per-key/team "Disable Global Guardrails" toggle silently stopped
working after #17042, which removed `disable_global_guardrails` from the
key/team request models and from the premium metadata allowlist. Without
those, the UI's top-level field was dropped by pydantic and never folded
into key `metadata`, so the runtime gate always read False and global
default_on guardrails kept running.

Restore the request-model fields (KeyRequestBase, NewTeamRequest,
UpdateTeamRequest) and the `LiteLLM_ManagementEndpoint_MetadataFields_Premium`
entry so the flag is promoted into metadata again. Because the key edit
form always submits the flag (false by default), guard the UI so it is
only sent when it actually changed (edit) or is enabled (create) — this
keeps the premium gate on enabling intact while not 403-ing non-premium
users who edit unrelated key fields, mirroring how guardrails/tags are
already stripped.

* test(guardrails): cover disable_global_guardrails toggle-off + clarify premium field comment

Add a prepare_metadata_fields case asserting `disable_global_guardrails: False`
overwrites an existing `True`, and rewrite the PREMIUM_METADATA_FIELDS comment to
explain why boolean premium fields are excluded from the empty-value strip loop.

* test(e2e): cover Team Admin view + member + key flows

Adds a new spec exercising the previously-uncovered team-admin manual-QA
items: viewing all team keys (including other members'), adding a member,
removing a member, and creating a team key with All Team Models. Also
seeds a dedicated invitee user so the add-member test can run in parallel
with the proxy-admin invite test without colliding on the team roster.

* test(e2e): harden team-admin member specs per review feedback

Address Greptile feedback on the Team Admin spec:
- locate the delete action via getByTestId("delete-member") instead of
  the fragile svg/img .last() selector
- match the seeded removable member by user_id (members_with_roles stores
  no email, so the roster renders user_id)
- assert exact success-toast strings rather than broad regexes that could
  match unrelated "success" text

…9252)

* docs: replace generated CLAUDE.md with hand-written guidance, remove AGENTS.md

Swap the auto-generated CLAUDE.md for a concise hand-written version that captures how we actually want agents to work in this repo: minimal comments, simplicity first, meaningful tests with a high mutation kill rate, PRs based off litellm_internal_staging rather than main, and curl against a live proxy as proof of fix instead of pasted pytest output. Remove AGENTS.md so there is one source of truth for agent guidance. The customer and company name confidentiality policy, along with the MCP available_on_public_internet note, are carried over from the previous CLAUDE.md.

* fix: further clarify communication guidelines

* docs: point GEMINI.md at CLAUDE.md instead of duplicating guidance

Replace the standalone GEMINI.md copy, which had already drifted from the new CLAUDE.md, with a one-line pointer so Gemini reads the same single source of truth.

* docs: simplify PR template test checklist item

Replace the rigid "at least 1 test is a hard requirement" checklist line with "I have added meaningful tests", which matches the testing guidance in CLAUDE.md, and tidy a comma into a semicolon in the scope-isolation item.

* docs: point AGENTS.md at CLAUDE.md instead of deleting it

Keep AGENTS.md so tools that read it still resolve guidance, but collapse it to the same one-line pointer to CLAUDE.md used by GEMINI.md, keeping a single source of truth.

* fix: make AI-generated rules more concise

* fix: spelling

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: make the .env usage more careful

* docs: restore MCP available_on_public_internet note to CLAUDE.md

The PR description states this note was carried over verbatim from the
previous CLAUDE.md, but it was dropped in the rewrite. Restore it so the
file matches the description and the team guidance is not lost.

* docs: restore browser storage and CI supply-chain safety notes to CLAUDE.md

These security-relevant rules were dropped in the rewrite. Restore the
sessionStorage-over-localStorage (XSS) guidance and the CI supply-chain
rules (no curl|bash, pin versions, verify checksums) so agents editing UI
or CI code are still steered away from those pitfalls.

* docs: move area-specific guidance into nested CLAUDE.md files

The MCP, browser-storage, and CI supply-chain notes are scoped to
particular parts of the tree, so move each into a nested CLAUDE.md that
Claude Code loads on demand when those files are touched: the MCP note
under the mcp_server gateway, the browser-storage rule under the UI
dashboard, and the CI supply-chain rules under .circleci. Keeps the root
CLAUDE.md focused on general guidance while the area notes surface where
they are relevant.

* docs: keep CI supply-chain note in root CLAUDE.md

CI guidance applies beyond .circleci (it also covers downloads in GitHub
workflows and any CI script), and CI work does not reliably touch a single
subtree, so a nested file under .circleci would not surface it dependably.
Keep it in the always-loaded root instead. The MCP and browser-storage
notes stay nested where they map cleanly to one area of the tree.

* fix: make it clear we prefer httpOnly

* chore: make ci rule more concise

* chore: make concise

Fix formatting and punctuation in MCP note.

* fix: don't include Claude attribution

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

…badge (#28502)

… for Claude Code on Sonnet/Opus 4.6 (#29304)

* fix(anthropic): don't inject output_config.effort=xhigh on models without xhigh

The legacy-thinking translator on the /v1/messages route mapped any
thinking.budget_tokens >= 24000 to effort=xhigh and injected it into
output_config without checking model support. Claude Code's default
thinking budget (31999) hit this bucket, so Sonnet 4.6 (and Opus 4.6)
on Bedrock/Vertex started returning

  400 output_config.effort: Input should be 'low', 'medium', 'high' or 'max'

Gate the xhigh choice on _supports_effort_level(model, "xhigh"), the
same capability check the reasoning_effort path already uses. Models
that advertise xhigh (Opus 4.7) keep it; everything else falls to high.

Fixes #29282

* test(anthropic): pin Opus 4.6 in legacy-thinking xhigh-clamp regression test

Opus 4.6 (bare, bedrock/invoke, vertex_ai) has supports_adaptive_thinking
but no supports_xhigh_reasoning_effort, so it hits the same clamping path as
Sonnet 4.6. It was named in the PR scope but lacked a pinned regression
guard; add the three variants to the parametrize list.

* test(e2e): cover Internal Viewer nav, key, and team-info gating

Three previously-uncovered manual-QA paths for the Internal Viewer role:

- Nav only renders the read-only sections; admin-only items
  (Internal Users, Organizations, Models + Endpoints) stay hidden.
- Virtual Keys page hides Create New Key, and the key detail view
  hides Regenerate / Reset Spend / Delete actions.
- Team info page hides Members and Settings tabs for the viewer.

* test(e2e): scope viewer nav to sidebar, strengthen tab assertions

Address review feedback on the Internal Viewer e2e spec:
- Scope the nav test to the sidebar complementary landmark and match
  items by link role + accessible name. The prior CSS nav, aside
  selector grabbed the top bar (the sidebar is a complementary
  landmark, not a <nav> tag), so the assertions never hit the real
  nav links.
- Land via navigateToPage so the networkidle wait settles the
  role-gated nav before asserting.
- Assert the Virtual Keys tab is visible (was only commented).
- Use toHaveCount(0) for hidden team tabs to match the nav block;
  tabs are conditionally rendered, not CSS-hidden.
- Drop redundant dismissFeedbackPopup calls (navigateToPage already
  dismisses internally).

* test(e2e): cover Internal User key modal, team info, key page

Three previously-uncovered manual-QA paths for the Internal User role:

- Create Key modal — confirm the team dropdown is populated with the
  user's teams (verifying the role-scoped UI flow exists).
- Team info page — confirm the Settings/Members tabs are hidden for a
  regular team member; only the read-only tabs render.
- Virtual Keys page — confirm the proxy's internal litellm-dashboard
  team keys never leak into an internal user's table.

* test(e2e): share clickTeamId helper, strengthen key-filter assertion

Address review feedback on the Internal User e2e spec:
- Extract clickTeamId into helpers/navigation.ts; import in both
  internalUser and teams specs instead of duplicating it.
- Anchor the litellm-dashboard absence check on the user's own seeded
  key so it cannot pass vacuously against an empty table.
- Drop redundant dismissFeedbackPopup calls (navigateToPage already
  dismisses internally).

* test(e2e): cover navbar Logout flow as proxy admin

The Logout button under the navbar User dropdown was an uncovered
manual-QA step. This test signs in as admin, opens the dropdown,
clicks Logout, then navigates to a protected page and asserts the
redirect to /ui/login — proving the session was cleared.

* test(e2e): fix logout dropdown trigger and account-menu selector

The button never rendered the literal text "User" (it shows initials +
display name), and the antd Dropdown uses trigger={["click"]}, so the
synthetic mouseover/mouseenter never opened the popup. Open it with a real
click on the button's aria-label ("Account menu — ...").

* fix(mcp): resolve key.access_group_ids → MCP servers (ungated)

A teamless virtual key whose unified access_group_ids grant an
MCP-granting access group now sees and can call that server instead of
getting an empty list / 403. The key path previously read only the
legacy object_permission; this folds key.access_group_ids into the
key's base scope (ungated), mirroring can_key_call_model's fallback.
The gated assigned_*-checked override is unchanged.

* fix(mcp): expand name/alias entries in key access-group servers

access_mcp_server_ids may hold server names/aliases, not just ids. The
new ungated key path now runs them through expand_permission_list at the
source, so the early-return and union branches both surface resolved ids
— matching the legacy object_permission path and the gated extras path.

…29273)

* fix(router): enforce deployment budgets for dynamically added models

Register deployment max_budget/budget_duration when models are added via
upsert_deployment (e.g. /model/new) so RouterBudgetLimiting matches startup
model_list behavior.

Co-authored-by: Cursor <[email protected]>

* fix(router): address CI lint and router coverage for budget sync helpers

Remove unused RouterBudgetLimiting import and add router unit tests for
deployment budget helper methods required by router_code_coverage.

Co-authored-by: Cursor <[email protected]>

* fix(router): clear stale deployment budget on upsert without limits

Unregister deployment budget config when max_budget/budget_duration are
removed, including upsert replace paths. Hoist provider budget logger
lookup outside the provider loop.

Co-authored-by: Cursor <[email protected]>

* Fix mypy

* fix black

---------

Co-authored-by: Cursor <[email protected]>

…9264)

* fix(proxy): map stripped batch body.model to proxy alias for auth

replace_model_in_jsonl rewrites JSONL body.model to the provider id before
upload; batch file access checks must resolve that id back to model_name
so keys granted the proxy alias are not rejected with 403.

Co-authored-by: Cursor <[email protected]>

* fix(proxy): surface resolved proxy alias in batch file 403 detail

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

…ing (#26857)

* feat(mcp): support stateless and stateful clients via session-id routing

- Add session_manager_stateful (stateless=False) alongside stateless
- Route by mcp-session-id: has ID → stateful, initialize (no ID) → stateful, else → stateless
- Peek POST body to detect initialize for routing; replay via wrapped receive
- Handle stale session IDs for both managers
- Add test_mcp_routing_initialize_to_stateful_no_session_to_stateless
- Update test_valid_mcp_session_id_is_preserved, test_concurrent_initialize_session_managers

Made-with: Cursor

* fix(mcp): respect stateful routing and harden initialize detection

Ensure streamable MCP requests are dispatched via the computed target session manager, and guard initialize detection against non-object JSON bodies. Update stale-session test patches to target the stateful manager so routing assertions remain correct.

Made-with: Cursor

* test(mcp): patch stateless/stateful managers in concurrency init test

Update concurrent session-manager initialization test to patch session_manager_stateless and session_manager_stateful directly, matching initialize_session_managers() behavior and preventing NameError from undefined mocks.

Made-with: Cursor

* Fix tests

* Fix tests

* Fix MCP stateful routing edge cases

* Fix stateful MCP auth context refresh

* Fix MCP stateful session cleanup

* fix(mcp): bind stateful sessions to creator and reject hijacks

Stateful mcp-session-id was usable by any authenticated proxy caller. Track
the session creator's hashed API key (or user_id) when a new session is
issued and reject mismatched callers with 403 before _set_or_update_auth_context
overwrites the stored MCPAuthenticatedUser. Also formats nested with-statements
in test_mcp_stale_session.py and fixes a pre-existing AsyncMock mismatch in
test_stale_mcp_session_id_is_stripped.

* fix(mcp): serialize concurrent requests on same stateful session

Bugbot's 'Concurrent requests share context' finding: _update_auth_context
mutates the single MCPAuthenticatedUser stored per session in place on
every request, so two requests sharing one mcp-session-id can overwrite
each other's mcp_servers / auth headers / oauth state / client_ip while
in-flight callbacks are still reading the same object.

Owner-binding alone narrows this to same-principal racing, but the
in-place mutation race remains. Add a per-session asyncio.Lock around
handle_request so concurrent same-session requests run sequentially. The
lock is allocated on demand and torn down with the rest of the session
state on DELETE / idle expiry.

Co-authored-by: Mateo Wang <[email protected]>

* fix(mcp): include OAuth2 bearer in stateful session owner fingerprint

UserAPIKeyAuth() for OAuth2 passthrough has no api_key/user_id, so every
OAuth caller fingerprinted to "anonymous" and could hijack another OAuth
caller's mcp-session-id. Hash the upstream Authorization header into the
fingerprint as oauth:<sha256>.

* fix(mcp): don't hold stateful session lock for streaming GETs

The per-session lock wraps handle_request, so a long-lived GET (SSE
stream held open for the life of the session) would block every
subsequent POST on the same mcp-session-id. Only POST/DELETE mutate the
shared MCPAuthenticatedUser, so it's sufficient to serialize those —
GETs run lock-free and stream concurrently.

* fix(mcp): allow None user_api_key_auth in MCPAuthenticatedUser

The set_auth_context / _set_or_update_auth_context / _update_auth_context
helpers in server.py all accept Optional[UserAPIKeyAuth] and pass it
straight into MCPAuthenticatedUser, but the dataclass-style constructor
typed user_api_key_auth as required UserAPIKeyAuth. Mypy flagged this on
the stateful-routing branch:

  server.py:3227: error: Incompatible types in assignment (expression
    has type "UserAPIKeyAuth | None", variable has type "UserAPIKeyAuth")
  server.py:3255: error: Argument "user_api_key_auth" to
    "MCPAuthenticatedUser" has incompatible type "UserAPIKeyAuth | None";
    expected "UserAPIKeyAuth"

Widen the parameter type to Optional[UserAPIKeyAuth] to match the call
sites. Runtime behavior is unchanged.

Co-authored-by: Mateo Wang <[email protected]>

* style: replace with new alias

* fix(mcp): fall back to client_ip in stateful session owner fingerprint

Addresses Greptile review on PR #26857: when no API key, user_id, or
OAuth bearer is available (e.g. unauthenticated/passthrough callers),
the owner fingerprint collapsed to a single 'anonymous' value, allowing
two unrelated callers to drive each other's stateful MCP sessions.

Fold client IP into the fingerprint as a fallback identity signal so
distinct anonymous sources do not share an owner identity.

Co-authored-by: Mateo Wang <[email protected]>

* Fix active stateful MCP session cleanup

* test(mcp): cancel leaked stateful auth-context cleanup task

initialize_session_managers() spawns a real asyncio.create_task running
_cleanup_expired_stateful_session_auth_contexts(). The
test_concurrent_initialize_session_managers test was saving and
restoring the session-manager context-manager globals but did not save,
cancel, or restore _stateful_auth_context_cleanup_task.

Because pyproject.toml sets asyncio_default_fixture_loop_scope=session,
the event loop is shared across tests in the same session, so the
leaked task kept running against module-level dicts for the rest of the
test run. Save and cancel the task in the finally block so the test
fully cleans up after itself.

Co-authored-by: Mateo Wang <[email protected]>

* Fix stateful MCP session fingerprinting

* Hash MCP session user owner fingerprints

* Fix stale MCP session DELETE cleanup

* fix(mcp): harden owner fingerprint hashing for non-str api keys

_owner_fingerprint_for assumed api_key/user_id supported .encode();
MagicMock-based tests (and any non-str truthy values) crashed with
TypeError before routing. Only hash str/bytes secrets; fall through
otherwise so MCP routing and session tests behave correctly.

Co-authored-by: Cursor <[email protected]>

* Fix MCP stateful cleanup loop resilience

* Fix stateful MCP initialize auth capture

* fix(mcp): drop orphan per-session lock when auth context absent

Defensive cleanup for _stateful_session_locks entries created on
sessions that never enter _stateful_session_auth_contexts. The
periodic cleanup loop only iterates auth_context_last_seen, so such
locks would otherwise live forever. Add a test that reproduces the
leak and verifies the request finalizer pops the lock.

Co-authored-by: Mateo Wang <[email protected]>

* chore(mcp): trim verbose comment on lock cleanup

Co-authored-by: Mateo Wang <[email protected]>

* Fix stateful MCP delete failure tracking

* fix test

* fix(mcp): cap routing-peek body size to bound pre-dispatch memory

Authenticated clients that POST without an mcp-session-id forced the proxy
to buffer the entire request body before routing, since the peek loop
drained every body chunk to decide whether the JSON-RPC method was
'initialize'. Cap the peek at 4 KB (more than enough for an initialize
envelope) and let the remainder stream through wrapped_receive into the
downstream handler.

* test: replace dall-e-3 with gpt-image-1 in health check and router tests (#27813)

OpenAI returns 'The model dall-e-3 does not exist' for the test account,
breaking test_openai_img_gen_health_check and test_image_generation.
Switch to gpt-image-1, matching the existing TestOpenAIGPTImage1 pattern.

* fix(tests): drop dall-e-only test classes; route live image tests via gpt-image-1

Second wave of failures from the 2026-05-12 DALL-E shutdown:
- tests/image_gen_tests/test_image_edits.py::TestOpenAIImageEditDallE2
  and tests/image_gen_tests/test_image_generation.py::TestOpenAIDalle3
  are explicitly named for the deprecated models and can't pass; remove.
  gpt-image-1 coverage already exists in sibling classes.
- tests/local_testing/test_router.py image gen tests use dall-e-3 only
  as a routing example; swap to gpt-image-1.
- tests/local_testing/test_custom_callback_input.py image_generation
  success/failure paths swapped to gpt-image-1.

* Fix MCP initialize session active tracking

Co-authored-by: Yassin Kortam <[email protected]>

* Fix MCP reinitialize session tracking

Co-authored-by: Yassin Kortam <[email protected]>

* Fix MCP reinitialize auth context aliasing

Co-authored-by: Yassin Kortam <[email protected]>

* Apply black formatting after merge

Co-authored-by: Mateo Wang <[email protected]>

* Run owner-binding 403 before consuming POST body

Co-authored-by: Mateo Wang <[email protected]>

* Harden MCP routing peek bound and stateful purge race

Co-authored-by: Mateo Wang <[email protected]>

* Remove inadvertently committed Next.js build artifacts

Co-authored-by: Mateo Wang <[email protected]>

* Run owner check before stale MCP session cleanup

Co-authored-by: Mateo Wang <[email protected]>

* fix(mcp): reverse cleanup ordering to terminate transport before clearing owner

Reverses _purge_expired_stateful_session_auth_contexts so the transport
is popped from server_instances and terminated BEFORE owner/auth tracking
is cleared. The previous order left a window where _stateful_session_owners
was already empty but server_instances still served the session, so a
concurrent request would observe expected_owner is None and bypass the
owner-binding check. Addresses Greptile review on PR #26857.

Co-authored-by: Mateo Wang <[email protected]>

* test(mcp): fully reset stateful session tracking in auth-context refresh test

Use _remove_stateful_session_tracking in teardown so the test no longer
leaks _stateful_session_auth_context_last_seen and _stateful_session_locks
between tests, matching the cleanup used by the sibling stateful tests.

* fix(mcp): cap concurrent stateful sessions per caller to bound memory

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: mateo-berri <[email protected]>
Co-authored-by: Mateo Wang <[email protected]>
Co-authored-by: Sameerlite <[email protected]>
Co-authored-by: yuneng-jiang <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>
Co-authored-by: Claude Babysitter <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

* Fix overiding of fastapi_response headers

* fix(bedrock): support tool search results and surface citations as annotations

Add an optional tool-message search_results path that maps directly to Bedrock toolResult.searchResult blocks, and convert Converse citationsContent into chat completion annotations for user-facing citation metadata.

Co-authored-by: Cursor <[email protected]>

* fix(format): align bedrock prompt factory with black

Reformat the updated bedrock prompt template conversion file so CI black --check passes.

Co-authored-by: Cursor <[email protected]>

* fix(bedrock): harden citations, search_results mapping, and token counting

Resolve mypy issues in citation parsing, only attach url_citation annotations when citation text is stitched into content, fall back to tool content when search_results is empty, and count search_results text in token/TPM preflight paths.

Co-authored-by: Cursor <[email protected]>

* fix(bedrock): extract tool result helpers to satisfy PLR0915

Refactor _convert_to_bedrock_tool_call_result into smaller helpers so lint passes without changing Bedrock tool result behavior.

Co-authored-by: Cursor <[email protected]>

* fix(bedrock): count all forwarded search_results fields in token estimates

Include source, title, content text, and citations when estimating tokens so large metadata cannot bypass TPM preflight checks. Reformat factory.py with black.

Co-authored-by: Cursor <[email protected]>

* fix(managed-files): skip content blocks without a type key in get_file_ids_from_messages

* fix(bedrock): stitch citations for any punctuation-only text block

* fix(bedrock): map null citation source/title to empty annotation strings

* fix(bedrock): advance citation offset for text-only citationsContent blocks

* fix(bedrock): complete citation TypedDicts for grounding annotations

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

* fix(mcp): ignore stale server ids during key permission validation

Prevent virtual key save failures when object_permission still includes MCP server IDs that were deleted from the registry. The validator now drops stale IDs before team scope checks and adds coverage for stale-ID and active unauthorized-ID behavior.

Co-authored-by: Cursor <[email protected]>

* fix(mcp): use DB find_many to detect stale server IDs, add tool-permission test

Addresses greptile feedback on PR #29128:

P1 – Race-condition / authoritative-signal concern:
  Extract _get_stale_mcp_server_ids() helper that performs a single
  DB find_many (LiteLLM_MCPServerTable) when prisma_client is available,
  making server-existence checks authoritative even during in-memory
  registry warm-up. Falls back to the in-memory registry only when no
  DB client is present (config-only deployments / unit tests).
  Pass prisma_client through both call sites in key_management_endpoints.

P2 – Missing test coverage for mcp_tool_permissions:
  Add test_validate_stale_ids_in_mcp_tool_permissions_silently_dropped
  to confirm stale server IDs that appear only as keys in
  mcp_tool_permissions are also stripped without raising 403.

Co-authored-by: Cursor <[email protected]>

* fix(mcp): also check in-memory registry when DB is present to avoid misclassifying config-file servers as stale

Config-file MCP servers are never written to LiteLLM_MCPServerTable, so a
DB-only query would classify them as stale and silently drop their IDs from
the authorization check, allowing unauthorized key access.

Fix: treat a server ID as stale only when it is absent from BOTH the DB and
the in-memory registry (which holds config-file servers).

Co-authored-by: Cursor <[email protected]>

* fix(mcp): normalize server aliases on key save

Co-authored-by: Sameer Kankute <[email protected]>

* fix: remove unused stale MCP helper and capture normalized object_permission on key generation

Co-authored-by: Yassin Kortam <[email protected]>

* test(mcp): keep validate_key_mcp_servers_against_team stub in sync with return contract

validate_key_mcp_servers_against_team now returns the (normalized) object_permission, and the key-generation helper assigns that return value back into the request data. The two key-generation tests stubbed the function with a bare AsyncMock, whose default MagicMock return value clobbered object_permission and skipped permission-record creation. Make the stubs pass object_permission through unchanged.

* fix(mcp): preserve provided object_permission fields on key update

The key-update path reconstructs data.object_permission from a full
model_dump(), which marks every field as set. Downstream
model_dump(exclude_unset=True) then emits models/blocked_tools/
search_tools as None, and those are non-nullable array columns, so the
Prisma write fails whenever the UI submits object_permission with only a
subset of fields populated (e.g. a TPM/RPM-only edit). Build the dict
with exclude_unset=True so the normalized object_permission keeps the
caller's original field set.

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: Sameer Kankute <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

…28860)

* feat(a2a): well-known agent-card discovery + LangGraph Platform mode

Adds a registration-time discovery flow so admins can paste an upstream
agent URL, see its skills/capabilities, pick what to expose, and have the
proxy front it with a LiteLLM-shaped agent card.

Backend (new litellm/proxy/a2a/ module):
- fetch_well_known_card walks /.well-known/agent-card.json,
  /.well-known/agent.json, /agent.json by default. langgraph_platform
  mode hits the canonical path with ?assistant_id=<id> (LangGraph
  serves one shared endpoint per deployment).
- merge_agent_card overlays LiteLLM overrides on the upstream card:
  drops upstream url, forces protocolVersion=1.0, replaces
  securitySchemes with LiteLLMKey bearer, emits supportedInterfaces
  pointing at the proxy, filters capabilities to a small allowlist,
  strips non-v1.0 fields.
- POST /v1/a2a/discover returns the raw upstream card (admin-only) so
  the UI can render skills/capabilities for selection.
- create/update/patch agent endpoints pre-generate the agent_id and
  run merge_agent_card before storing, so DB.agent_card_params already
  embeds the proxy-fronted URL.

UI (ui/litellm-dashboard):
- New AgentCardDiscovery component with a parent-driven plan:
  discovery_mode + params + display URL. For LangGraph the parent
  composes (api_base, assistant_id); for pure A2A it uses the url
  field. Component hides the manual URL input when the parent drives.
- add_agent_form wires discovery for every non-custom agent type and
  overlays the user's selections onto agent_card_params at submit,
  fixing the bug where dynamic agent forms ignored discovery picks.

Completion-bridge fixes (paired):
- Add kind: "message" to A2A response messages and unwrap result
  so it's a Message directly per spec (matches a2a SDK
  SendMessageResponse validation).
- Forward A2A metadata to LangGraph runs via extra_body.metadata.

* fix(a2a): preserve agent url, fix streaming chunk envelope, and protect forwarded metadata

- Streaming chunk: move final out of the message object into the
  result envelope per the A2A spec.
- Agent card merge: keep upstream url on the stored card so the
  runtime invocation path can locate the upstream backend; the public
  well-known endpoint already rewrites this field to the proxy URL
  before exposing it to clients.
- Completion bridge: apply A2A forward metadata after merging
  litellm_params so an agent-configured extra_body cannot
  overwrite the forwarded metadata.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(a2a): fix legacy streaming chunk, agent card test, and metadata merge

- providers/litellm_completion: move 'final' out of the message object
  into the result envelope per the A2A spec (matches the bridge fix).
- agent endpoints test: the runtime invocation path now preserves the
  top-level 'url' on the stored card, so update the assertion to match.
- completion bridge metadata: when forwarding A2A metadata via
  extra_body.metadata, merge into any existing extra_body.metadata
  instead of replacing it, so an agent-configured metadata block is
  preserved (forward metadata still wins on key conflicts).

Co-authored-by: Yassin Kortam <[email protected]>

* fix(a2a): remove dead duplicate transformation dir; drop SSRF-prone headers field from /v1/a2a/discover

Co-authored-by: Yassin Kortam <[email protected]>

* fix(a2a): revert accidental html→index.html rename from afc8b10

The commit afc8b10 bundled real A2A fixes alongside an unintended
re-introduction of the */index.html layout that 8513d7f had already
reverted. Restore all 35 static-export pages back to the flat *.html
structure that matches the upstream main branch.

Co-authored-by: Cursor <[email protected]>

* fix(a2a): address PR review comments

UI:
- Auto-trigger discovery when connection details are filled; remove
  the "Use these selections" button (selection syncs live to parent,
  user just clicks Next).
- Edit Settings: auto-discover upstream card on open; cross-check with
  DB-stored card so only already-saved skills/capabilities are pre-ticked.
- Extract shared buildDiscoveryRequest + selectionsFromSavedAgentCard
  helpers into agent_discovery_utils.ts so both add and edit flows share
  the same logic.

Backend:
- agent_card.py: rename the proxy security requirements field from the
  non-standard ``securityRequirements`` to the spec-correct ``security``
  key (matches AgentCard TypedDict and A2A/OpenAPI convention).
- agent_card.py: remove ``securityRequirements`` from _ALLOWED_TOP_LEVEL_KEYS.
- endpoints.py: _build_merged_agent_card now forwards agent_name and
  description from the request so the stored card reflects the admin-
  supplied name, not just whatever the upstream card advertised.
- utils.py: remove overly-broad ``or "parts" in result`` fallback; use
  ``kind == "message"`` check only to avoid false matches on future
  result types that happen to include a ``parts`` field.
- test_agent_card.py: update assertions to expect ``security`` key.

Co-authored-by: Cursor <[email protected]>

* fix: restore Next.js metadata directories to match upstream main

The previous revert removed __next.* metadata subdirectories from git
tracking entirely, but these directories exist on origin/main alongside
the flat .html files. Restore them via checkout from origin/main so the
PR diff only reflects actual code changes.

Co-authored-by: Cursor <[email protected]>

* fix(a2a): drop dead headers option from discoverAgentCardCall

The backend /v1/a2a/discover endpoint no longer accepts a headers field
(removed in 78591b2 for SSRF safety), so any headers passed through
DiscoverAgentCardOptions were silently discarded by the API request
body. Remove the field and the conditional that copies it onto the
request body.

* fix(a2a): skip merge for non-A2A agents and align pydantic-ai result shape

The agent create/update/patch handlers ran the LiteLLM-fronting merge
unconditionally, so registrations that did not provide
agent_card_params still ended up with a synthesised card carrying
supportedInterfaces, securitySchemes, and default skills. Gate the
merge on a non-empty agent_card_params so plain chat/LLM agents stay
non-A2A in the registry.

Also move kind: 'message' inside the a2a_message dict in the Pydantic
AI non-streaming response so its construction matches the completion
bridge rather than spreading kind on top of a separate dict.

* Fix three bugs in A2A discovery flow

1. UI: Stabilize discoveryRequest deps to avoid redundant /v1/a2a/discover
   API calls. The parent rebuilds the discoveryRequest object on every form
   keystroke, so depend on primitive proxies (discovery_mode + serialized
   params) rather than the object identity. Read the actual object via a
   ref inside handleDiscover.

2. Backend: Route the well-known card fetch through async_safe_get so the
   admin /v1/a2a/discover endpoint can't be used to probe private/loopback
   addresses or cloud metadata endpoints. SSRFError is a separate handled
   case so it surfaces a clear AgentCardDiscoveryError.

3. Streaming: Make openai_chunk_to_a2a_chunk emit the same flat result
   shape as the non-streaming response (kind/role/parts/messageId at the
   result level), with envelope-level 'final' added. Matches the existing
   create_artifact_update_event pattern and lets consumers read a uniform
   result shape across streaming and non-streaming.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(a2a/ui): include savedAgentCard in handleDiscover deps

The previous deps list omitted savedAgentCard, so handleDiscover (and
the resetSelections it calls) kept the closure's saved-card value even
after the parent refetched the agent. Clicking 'Re-discover' would
then pre-select skills against stale data. Adding savedAgentCard to
the deps array forces the callback to refresh whenever the saved card
changes.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(a2a): align pydantic-ai test + docstring with direct-Message result shape

The non-streaming A2A response was changed so that `result` is the Message
itself (kind="message"), per spec / SendMessageResponse. Update the
PydanticAITransformation._transform_to_a2a_response test and docstring that
still described the old `result.message` envelope so internal consumers
match the producer.

* fix(a2a): strip additionalInterfaces and let configured metadata win over A2A request

- merge_agent_card no longer carries upstream additionalInterfaces through;
  storing those alternate URLs would let authenticated agent callers reach
  the backend directly and bypass proxy auth/budget/logging.
- apply_forward_metadata_to_completion_params now layers client-supplied A2A
  metadata UNDER any agent-owner-configured extra_body.metadata, so server-set
  run metadata stays authoritative on key conflicts.

* fix(agents): merge agent card even when agent_card_params is an empty dict

Treat an explicitly provided empty agent_card_params ({}) as 'card
provided but empty' instead of 'no card', so the LiteLLM-fronting merge
still injects securitySchemes, supportedInterfaces, and protocolVersion.
Without this, the well-known endpoint could serve a bare card with only
a rewritten url, advertising no authentication to A2A clients.

Co-authored-by: Yassin Kortam <[email protected]>

* refactor(a2a): drop dead openai_chunk_to_a2a_chunk helper

The deprecated single-chunk helper has no callers anywhere in the
codebase — the streaming path emits proper A2A events via
create_task_event / create_status_update_event /
create_artifact_update_event in handler.py. Removing the dead method
also eliminates the inconsistency where the unused chunk inlined the
envelope-level final flag inside the Message result.

* fix(a2a): scope a2a lazy-feature so it doesn't subsume /v1/a2a/discover

- _lazy_features.py: use /a2a prefix + /message/send suffix for the
  a2a feature so a request to /v1/a2a/discover no longer triggers the
  a2a_endpoints module to load alongside a2a_registration.
- agent_endpoints/endpoints.py: drop the no-op description override
  kwarg from _build_merged_agent_card and its three call sites. The
  upstream card's description is already preserved by merge_agent_card's
  deepcopy, so passing it explicitly did nothing.

* style: black-format litellm/a2a_protocol/litellm_completion_bridge/transformation.py

* fix: address PR bugfix review for a2a discovery + metadata forwarding

- agent create form (add_agent_form.tsx): drop the skills.length > 0
  guard so an admin can clear all discovered skills during creation,
  matching the edit form's overlay behavior (consistency between
  create and edit flows).

- agent_card_discovery.tsx: stop including savedAgentCard in the
  handleDiscover useCallback deps. Read it via a ref inside
  resetSelections instead, so a parent-driven re-render that hands us
  a new savedAgentCard object reference (e.g. a background refresh of
  the agent record) does not recreate handleDiscover and re-fire the
  auto-discover effect, which would otherwise overwrite in-progress
  user edits in parent-driven mode (debounceMs = 0).

- a2a_endpoints.invoke_agent_a2a: skip 'metadata' when moving
  litellm params off of A2A MessageSendParams into body. The A2A
  protocol defines params.metadata as a first-class request-level
  field, and the completion bridge's get_forward_metadata is supposed
  to merge it with message.metadata. Previously the proxy always
  stripped params.metadata before constructing MessageSendParams, so
  the params-level branch in get_forward_metadata was dead code in
  the proxy flow.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(a2a): return 404 from get_agent_card when agent has no card

* fix(agents): apply discovery overlay uniformly on create and dedupe ALLOWED_CAPABILITY_KEYS

- buildAgentData now applies overlayDiscoveredCardParams after every
  non-custom branch (a2a, use_a2a_form_fields, dynamic) so types with
  credential_fields no longer silently drop discovered skills,
  capabilities, input/output modes, provider, and icon/doc URLs on
  submit. Mirrors the edit flow in agent_info.tsx.
- Export ALLOWED_CAPABILITY_KEYS from agent_discovery_utils and import
  it in agent_card_discovery so the rendering and selection-filtering
  logic share a single source of truth.

Co-authored-by: Yassin Kortam <[email protected]>

* ci(proxy-endpoints): wire tests/test_litellm/proxy/a2a into the shard

The two new test files (test_discovery.py, test_agent_card.py) were
not picked up by any pytest path, so their coverage never reached
codecov and patch coverage fell below the auto target.

* fix(ui): overlay discovered name/description in create flow for dynamic agents

Mirror the edit-form overlay in agent_info.tsx so dynamic agent types
(e.g. LangGraph) whose forms don't register name/description as
Form.Items don't silently lose those discovery-panel edits on save.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(a2a): default merged agent card version, null-guard runtime URL lookup, scope discovery auto-fire to A2A types

- merge_agent_card now defaults version to 1.0.0 when upstream omits it
  (A2A v1.0 schema requires the field).
- invoke_agent_a2a guards against agent_card_params being None so plain
  chat agents routed via the A2A path return a JSON-RPC error instead of
  AttributeError.
- buildDiscoveryRequest no longer falls back to any URL-shaped credential
  field for non-A2A agent types (Azure AI Foundry, Bedrock AgentCore,
  Vertex). Discovery only auto-fires for pure A2A and use_a2a_form_fields
  runtimes; the manual URL input remains available as an escape hatch.

* fix(ui): extract overlayDiscoveredCardParams + debounce parent-driven discovery

Two findings from greptile review:

1. `overlayDiscoveredCardParams` was copy-pasted between `add_agent_form.tsx`
   and `agent_info.tsx`. Move it to `agent_discovery_utils.ts` so the create
   and edit flows share the same overlay logic and there's only one place to
   update when discovered fields change.

2. `agent_card_discovery.tsx` used a zero-debounce path for parent-driven
   mode, which fires one discovery HTTP request per keystroke when an admin
   types into the parent form's URL / api_base / assistant_id fields (the
   parent rebuilds the plan from watched form values every render). Apply
   the same 400ms debounce uniformly.

* fix(a2a): preserve discovery name edit, default discovery headers, sync url on re-discover

- _build_merged_agent_card: prefer card-supplied name over agent_name so
  the discovery panel's editable 'Name (shown to API clients)' value is
  not silently overwritten by the internal identifier.
- async_safe_get call in fetch_well_known_card: pass headers or {} to
  avoid TypeError({**None, 'Host': ...}) when URL validation is enabled
  in production (default).
- agent_info handleApplyDiscoveredCard: set url: selection.upstream_url
  in fieldsToSet so re-discovery during edit refreshes the form's URL
  field for pure A2A agents (matches add_agent_form).

Co-authored-by: Yassin Kortam <[email protected]>

* fix(a2a): scrub upstream url from /public/agent_hub cards

Public agent_hub returned agent_card_params verbatim, exposing the
retained upstream backend url to unauthenticated callers. Rewrite the
url to the proxy /a2a/{agent_id} entrypoint on response, matching the
behavior of the authenticated well-known agent-card endpoint, so the
backend cannot be reached outside LiteLLM's auth, budget, and logging
path.

* fix(a2a): include suffix-matched routes in lazy warm openapi fragment

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

…an (#29315)

* fix(proxy): link passthrough success spans to the SERVER root OTEL span

Passthrough requests never wired user_api_key_dict.parent_otel_span into the
logging metadata, so on success the litellm_request span orphaned into its own
trace and the "Received Proxy Server Request" root span was never ended. Setting
it once in _init_kwargs_for_pass_through_endpoint fixes both the non-streaming
and streaming paths, since update_environment_variables copies that metadata onto
the logging object's model_call_details, which is what the OTEL handler reads.

Resolves LIT-3443

* fix(proxy): set passthrough parent span after client metadata merge

Greptile flagged that litellm_parent_otel_span was assigned before the
_metadata.update() calls that merge request-body metadata, so a client body
mirroring the internal key could overwrite the real span with a JSON scalar
and null the fix for that request. Move the assignment after the merge and
add a regression test that fails on the old ordering.

* fix(proxy): also set user_api_key after client metadata merge

Per Greptile, user_api_key had the same clobber window as the parent span: a
passthrough request body mirroring the key could overwrite the authenticated
value in the logged metadata. Move it into the same post-merge block and add a
deterministic contract test asserting both internal keys resist client-supplied
metadata.

* test(proxy/proxy_server): pin forwarding routes (PR2) (#28887)

* test(proxy): pin proxy_server.py forwarding-route behavior

PR2 of the proxy_server.py behavior-pinning project: fills the 12
forwarding-route test files added by the harness PR with happy + error
pins for all 52 LLM-facing routes (models, chat/completions, completions,
embeddings, moderations, audio, assistants, threads, utils, model-info,
model-metrics, queue). Every happy-path test asserts the full response
dict via normalize() so the gate enforces real shape pinning rather
than status codes.

* test(proxy): drop task-plumbing comments from PR2 test files

* test(proxy): tighten PR2 error-path status-code pins

Apply the same review feedback Greptile gave on PR1 (#28856) and PR3
(#28850) to PR2's forwarding-route tests:

- Replace permissive `>= 400` / `in (X, Y)` status assertions with the
  exact 500/405 the handler actually returns, so a regression that
  silently shifts the code now fails the pin.
- Add a body-presence check alongside each tightened status assertion
  to satisfy _pin_check.py's no-status-only rule.

---------

Co-authored-by: Claude <[email protected]>

* test(proxy): pin proxy_server.py non-route surface behavior (PR1) (#28856)

* test(proxy): pin proxy_server.py non-route surface behavior (PR1)

Fills the 7 PR1 placeholder files under tests/test_litellm/proxy/proxy_server/
with behavior pins for the non-route surface of proxy_server.py:
lifecycle/init/shutdown, ProxyConfig class methods, DB-overlay config scrubbers,
spend counters, background-health helpers, OpenAPI customization, exception
handlers, and streaming-generator helpers.

233 tests cover 101 pin-list symbols (1+ happy + 1+ error each). New-tests-only
coverage on litellm/proxy/proxy_server.py: 32.80% line / 20.91% branch (PR1
gate: 25% line / 18% branch). Full directory runs in ~22s with -n 4.

Plan: https://www.notion.so/Plan-Pin-proxy_server-py-behavior-2026-05-25-36c43b8acdab81ee845fd5365128a2fc

* test(proxy): address Greptile review comments on test_lifecycle.py

- test_initialize_signature_is_async_with_expected_params: hard-code
  expected_param_count so a signature change actually trips the gate
  (previously both sides of the comparison were len(sig.parameters)).
- test_check_request_disconnection_invalid_when_connected_times_out:
  patch asyncio.sleep so the test no longer spins for ~1.2 s of real
  wall-clock; timeout lowered to 0.05 s.

---------

Co-authored-by: Claude <[email protected]>

* test(proxy/proxy_server): pin control-plane routes (PR3) (#28850)

* test(proxy/proxy_server): pin misc routes (PR3, partial)

Adds happy + error tests for the misc control-plane routes:
GET /, /routes, /adaptive_router/state, /get_logo_url,
/get_image, /get_favicon.

Also gitignores .pin_list.txt (used by the pin gate).

* test(proxy/proxy_server): pin login/SSO routes (PR3, partial)

Adds happy + error tests for the 5 login/SSO control-plane routes:
GET /fallback/login, POST /login, POST /v2/login, POST /v3/login,
POST /v3/login/exchange. Mocks authenticate_user and
create_ui_token_object at their imported location.

* test(proxy/proxy_server): pin onboarding routes (PR3, partial)

Adds happy + error tests for the 2 onboarding control-plane routes:
GET /onboarding/get_token, POST /onboarding/claim_token. Wires a
MagicMock async context manager for prisma_client.db.tx() and
signs the onboarding JWT with the patched master_key.

* test(proxy/proxy_server): pin model_cost_map reload routes (PR3, partial)

Adds happy + error tests for the 5 model-cost-map control-plane routes:
POST /reload/model_cost_map, POST|DELETE|GET
/schedule/model_cost_map_reload(/status), GET /model/cost_map/source.
Attaches litellm_config to mock_prisma per-test (the table is not in
the default _PRISMA_TABLES fixture).

* test(proxy/proxy_server): pin anthropic_beta_headers reload routes (PR3, partial)

Adds happy + error tests for the 4 anthropic-beta-headers control-plane
routes: POST /reload/anthropic_beta_headers, POST|DELETE|GET
/schedule/anthropic_beta_headers_reload(/status). Stubs
db.litellm_config (not in default _PRISMA_TABLES) and monkeypatches
reload_beta_headers_config so no network calls fire.

* test(proxy/proxy_server): pin invitation routes (PR3, partial)

Adds happy + error tests for the 4 invitation control-plane routes:
POST /invitation/new, GET /invitation/info, POST /invitation/update,
POST /invitation/delete. Patches _user_has_admin_privileges /
_user_has_admin_view to avoid extensive get_user_object mocking.

* test(proxy/proxy_server): pin config CRUD routes (PR3, partial)

Adds happy + error tests for the 8 config-CRUD control-plane routes:
POST /config/update, POST|GET /config/field/update|info, GET /config/list,
POST /config/field/delete, POST /config/callback/delete,
GET /get/config/callbacks, GET /config/yaml. Attaches litellm_config
to mock_prisma per-test.

* test(proxy/proxy_server): tighten pin assertions per review

- test_routes_misc.py: `b"" in response.content` is trivially true;
  replace with `len(response.content) > 0` so an empty 405 body trips
  the gate.
- test_routes_login_sso.py: `len(response.content) >= 0` is trivially
  true; tighten to `> 0`.
- test_routes_anthropic_beta.py: replace brittle string-literal checks
  on the serialized JSON (`'"interval_hours": 12' in payload`) with
  `json.loads` + dict access so the assertion survives any serializer
  spacing.
- test_routes_config.py: `assert status_code in (404, 500)` was too
  permissive; the handler re-raises HTTPException(404) verbatim, so
  pin 404 strictly.

---------

Co-authored-by: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>

* test(e2e): cover Internal User create-key flow when in no teams

The seeded e2e-internal-user is in two teams, so the "no team" branch
of the Create Key modal — where the team dropdown must render empty —
was unreachable. Seeds a [email protected] user and adds a spec that
logs in fresh, opens the modal, and asserts the dropdown has zero
options.

* test(e2e): harden no-team dropdown assertion + add with-teams counterpart

Replace the one-shot count() check with a settled-empty assertion: wait for
the dropdown's loaded "No teams found" state before asserting zero options,
so the test can't pass on a transient empty frame while the team-options
request is still in flight.

Add internalUserWithTeams.spec.ts as the differential partner; it logs in as
the seeded e2e-internal-user (two team memberships) and asserts the dropdown
lists exactly those teams. Without it, the no-team spec's zero-options
assertion would still pass against a regression that empties the dropdown for
every user.

#29077)

* test(e2e): assert internal-user navbar identity is scoped to that user

The existing login.spec.ts only checks the admin's navbar identity.
This adds the symmetric check for the internal user — verifying the
account button + dropdown surface the internal user's email, id, and
role, and that no admin-scoped values leak through.

* test(e2e): harden navbar identity test per review feedback

Locate the user dropdown panel by a data-testid on the popupRender div
instead of Ant Design internal + Tailwind class names, so styling
refactors no longer risk breaking the identity-scoping assertions.
Source the seeded user emails/ids from shared constants (match seed.sql)
instead of hardcoding them inline.

…ce spans (#29319)

… providers (#28868)

* feat(anthropic/messages): in-gateway context_management polyfill for non-Anthropic providers

- Add `context_management/` module with `clear_tool_uses_20250919` editor
  dispatched before chat-completions translation on `/v1/messages`
- Hard-protect most-recently completed tool_result from being cleared
- Attach `context_management.applied_edits` to both non-streaming and
  streaming (final `message_delta`) responses
- Bedrock Converse: forward `context_management`; filter to
  `compact_20260112`-only edits with `compact-2026-01-12` beta header
- token_counter: guard Anthropic-format tools (no `function` key) to
  prevent AttributeError during polyfill token counting
- Streaming: handle empty-choices usage-only trailing chunks
- Skip polyfill when `litellm.drop_params = True`

Co-authored-by: Cursor <[email protected]>

* fix(bedrock): pop None context_management before sending to Bedrock Converse

If context_management is forwarded as None (e.g. when mapping returns
None for an invalid format), _filter_context_management_for_bedrock_converse
previously returned early without removing the key, leaving
"context_management": null in the request and causing a validation
error. Pop the key when the value is not a dict.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(bedrock/converse): pop None context_management; extract helpers to fix PLR0915

Co-authored-by: Cursor <[email protected]>

* fix(anthropic/messages): check per-request drop_params alongside global

Co-authored-by: Cursor <[email protected]>

* fix(anthropic/messages): preserve drop_params for downstream and respect explicit False

Co-authored-by: Yassin Kortam <[email protected]>

* fix: lazy debug logging in clear_tool_uses; remove unused context_management constants

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic/messages): guard context_management polyfill with try/except

Wrap apply_context_management() in a try/except so any failure (e.g.
litellm.token_counter raising on an unknown tokenizer or unexpected
message format) is logged but does not crash the underlying LLM
request. The polyfill is a best-effort additive feature; on failure we
forward the original messages without applied edits.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(token_counter): guard None input_schema in Anthropic tool fallback

Use `or {}` instead of `.get(..., {})` so explicit null parameters do not
raise AttributeError when formatting function definitions for token counting.

Co-authored-by: Cursor <[email protected]>

* fix: minimize context_management polyfill threading

- Use None (not empty list) for polyfill_applied_edits when context
  management isn't requested, so semantics of 'feature not requested'
  vs 'feature requested but no edits applied' are distinct.
- In the streaming iterator, only pass applied_edits to the per-chunk
  translator on the final (finish_reason) chunk; intermediate chunks
  ignore it anyway, and this makes intent explicit on both sync and
  async paths.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(context_management): align tool_use counts and normalize list spec

- _count_tool_uses now requires a string id, matching _collect_tool_use_ids_in_order so the tool_uses trigger can't fire on blocks that aren't clearable.
- apply_context_management dispatcher now accepts the OpenAI list form and normalizes it via AnthropicConfig.map_openai_context_management_to_anthropic, so the polyfill path no longer silently no-ops on list input.

Co-authored-by: Yassin Kortam <[email protected]>

* feat(context_management): add compact_20260112 polyfill for non-Anthropic providers

Implements an in-gateway compaction polyfill that summarizes long conversations
using a configurable model when `compact_20260112` is requested for non-Anthropic
targets (e.g. OpenAI, Gemini), matching Anthropic's context management beta
behaviour for those providers.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(compact): skip tool_result-only user turns; bedrock: elif for context_management

- compact_20260112 Phase D: when keeping the last user turn after a full
  summary, skip role=user turns whose content is exclusively tool_result
  blocks. Such turns translate to OpenAI tool-role messages with no
  preceding assistant tool_calls (those got summarized away), which
  non-Anthropic providers reject. Fall back to a synthetic continuation
  prompt if no eligible user question exists, so the downstream call
  always has a non-empty user message.
- bedrock converse: chain the context_management param as elif so it
  follows the same if/elif pattern as the surrounding thinking/
  reasoning_effort checks.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic): post-compaction question selection, system type, sync stream merge

- compact.py: select last user question from effective_messages (post-compaction slice) instead of raw messages, so prior summarized turns aren't reintroduced
- handler.py: widen _prepare_completion_kwargs system parameter type to Union[str, List[Dict]] matching PolyfillResult.system
- streaming_iterator.py: mirror async hold-and-merge logic in sync __next__ so context_management is attached to the final merged message_delta when stop_reason and usage arrive in separate chunks

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic/messages): apply context_management on sync path; clear held stop_reason chunk in async iterator

- Sync `anthropic_messages_handler` was silently dropping the
  `context_management` kwarg via `ANTHROPIC_ONLY_REQUEST_KEYS` after the
  polyfill was moved into the async handler. Bridge to the async
  dispatcher with `run_async_function` so `litellm.messages.create()`
  callers keep working (regressed e.g. `clear_tool_uses_20250919`).
- In the streaming iterator's `__anext__` `StopIteration` handler, clear
  `self.holding_stop_reason_chunk` after capturing it (matches `__next__`)
  so a subsequent call doesn't re-emit the same chunk.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(bugfixes): bedrock None context_mgmt; stream per-instance queue; sync polyfill; trailing-chunk passthrough

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic): silently drop trailing chunks after usage; remove dead _polyfill_result key

- streaming_iterator: in sync __next__, after the usage chunk has been
  merged and emitted, silently consume any trailing provider events
  via 'continue' instead of forwarding them through the queue. Trailing
  chunks would translate to content_block_delta or message_delta and
  violate Anthropic SSE ordering after the final message_delta. The
  async __anext__ already drops these via 'if not self.queued_usage_chunk:'
  gating, so this aligns sync and async behavior.

- handler: drop unused '_polyfill_result' from ANTHROPIC_ONLY_REQUEST_KEYS.
  PolyfillResult is passed as an explicit arg to the adapter methods, never
  through extra_kwargs, so the entry was dead code.

Co-authored-by: Yassin Kortam <[email protected]>

* refactor(anthropic): extract usage-merge helper; guard empty slice-only compaction result

- Extract the duplicated hold-and-merge usage logic from the sync __next__ and
  async __anext__ paths into a shared _merge_usage_into_held_stop_reason_chunk
  helper so the subtle cache-token / context_management attachment lives in
  exactly one place.
- In the compact_20260112 slice-only path, fall back to _select_last_user_question
  when _strip_compaction_blocks produces an empty list (e.g. messages ending on
  an assistant turn whose only content was the compaction block) so the
  downstream API never receives an empty messages array.

Co-authored-by: Yassin Kortam <[email protected]>

* refactor(anthropic/context_management): streaming iterator compaction fixes and compact polyfill improvements

- Extract usage-merge helper; guard empty slice-only compaction result
- Silently drop trailing chunks after usage; remove dead _polyfill_result key
- Fix bedrock None context_mgmt; stream per-instance queue; sync polyfill; trailing-chunk passthrough
- Apply context_management on sync path; clear held stop_reason chunk in async iterator
- Fix post-compaction question selection, system type, sync stream merge
- Skip tool_result-only user turns; bedrock: elif for context_management
- Add streaming iterator compaction test suite

Co-authored-by: Cursor <[email protected]>

* revert(html): restore flat *.html naming in _experimental/out

Reverses the accidental rename from *.html → */index.html introduced in
15ea941. All 35 files moved back to their original flat paths so the
directory structure matches litellm_internal_staging.

Co-authored-by: Cursor <[email protected]>

* revert(config): restore proxy_server_config.yaml to litellm_internal_staging

Co-authored-by: Cursor <[email protected]>

* Fix: skip client compaction pre-processing when compact_20260112 polyfill will run

The _prepare_context_managed_request helper unconditionally applied
apply_client_compaction_block_history before invoking the polyfill. When
the request also configured a compact_20260112 spec, that pre-processing
consumed the client-sent compaction block and collapsed the message history
to just the latest user question, starving the polyfill of conversation
context. The polyfill's own Phase A (_slice_around_compaction_block)
already handles client compaction blocks correctly and inspects the full
post-compaction tail for the token-threshold check, so the pre-processing
is both redundant and destructive in this case.

Now the pre-processing only runs when no compact_20260112 polyfill spec
will execute (no spec, drop_params on, or only non-compact edits like
clear_tool_uses_20250919).

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic): plug compaction-block leak + iteration-usage gaps in streaming adapter

- handler: when polyfill_will_run skipped client-history pre-processing
  and the polyfill ultimately returned None (best-effort swallow on
  unexpected error), apply the slice-only fallback before returning so
  Anthropic-specific 'compaction' content blocks don't leak to non-
  Anthropic backends that would reject them.
- streaming_iterator: precompute will_merge_into_held so we don't pass
  applied_edits into the translator when the resulting processed_chunk
  will be discarded by the held stop-reason merge path.
- streaming_iterator: augment processed_chunk with iterations usage in
  the holding_chunk branch (sync and async) for parity with the other
  emission branches; ensures usage.iterations is attached on the rare
  message_delta-reaches-holding_chunk path.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic): correct streaming usage iteration + translate tools for token counting

- streaming_iterator: skip the trailing "message" iteration entry in the
  final message_delta when the held stop_reason chunk carries placeholder
  zero usage (no separate usage chunk arrived). Reporting zero tokens was
  misleading and inconsistent with the non-streaming path which always
  has real usage data.
- streaming_iterator: drop two redundant type checks inside branches
  that are already guarded by an outer message_delta type check.
- compact._count_effective_tokens: translate Anthropic-shaped tools
  (input_schema) to OpenAI shape before passing to litellm.token_counter
  so threshold checks aren't skewed by tokenizer paths that expect the
  OpenAI tool wrapper.

Co-authored-by: Yassin Kortam <[email protected]>

* Fix lint

* fix(anthropic): plug content drop, compaction SSE shape, and compaction leak

- Sync streaming __next__ no longer drops a buffered holding_chunk when
  the usage-merge path has already fired. Restoring the prior unconditional
  flush behavior preserves provider-emitted content (the SSE-ordering nit
  of a trailing content delta is preferable to silent content loss).
- compaction content_block_start now carries the full block shape
  ({"type": "compaction", "content": ""}) to match the text-block
  pattern and Anthropic's native streaming shape, so clients that key off
  content_block_start see the field.
- apply_compact_20260112 now slices around / strips compaction blocks
  before the opt-in gate check. Previously, when summary_model was not
  configured the editor returned the raw messages, leaking Anthropic-only
  compaction content blocks to non-Anthropic providers that reject them.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic): resolve mypy types in context management polyfill

Use AppliedEdit and CompactionBlock consistently in the dispatcher and streaming adapter.

Co-authored-by: Cursor <[email protected]>

* fix(anthropic): flush held content chunk in async streaming path

Mirror the sync __next__ behavior: always flush a buffered
holding_chunk after the stream ends, even when usage was already
merged + emitted. Previously the async __anext__ kept the flush
inside the 'if not self.queued_usage_chunk:' guard, silently
dropping the last content delta on the proxy's primary path.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic adapter): correct sync streaming, surface polyfill failures, decouple sync path from proxy router

- translate_completion_output_params_streaming: add is_async flag so the
  sync handler returns Iterator[bytes] instead of an unusable
  AsyncIterator. Async callers keep the existing behavior via the
  default is_async=True.
- _run_polyfill_if_enabled: when the polyfill crashes and the spec
  requested non-compact edits (e.g. clear_tool_uses_20250919), raise an
  AnthropicContextManagementError instead of silently returning None so
  those edits are not dropped without an error surface. The
  compaction-block-slicing safety net remains for compact-only specs.
- anthropic_messages_handler (sync): stop auto-attaching the proxy
  llm_router. run_async_function bridges to a new thread's event loop;
  reusing the proxy's loop-bound httpx clients there causes
  'Event loop is closed' errors. The summary editor falls back to
  litellm.acompletion when llm_router is None.

Co-authored-by: Yassin Kortam <[email protected]>

* fix: address bug detection findings in token counter and streaming iterator

- token_counter: guard against non-dict 'function' field in tool dicts
  and skip tools missing a name to avoid emitting 'type None = ...' which
  would produce inaccurate token counts.
- streaming_iterator: change sync __next__ generic-error path to raise
  StopIteration (was StopAsyncIteration), so sync iteration cleanly stops.
- streaming_iterator: centralize context_management attachment so the
  held-stop_reason direct-flush path defensively re-attaches applied_edits
  to match the merge path's guarantee.

Co-authored-by: Yassin Kortam <[email protected]>

* Fix lint

* fix: correct COMPACT_MIN_TRIGGER_TOKENS to 50_000

Co-authored-by: Yassin Kortam <[email protected]>

* Fix lint

* Fix lint

* Fix lint

* fix(compact): reduce to last user question when summary_model not configured but prior compaction block exists

Aligns the summary_model_not_configured path with the under-threshold and
client-compaction-block paths, which both reduce post-compaction messages
to just the latest user question so the downstream provider doesn't get
the summary on system prefix AND the full post-compaction history.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(compact): forward caller system prompt to summary model call

The default summarization instructions reference "the initial task above"
and "the raw history above", but the system prompt that holds that task
was not being forwarded to the summary model. The summary call now
prepends an OpenAI-shaped system message translated from the original
Anthropic-shaped system (str or content-block list) so the summarizer
has the agent role and initial task in scope.

* fix(compact_20260112): set default max_tokens and merge prompt when last turn is user

- Set COMPACT_SUMMARY_MAX_TOKENS default for the summary call so providers
  like Anthropic (which require max_tokens) don't silently fail and degrade
  to summary_call_failed.
- When the trailing translated message is already a user turn, merge the
  summarization prompt into it instead of appending a second user turn.
  Avoids consecutive role=user messages that strict providers reject.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic adapter): move current_content_block_start to __init__

Move the default TextBlock dict from a class-level attribute to __init__ so
concurrent stream instances don't share the same mutable dict. The class-level
default could be mutated in-place via tool_block['name'] = original_name in
_should_start_new_content_block, leaking state across streams. This mirrors
the existing fix already applied to chunk_queue.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(compact_20260112): surface error states + strip tool_result blocks in last user question

applied_edits_for_response() now includes compact_20260112 edits that
carry an error field (summary_model_not_configured, summary_call_failed,
summary_extraction_failed) so clients and operators can see why
compaction was requested but not applied.

_select_last_user_question() now strips tool_result blocks from mixed
[tool_result, text] turns rather than passing them through as-is. After
compaction the paired tool_use assistant turn no longer exists, so
forwarding tool_result blocks translates to orphaned role=tool messages
on non-Anthropic providers and produces a 400.

* fix(compact_20260112): carry prior compaction summary into Phase C summary call

When a request already contains a compaction block, Phase A slices
`effective_messages` to the turns since that block. Previously Phase C
passed the original `system` to the summary model, so multi-round
compaction silently dropped accumulated history each time the polyfill
fired. Pass `augmented_system` (original system + prior summary
prefix) so the summary model can produce a comprehensive summary that
incorporates both the prior round's context and the current slice.
`summarized_system` for the downstream call stays built from the
original `system` + new `summary_text`.

* refactor: delegate handler spec normalization to dispatcher

_normalize_spec_edits in adapters/handler.py duplicated the spec-shape
normalization already implemented by _normalize_spec in
context_management/dispatcher.py. The two could drift: a change in one
(e.g. supporting a new spec shape) without the other would cause the
handler's polyfill_will_run prediction to disagree with the
dispatcher's actual behavior, breaking the client-history pre-processing
skip.

Have the handler delegate to the dispatcher's _normalize_spec while
keeping handler-specific concerns (drop_params short-circuit, swallow
mapping exceptions) at the wrapper level.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(compact_20260112): surface warning-only applied edits in response

`applied_edits_for_response()` previously hid `compact_20260112` edits when
they had only warnings (no compaction block, no error). This dropped
diagnostically important warnings such as
`unsupported_trigger_type_X_using_input_tokens` and
`pause_after_compaction_ignored` whenever the conversation was under the
trigger threshold. Operators now see these warnings in the response.

Co-authored-by: Yassin Kortam <[email protected]>

* fix: address two low-severity context_management edge cases

- streaming_iterator: keep `sent_content_block_finish` in sync with the
  compaction block's emitted start/delta/stop lifecycle and reset it when
  the next text block's start is queued.
- bedrock _map_context_management_param: match dispatcher `_normalize_spec`
  behavior — only run the OpenAI→Anthropic mapper on list inputs; pass
  dict inputs through unchanged so already-Anthropic-format values aren't
  silently dropped.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(compact_20260112): use beta-header constant; require type discriminator; skip sync bridge when idle

- bedrock: replace hardcoded "compact-2026-01-12" beta string with
  ANTHROPIC_BETA_HEADER_VALUES.COMPACT_2026_01_12.value in both
  Converse (_filter_context_management_for_bedrock_converse) and Invoke
  (anthropic_claude3) compact-edit handlers.
- types: mark the "type" discriminator as Required[...] on the new
  CompactionBlock and UsageIteration TypedDicts so the discriminator
  is not silently optional under total=False.
- adapters/handler: short-circuit the sync /v1/messages adapter path
  before spawning the run_async_function worker-thread event loop when
  the request has no context_management spec and no client-sent
  compaction block in the message history.

Test plan:
- uv run pytest tests/test_litellm/llms/anthropic/experimental_pass_through/     tests/test_litellm/llms/bedrock/test_converse_context_management.py -q
  (370 + 10 = 380 passed)
- uv run pytest tests/test_litellm/llms/azure_ai/claude/test_azure_anthropic_transformation.py     tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_transformation.py     -k compact (3 passed)

* fix(compact_20260112): include system prompt tokens in threshold check

The threshold check in Phase B previously counted only message tokens and
the compaction-block content, omitting the system prompt entirely. When
the system carried a prior compaction summary (via _augment_system_with_summary)
or was otherwise large, the threshold could fire later than intended,
allowing the conversation to exceed the model's context window before
compaction activated.

_count_effective_tokens now also counts the (augmented) system prompt
text. The caller passes compaction_block=None when augmented_system
already includes the prior summary, to avoid double-counting.

Co-authored-by: Yassin Kortam <[email protected]>

* Fix SSE ordering and compaction state machine bugs in AnthropicStreamWrapper

- Suppress holding_chunk flush after final message_delta has been emitted
  (queued_usage_chunk == True) so a trailing content_block_delta cannot
  follow message_delta, which strict Anthropic SDK clients may reject.
  When usage has not yet been merged, flush the holding_chunk *before*
  the held stop_reason chunk so SSE ordering remains correct.

- Replace _queue_compaction_block_events with _next_compaction_event,
  emitting the compaction start/delta/stop events one at a time. The
  state machine flags (sent_content_block_finish) and content block
  index now advance atomically with the terminal stop event actually
  being returned to the caller, eliminating the transient inconsistent
  state where flags say the block is finished while its stop event is
  still buffered.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(compact_20260112): enforce parent key/team allowlist on summary model

The compact_20260112 polyfill summary subrequest used llm_router.acompletion
directly, bypassing the proxy auth checks that gate model access for the
parent key/team. A caller whose key/team was not authorized for the
configured context_management_summary_model could still cause the proxy to
invoke that model and return its output as a compaction block.

Pull the parent's UserAPIKeyAuth out of litellm_metadata in the handler,
thread it through the dispatcher into apply_compact_20260112, and gate the
summary call on _can_object_call_model for both key-level and team-level
allowlists. Failures land as applied_edits[0].error =
summary_model_access_denied without raising. SDK callers (no UserAPIKeyAuth)
remain unaffected.

* fix(compact_20260112): distinguish access-denied from transient errors; greedy summary regex

- _check_summary_model_access now catches ProxyException explicitly for access
  denials and logs unexpected exceptions separately. Both still fail closed,
  but operators can now tell a denied key/team apart from a router internal
  raising during the check.
- _SUMMARY_TAG_RE switches from non-greedy to greedy so a stray </summary>
  inside the model's summary content no longer silently truncates the
  captured text.

* fix(compact_20260112): type object_type as Literal for mypy

* fix(compact_20260112): attribute summary subcall spend to parent key/team

The compact_20260112 polyfill summary subrequest propagated metadata via
the Anthropic-shape `metadata` parameter, which only carries `user_id`.
The proxy auth fields used for spend attribution (`user_api_key`,
`user_api_key_team_id`, `litellm_call_id`, ...) live in
`data["litellm_metadata"]`. As a result, summary subcalls landed on the
router with an empty propagated metadata and the resulting tokens were
not attributed to the caller's key/team budget.

Rename the polyfill chain's spend-propagation parameter to
`litellm_metadata` and pull it from `kwargs["litellm_metadata"]` in
both the async and sync handlers, so the post-call hooks see the parent
key/team and bill the summary tokens accordingly. Add an
`_extract_proxy_litellm_metadata` helper and refactor
`_extract_user_api_key_auth` to use it.

* chore(anthropic adapters): remove unused _extract_user_api_key_auth helper

Co-authored-by: Yassin Kortam <[email protected]>

* chore(compact_20260112): non-greedy summary regex; use COMPACT_EDIT_TYPE in bedrock filter

- Make _SUMMARY_TAG_RE non-greedy so a response with multiple <summary>
  blocks captures only the first complete block.
- Replace the hardcoded 'compact_20260112' literal in
  _filter_context_management_for_bedrock_converse with the shared
  COMPACT_EDIT_TYPE constant.

* fix: bug fixes from PR review

- streaming_iterator: don't set sent_content_block_finish during compaction
  block lifecycle; that flag tracks the regular text/tool_use/thinking block
  state machine, conflating the two leaks bad state to introspection paths.
- compact._call_summary_model: send propagated proxy auth/spend-attribution
  fields as 'litellm_metadata' instead of 'metadata' so the router's post-call
  hooks attribute summary tokens to the caller's key/team budget.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(anthropic-streaming): insert content_block_stop between held delta and final message_delta

When the stream exhausts with both `holding_chunk` (a content_block_delta)
and `holding_stop_reason_chunk` (a message_delta) buffered, the after-loop
cleanup previously emitted them back-to-back, producing the invalid
Anthropic SSE sequence `content_block_delta -> message_delta`. Insert a
`content_block_stop` between them in both the sync `__next__` and async
`__anext__` paths so the emitted ordering remains
`content_block_delta -> content_block_stop -> message_delta`.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(compact_20260112): propagate allowed_model_region to summary subrequest

The router enforces region restrictions by reading allowed_model_region
from top-level request kwargs (Router._common_checks_available_deployment),
but the compact_20260112 summary subrequest only forwarded litellm_metadata.
A region-restricted caller could trigger compaction and have their conversation
summarized by a deployment outside the permitted region.

Extract allowed_model_region from user_api_key_auth and pass it through
_call_summary_model as a top-level kwarg so the router applies the same
region constraints the parent request would.

* fix(anthropic adapter): emit content_block_stop before held message_delta in drain paths

Co-authored-by: Yassin Kortam <[email protected]>

* feat(context_management): configurable summary max_tokens; surface ignored knobs

- compact_20260112: read summary max_tokens from general_settings
  (context_management_summary_max_tokens) so operators can fit the
  chosen summary model's output budget; falls back to the compiled
  default for missing or invalid values.

- clear_tool_uses_20250919: log unsupported knobs at warning level
  (was debug, which silently dropped misconfiguration) and surface
  them as warnings on the AppliedEdit so clients see what was ignored.

* fix(compact_20260112): bound _call_summary_model with timeout

A slow or unresponsive summary model previously hung the parent
/v1/messages request with no escape hatch. Pass a 60s timeout on the
litellm.acompletion / llm_router.acompletion subrequest; on timeout the
existing summary_call_failed path forwards the request without
compaction rather than blocking indefinitely.

* fix(compact_20260112): preserve post-compaction tail on slice-only path

When a prior compaction block is present and the request is under threshold,
the polyfill was reducing downstream messages to just the latest user
question. The prior summary only covers turns before the compaction block,
so dropping the post-compaction tail silently lost recent context — a
multi-turn conversation that stayed below the threshold would arrive at the
model with no memory of any turn after the prior compaction.

Forward the already-stripped post-compaction tail unchanged on both the
under-threshold path and apply_client_compaction_block_history. Fall
back to _select_last_user_question only when the strip leaves nothing
for the downstream call to answer.

* fix(compact_20260112): enforce user/project/team-member model scopes on summary subrequest

The local gate previously only checked the parent key's and team's
allowed-model lists. A caller restricted by a personal user, project,
or per-team-member allowed_models scope could still trigger the
configured summary model and receive its <summary> output as a
compaction block, because llm_router.acompletion bypasses the proxy
common_checks path. Extend _check_summary_model_access to also load
the user_object, project_object, and team_membership and run the
matching allowlist check at each scope before invoking the summary
model.

* fix(compact_20260112): enforce summary model per-model budget and propagate budget metadata

* fix(compact_20260112): forward post-compaction tail when summary model unconfigured

* fix(anthropic endpoints): run failure hook on 500-level context management errors

* fix(compact_20260112): enforce summary model rate limit before summary call

* fix(compact_20260112): propagate end-user/project budget scope to summary call

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>
Co-authored-by: mateo-berri <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>

…28830)

Allow self-hosted installs to override the default LiteLLM sender address
via RESEND_FROM_EMAIL, matching SendGrid's SENDGRID_SENDER_EMAIL pattern.

Co-authored-by: Cursor <[email protected]>

…28728)" (#29326)

This reverts the Bedrock CI account migration (#28728). The original account
(888602223428) was put under an AWS security restriction after a leaked key
and has since been reactivated, while the replacement account (941277531214)
lacks access to several models the suites exercise (legacy Bedrock Claude 3
models, Cohere, Nova Canvas image gen, Bedrock batch inference, and flagship
Opus). Pointing CI back at the reactivated account restores that coverage.

This is the exact inverse of #28728: all hardcoded 941277531214 references go
back to 888602223428 (provisioned/imported-model ARNs, AgentCore runtime ARNs
and their suffixes, batch execution role ARN, and the example proxy config),
the S3 buckets revert to litellm-proxy and load-testing-oct, the guardrail IDs
revert to wf0hkdb5x07f and ff6ujrregl1q, the SageMaker endpoint and Knowledge
Base revert to their original ids, and the live-call tests go back to the
legacy model strings. The grid_spec fail_reason workaround for the unentitled
Opus cells is dropped while keeping the unrelated bedrock_effort_ceiling field
added after the migration.

The CircleCI AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env vars still point at
941277531214 and must be set to the reactivated account's fresh credentials
separately via the CircleCI API; AWS_REGION_NAME stays us-west-2.

…29249)

* fix(mcp): preserve source_url in GET /v1/mcp/server list responses

The list endpoint builds responses from the in-memory registry, but
source_url was dropped during the DB-to-registry roundtrip even though
GET /v1/mcp/server/{id} returned it correctly from the database.

Co-authored-by: Cursor <[email protected]>

* fix(tests/mcp): set source_url on MagicMock table records

MagicMock auto-creates source_url as a mock object, which fails MCPServer
Pydantic validation after source_url was wired through build_mcp_server_from_table.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>

…es (#29253)

* fix(mcp): preserve omitted fields on PUT /v1/mcp/server partial updates

Use model_dump(exclude_unset=True) for updates so schema defaults (transport=sse,
allow_all_keys=false, etc.) are not written when callers omit them. Serialize
JSON fields from the filtered dict and only force is_byok on create.

Co-authored-by: Cursor <[email protected]>

* fix(mcp): allow explicit alias=None on MCP server partial updates

Snapshot caller-provided fields before normalization so omitted alias is not written while an intentional alias=None still clears the stored value.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>

…Opus 4.7 self-heal) (#29344)

* test(logging): align DB metrics event_metadata assertions with safe redaction

PR #28909 hardened log_db_metrics to emit a minimal, non-sensitive
event_metadata (only table_name when present, otherwise None) instead of
dumping function_name, function_kwargs, and function_args onto the span. The
test in test_log_db_redis_services was not updated and still asserted
"function_name" in event_metadata, which raised TypeError (argument of type
'NoneType' is not iterable) and turned the logging_testing CI job red on
litellm_internal_staging.

Update test_log_db_metrics_success to assert event_metadata is None when no
table_name is passed, and add test_log_db_metrics_event_metadata_is_safe as a
regression guard verifying that only the table name surfaces and that sensitive
kwargs (tokens, prisma client) are never dumped.

* test(bedrock): self-heal opus-4-7 grid cells when unentitled on CI

The bedrock-claude-opus-4-7 converse cells are unentitled on the Bedrock CI
account, so they were marked xfail. xfail keeps reporting them as expected
failures even after access is granted, so the wire translation never gets
verified again. Now the cell makes the call and skips only when Bedrock
replies "is not available for this account"; the moment the model is
entitled the same cells run their full assertions with no edit.

A focused unit test pins the tolerance predicate so any other failure still
surfaces loudly and the available path still runs the assertions.

…9343)

* refactor(proxy/auth): normalize Bearer prefix in safe-hash helper

UserAPIKeyAuth._safe_hash_litellm_api_key now strips a leading
"Bearer "/"bearer " prefix before its existing sk-/JWT classification, so
the helper produces the same hashed output regardless of whether the
caller stripped the Authorization header prefix or passed the header
value through unchanged.

* refactor(proxy/auth): make Bearer-prefix strip case-insensitive

Per RFC 7235 the HTTP authorization scheme token is case-insensitive.
Replace the two-prefix loop with a single case-insensitive check so the
helper normalizes "Bearer ", "bearer ", "BEARER ", and any mixed-case
variant before classifying the remainder as sk- or JWT. The contract
test gains coverage of "BEARER " and "BeArEr ".

* test(mcp): align auth-handler test expectations with safe-hash helper

The two MCP auth tests asserted that UserAPIKeyAuth(api_key="Bearer ...")
retained the raw header bytes on the api_key field. _safe_hash_litellm_api_key
now normalizes that input — stripping the Bearer prefix and hashing the
resulting sk- key — so the expectations move to the normalized form:
the bare token in the parametrize case, and hash_token("sk-...") in the
backward-compat assertion. This matches what the real auth flow produces
(the builder strips Bearer and the DB stores the hashed token), so the
mocks now line up with production rather than with the un-normalized
validator output.

…28418)

* fix(guardrails): return HTTP 400 for litellm content filter blocks

Align litellm_content_filter hard rejects with the standard guardrail block status code so clients receive 400 instead of 403.

Co-authored-by: Cursor <[email protected]>

* fix(guardrails): return HTTP 400 for custom code guardrail blocks

Pre-call custom code guardrail blocks now raise HTTPException(400) instead of using the passthrough ModifyResponseException path that returned a synthetic 200 response.

Co-authored-by: Cursor <[email protected]>

* fix(guardrails): preserve custom code passthrough blocks

Keep standalone custom code guardrail blocks on the passthrough contract while covering policy pipeline block handling for passthrough-style guardrail interventions.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>

…#29202)

* fix(proxy): restrict vector store index create/delete to proxy admins

Prevent non-admin API keys from registering indexes via POST /v1/indexes or deleting Azure AI Search indexes through managed pass-through routes.

Co-authored-by: Cursor <[email protected]>

* fix(proxy): tighten vector store index lifecycle checks

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>

…29160)

* feat(pass_through): extend passthrough_managed_object_ids to Azure

Adds managed ID minting/resolution for Azure passthrough endpoints
(/azure/...) alongside the existing OpenAI passthrough support.

Key changes:
- pass_through_endpoints.py: detect azure/azure_ai custom_llm_provider
  (string or enum) to set _is_managed_id_provider and _managed_id_provider;
  both INPUT and OUTPUT rewrite blocks now fire for Azure.
- llm_passthrough_endpoints.py: forward custom_llm_provider into
  create_pass_through_route so it reaches pass_through_request (was None).
- managed_id_rewriter.py: extend _PASSTHROUGH_PREFIX_RE and _canonical_path
  to strip /azure/openai prefix and add /v1/ for Azure paths that omit it;
  add ("azure", method, path) entries to BUILTIN_OUTPUT_ID_FIELD_MAP for
  files and batches endpoints.
- managed_id_codec.py / types/utils.py: supporting codec and enum constant.
- proxy_server.py: register llm_passthrough_router before batches_router to
  prevent route collision for /openai_passthrough/* paths.

Co-authored-by: Cursor <[email protected]>

* fix(pass_through): remove unused imports for ruff F401

Co-authored-by: Cursor <[email protected]>

* fix(pass_through): satisfy mypy for optional parsed_body

Co-authored-by: Cursor <[email protected]>

* fix(pass_through): compute query params string after managed-ID rewrite

Move requested_query_params_str computation to after the managed-ID
input rewrite block so logging_url reflects the rewritten raw-provider
query params actually sent upstream, instead of the original
managed IDs.

Co-authored-by: Yassin Kortam <[email protected]>

* Add support for managed ids for passthrough responses api

* Add support for list batches and list files

* style: run Black on passthrough managed ID files

Fix CI formatting for managed_id_rewriter.py and pass_through_endpoints.py.

Co-authored-by: Cursor <[email protected]>

* fix(passthrough): parse json file_object and implement before-cursor pagination

- Parse row.file_object via json.loads when Prisma returns it as a string;
  mirrors openai_files_endpoints/common_utils.py so list responses keep all
  stored detail fields (status, timestamps, etc.).
- Implement the previously-parsed-but-unused 'before' cursor for list
  pagination by flipping fetch order to ascending with a 'gt' bound on
  created_at, then reversing rows so the response stays newest-first.

Co-authored-by: Yassin Kortam <[email protected]>

* Remove logger

* refactor: split list_passthrough_ids_from_db to fix PLR0915

Extract pagination, fetch, and serialization helpers so the main list
function stays under the statement limit without changing behavior.

Co-authored-by: Cursor <[email protected]>

* fix: scope passthrough managed ID dedup and list by provider

Validate embedded provider before reusing deduped file/object rows so
OpenAI and Azure cannot share the same managed ID for an identical raw ID.
Filter list responses to rows whose managed IDs decode to the current
provider, with over-fetch scanning when needed.

Co-authored-by: Cursor <[email protected]>

* fix(managed_id_rewriter): cap pagination trim at effective limit

When raw_limit > 100, fetch_limit is capped at 101 (one extra row to
detect has_more), but trimming with rows[:raw_limit] failed to drop
the sentinel row. Use the capped effective limit instead.

Co-authored-by: Yassin Kortam <[email protected]>

* fix: cross-provider object collision and fail-closed list error handling

Greptile P1: move provider check before access check in _mint_or_reuse_object
so a cross-provider raw ID collision (OpenAI and Azure share the same batch_
ID) falls through to mint a new provider-scoped row instead of raising 404.

Veria-ai medium: _fetch_provider_scoped_list_rows now always returns
(page, has_more) — DB errors break out of the scan loop and return matched
rows so far. list_passthrough_ids_from_db never returns None for a recognised
list route, so the caller can never fall through to the upstream provider.

Co-authored-by: Cursor <[email protected]>

* fix: namespace passthrough model_object_id by provider to prevent unique violation

Store model_object_id as 'passthrough:{provider}:{raw_id}' instead of the
bare raw ID so OpenAI and Azure can each own a row for the same raw batch ID
without hitting the @unique constraint. Dedup lookup uses the same namespaced
key so it is implicitly provider-scoped and the _managed_id_matches_provider
check is no longer needed on the object path.

Co-authored-by: Cursor <[email protected]>

* fix: gate list interception on managed_files hook like input/output rewrites

Without the hook no managed IDs are minted so the DB is empty. Intercepting
GET /v1/files without the hook returned an empty list and hid the caller's
real upstream files/batches. Matches the guard used by the input and output
rewrite blocks.

Co-authored-by: Cursor <[email protected]>

* fix: set has_more=True when scan cap is hit with a full final DB batch

When max_scans (20) is exhausted and the last DB page was full-sized,
there are almost certainly more rows beyond the scan window.  Track
last_batch_full across iterations so the scan_cap_hit condition sets
has_more=True in that case, preventing silent pagination truncation in
high-mixed-provider pools.

Co-authored-by: Cursor <[email protected]>

* fix(managed_id_rewriter): scope pagination cursor lookup to caller-owned rows

Prevent a cross-tenant timing oracle by constraining the after/before
cursor row lookup to the caller's owner_filter, and cover the real Azure
responses path form (no /v1/) in tests.

* fix(managed_id_rewriter): align passthrough list metadata with direct GET

Persist upstream file metadata when minting a managed file ID and rewrite
nested batch file IDs before snapshotting the object, so DB-served file/batch
list responses return the same fields and managed IDs as a direct endpoint
GET.

* fix(managed_id_rewriter): degrade to raw id on cross-owner object collision

The OUTPUT (mint) path of _mint_or_reuse_object raised HTTPException(404)
when a dedup hit on the namespaced model_object_id belonged to a different
owner, converting a successful upstream batch/response creation into a 404
for the caller. Two upstream accounts under one provider name can issue the
same raw id, so this is reachable in multi-tenant deployments.

Return the caller's raw id unmanaged instead: the upstream create already
succeeded, a new managed row can't be minted (model_object_id is @unique),
and reusing the other owner's managed id would later fail the access check.

* fix(managed_id_rewriter): scope list cursor by provider and cap body-rewrite recursion depth

* perf(managed_id_rewriter): push batch list provider scope to DB and anchor canonical-path prefix

Object (batch) list rows store model_object_id as passthrough:{provider}:{raw},
so the provider filter is now applied at the indexed DB column, collapsing the
application-layer multi-scan to a single query for that table. File rows keep
the decode-based scan since they have no provider column.

Anchor the canonical-path prefix regex at a path boundary so routes such as
/openai_realtime/... are no longer mis-stripped.

* fix(managed_id_rewriter): refresh stored batch snapshot on reuse

The dedup-reuse path in _mint_or_reuse_object returned the existing managed id
without updating the stored file_object, so DB-served list responses kept the
creation-time snapshot and showed null output_file_id/error_file_id even after
the batch completed. Refresh the snapshot when an owned row is reused so the
list reflects the batch's latest state.

* fix(managed_id_rewriter): deny cross-owner object access on retrieve/cancel/delete

Returning the raw id when can_access_resource fails only made sense for create
responses, where the caller's own upstream create succeeded under a raw id that a
different owner already holds. On retrieve/cancel/delete the caller reaches that
branch only by supplying another tenant's raw id (which bypasses the managed-id
input gate), so echoing the upstream object back leaked it cross-tenant. Restrict
the raw fallback to create routes and return 404 otherwise.

* fix(managed_id_rewriter): deny cross-owner file access on retrieve/delete

_mint_or_reuse_file scoped the raw file dedup lookup to the current caller, so a
raw file-... id belonging to another tenant was never found and the OUTPUT path
minted a fresh managed id for that same upstream file under the caller. A raw id
only reaches this path by skipping the managed-id input gate (raw provider ids
are opt-out), so a different-owner row means the caller is touching someone
else's file. Look up flat_model_file_ids globally and run can_access_resource;
deny with 404 on retrieve/delete and leave the raw id unmanaged on create, which
mirrors the cross-owner handling already in _mint_or_reuse_object.

* fix(managed_id_rewriter): deterministic provider-scoped file dedup

Replace the unscoped find_first in _mint_or_reuse_file with a find_many
ordered by created_at and an application-layer provider filter. The file
table has no provider column, so a raw file id shared across OpenAI and
Azure could map to one row per provider; find_first then picked a row
non-deterministically and, on a provider mismatch, minted a fresh managed
row on every call, accumulating duplicates. Selecting the oldest matching
same-provider row the caller can access keeps reuse stable and prevents
duplicate rows while preserving the cross-tenant deny/leave-raw behaviour.

* refactor(pass_through): scope passthrough managed IDs on the explicit provider

Move the openai/azure detection out of pass_through_request into
resolve_passthrough_managed_id_provider in llms/base_llm/managed_resources,
and key managed-ID rewriting on the forwarded custom_llm_provider rather than
the upstream URL's EndpointType. The helper documents why azure and azure_ai
collapse to one "azure" scope (they share the same Azure OpenAI files/batches
surface, so an ID minted on one must resolve on the other) and returns None for
any other provider so a third-party OpenAI-compatible endpoint never triggers
managed-ID minting.

Add TestManagedIdProviderScope covering the azure_ai -> azure collapse and the
non-openai/azure exclusion.

* test(log_db_metrics): assert sanitized event_metadata contract

test_log_db_metrics_success still asserted the legacy event_metadata
shape (function_name/function_kwargs/function_args), which #28909
intentionally removed so that live Prisma clients, OTel spans, and
secrets never land on a service-log span. The decorator now emits only
a sanitized payload: None when no table_name is present, and
{"table_name": ...} when it is. Update the test to verify both branches
of that contract.

* fix(managed_id_rewriter): page provider-scoped file list by offset

The file list scan advanced its cursor with a strict created_at boundary.
When several rows shared a created_at timestamp and a non-matching provider
row sat on the page boundary, the next query skipped the remaining rows at
that timestamp, dropping matching files from the response. Page by a stable
offset over a total order (created_at plus the unique id column) so tied
rows are never skipped or repeated.

* fix(managed_id_rewriter): push file-list provider scope to the DB

The file-list helper had no provider column to query, so it scanned the
table and filtered by decoding each managed ID in the application layer,
capped at 20 pages. For an admin with a large mixed-provider file pool
that cap could truncate a page.

Mint now writes a _passthrough_provider:{provider} marker into
flat_model_file_ids, giving the file table the same DB-queryable provider
scope object rows already get from the namespaced model_object_id. The
list helper pushes the scope into the query so a single round-trip serves
the page. The scan loop, the offset paging, and the cap are gone, so pages
can no longer truncate, leak the other provider, or skip rows that share a
created_at timestamp.

* fix(managed_id_rewriter): deny raw provider IDs that map to another tenant's managed resource

Clients only ever receive managed IDs on passthrough, so a raw file/batch/response ID for another tenant's managed object can only be recovered by decoding that tenant's managed ID. Raw IDs were forwarded upstream untouched (deliberate opt-out), which on a retrieve/cancel/delete executed upstream before the response-side ownership check ran, leaking a cross-tenant action.

Guard raw provider IDs on the input path: when a raw file-/batch_/resp_ ID resolves to a managed row the caller cannot access, return 404 before forwarding. Genuinely unmanaged raw IDs (no DB row) and IDs the caller owns are left untouched, preserving the opt-out.

* test(managed_id_rewriter): cover azure_ai and pre-versioned azure passthrough paths

* fix(managed_id_rewriter): fall back to raw id when persistence fails

A DB persistence failure after a successful upstream create left the
client holding a minted managed ID with no backing row, so every later
resolve returned 404 and the resource was permanently unreachable. Mint
the managed ID only when the row is stored; on persistence failure return
the raw provider id, matching the no-persistence-available fallback, so
the freshly-created resource stays reachable.

* fix(managed_id_rewriter): log only rewritten query param keys

* fix(managed_id_rewriter): use compound (created_at, id) list cursor boundary

A timestamp-only lt/gt cursor boundary skips list rows that share the
cursor row's created_at across a page boundary, silently dropping them.
Compare the unique id (the secondary sort key) alongside created_at so
the page walk stays complete when timestamps tie.

* fix(managed_id_rewriter): converge concurrent object creates on one managed id

* fix(managed_id_rewriter): bound raw-id guard DB lookups per request

The INPUT guard fired one DB lookup for every file-/batch_/resp_ prefixed
string in the path, query, and body. The file-id guard is an unindexed
array-containment scan over LiteLLM_ManagedFileTable, so an authenticated
caller could amplify a single passthrough request into thousands of
full-table scans by packing a body with id-shaped strings.

De-dupe raw ids within a request and cap the distinct guard lookups,
failing closed with 400 instead of skipping the guard. Legitimate callers
hold managed ids (resolved via an indexed unified_*_id lookup, not the
guard), so the cap only trips under abuse.

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

#29256)

* fix(proxy): enforce allowed_passthrough_routes for auth=true pass-through

Pass-through endpoints with auth=true were injected into openai_routes,
so teams with openai_routes access bypassed per-team allowed_passthrough_routes.
Gate auth-enforced pass-through at JWT, virtual-key, and non-admin route checks.

Co-authored-by: Cursor <[email protected]>

* fix(proxy): clarify JWT passthrough denial

Co-authored-by: Cursor <[email protected]>

* fix(proxy): make pass-through auth checks method-aware

Prevent allowlist bypass when the same path is registered with different auth settings per HTTP method.

Co-authored-by: Cursor <[email protected]>

* Fix passthrough route auth checks

* fix(proxy): reject unregistered pass-through HTTP methods

Enforce method-aware JWT checks and return 405 when stale FastAPI routes accept requests outside the current pass-through registry.

Co-authored-by: Cursor <[email protected]>

* fix(proxy): remove duplicate request_method in JWT team lookup

Fixes SyntaxError on proxy startup caused by passing request_method twice to find_team_with_model_access.

Co-authored-by: Cursor <[email protected]>

* Fix passthrough route auth enforcement

* fix(proxy): raise passthrough-specific 403 directly in virtual-key path

* fix(proxy): load team for RBAC role-claim JWT passthrough gating

* Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326)

This reverts the Bedrock CI account migration (#28728). The original account
(888602223428) was put under an AWS security restriction after a leaked key
and has since been reactivated, while the replacement account (941277531214)
lacks access to several models the suites exercise (legacy Bedrock Claude 3
models, Cohere, Nova Canvas image gen, Bedrock batch inference, and flagship
Opus). Pointing CI back at the reactivated account restores that coverage.

This is the exact inverse of #28728: all hardcoded 941277531214 references go
back to 888602223428 (provisioned/imported-model ARNs, AgentCore runtime ARNs
and their suffixes, batch execution role ARN, and the example proxy config),
the S3 buckets revert to litellm-proxy and load-testing-oct, the guardrail IDs
revert to wf0hkdb5x07f and ff6ujrregl1q, the SageMaker endpoint and Knowledge
Base revert to their original ids, and the live-call tests go back to the
legacy model strings. The grid_spec fail_reason workaround for the unentitled
Opus cells is dropped while keeping the unrelated bedrock_effort_ceiling field
added after the migration.

The CircleCI AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env vars still point at
941277531214 and must be set to the reactivated account's fresh credentials
separately via the CircleCI API; AWS_REGION_NAME stays us-west-2.

(cherry picked from commit f11c12d)

* fix(proxy): scope pass-through 405 to registry routes; grant rerank passthrough in rpm tests

The auth=true pass-through 405 guard fired for mapped provider routes
(e.g. /assemblyai/*) that are not in the in-memory registry, since
get_registered_pass_through_route returns None for them while
is_registered_pass_through_route matches via mapped_pass_through_routes.
Only raise 405 when the path is registered but the request method is not
allowed, so mapped provider pass-throughs fall through to the default
target params as before.

The rpm-limit pass-through tests register /v1/rerank with auth=true but
gave their keys no allowed_passthrough_routes, so the new default-deny
returned 403 before the rate limiter ran (non-deterministically,
depending on registry insertion order). Grant the keys explicit
passthrough access so the tests exercise rate limiting under the new
auth model.

* fix(proxy): guard request method lookup against scopes without a method

Starlette's Request.method property reads scope["method"] and raises
KeyError when the scope omits it (e.g. minimally-constructed test
requests). getattr only swallows AttributeError, so the new
_get_request_method helper propagated the KeyError up through
user_api_key_auth and surfaced as a ProxyException. Catch KeyError
(and AttributeError) and fall back to None.

* test(passthrough): pin SERVER_ROOT_PATH in unregistered-method test

test_custom_proxy.py sets os.environ['SERVER_ROOT_PATH'] = '/my-custom-path'
at module import with no cleanup. When that module is collected into the same
xdist worker as this test, the leaked root path is prepended to registered
pass-through paths, so is_registered_pass_through_route misses '/test/path'
and the handler returns 404 instead of the expected 405 (order-dependent).
Pin SERVER_ROOT_PATH to '' so the test is deterministic.

* test(passthrough): restore regression coverage for non-auth-enforced pass-through via llm_api_routes

* fix(proxy): record auth flag in pass-through registry for allowlist enforcement

Auth-enforced pass-through detection inferred enforcement from the FastAPI
dependency stored at registration time. The management create and update
endpoints register routes with dependencies=None even though auth defaults to
true, so is_auth_enforced_pass_through_route treated those DB-created routes as
unenforced. A key allowed for llm_api_routes could then call a management-created
auth-enabled pass-through route without matching allowed_passthrough_routes.

Store the auth setting on each registry entry and read it directly when deciding
whether the allowlist applies, instead of deriving it from dependency metadata.

* fix(proxy): include bool in pass-through registry value type for auth flag

The auth flag stored in _registered_pass_through_routes is a bool, which
was not part of the registry value Union, so mypy rejected the dict literal.
Add bool to the Union and narrow route_methods to a list before the
membership check so the in-operator stays valid.

* fix(proxy): preserve stored auth flag on pass-through endpoint update

model_dump(exclude_none=True) re-included the auth=True default whenever
a partial update omitted auth, silently flipping an existing auth=false
pass-through to auth-enforced and 403ing every team/key without
allowed_passthrough_routes. Merge only explicitly set fields via
exclude_unset so omitted fields keep their stored value.

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

…gnment (#29313)

* fix(mcp): make key.access_group_ids grants additive over team ceiling

A key whose unified access_group_ids grant a private MCP server was
having that grant intersected against its team's MCP ceiling, so a key
in a team scoped to other servers (or with no own scope) lost the
granted server entirely. Resolve access_group_ids once as ungated
additive grants and union them on top of the key/team ceiling instead
of folding them into the key scope that gets intersected.

* test(mcp): align key access-group tests with additive-grant model

The previous commit moved key.access_group_ids resolution out of the
intersected key ceiling (_get_allowed_mcp_servers_for_key) and into the
ungated additive grant path (_get_key_access_group_mcp_server_extras),
unioned on top of the team ceiling. Five tests from #28890/#29195 still
asserted the old gated / in-key-scope contract and failed:

- _get_allowed_mcp_servers_for_key now returns the object_permission
  ceiling only and never resolves access_group_ids; two tests now assert
  the group resolver is not called from that path (with and without an
  object_permission present).
- The extras path is ungated, so a group whose assigned_team_ids /
  assigned_key_ids exclude the caller still contributes its servers.
- The end-to-end test asserts the grant surfaces via the extras path
  rather than the base key path.
- Dropped test_key_access_group_ids_empty_returns_no_extras; the empty
  case is already covered by the extras family's no-groups test.

* feat(auth): gate member access-group assignment on keys behind opt-in

Non-admin team members could attach access_group_ids to keys they create
or update, letting them self-grant resources (MCP servers/models) the team
admin never intended. Add an opt-in KEY_ACCESS_GROUP_ASSIGNMENT team-member
permission (default-deny) enforced at /key/generate and /key/update; proxy
and team admins bypass. Surfaces automatically as a checkbox in the team
Member Permissions UI.

* fix(auth): gate access-group assignment on /key/regenerate too

RegenerateKeyRequest inherits access_group_ids and prepare_key_update_data
persists it, so a non-admin key owner could self-grant access groups by
regenerating. Apply the same opt-in member gate using the existing key's
team.

* test(auth): cover member access-group gate and additive MCP grants

Add unit tests for enforce_member_can_assign_access_groups (deny without
opt-in, allow with opt-in, and proxy-admin / team-admin / non-team-key
bypasses) and for _get_key_access_group_mcp_server_extras (no-auth and
no-resolved-servers return empty, resolved ids are expanded, errors
degrade to no grants).

…eroing counter (#29358)

* fix(reset_budget): write only {spend, budget_reset_at} and stop pre-zeroing counter

ResetBudgetJob's batched update_data path shipped the full key/user/team
model on each reset. Prisma rejects object_permission_id and budget_limits
on the update input type, so any row carrying those fields detonated the
entire batch -- spend never reset, budget_reset_at never advanced. After
v1.84.0 started populating object_permission_id on UI-created keys, this
fires routinely.

_reset_budget_common also zeroed the cross-pod spend counter before the
DB write, so failed resets left enforcement reading 0 from the counter
while the DB still held the over-budget spend, admitting requests past
the cap until the counter naturally re-saturated from new reservations.

Switch the write to per-row narrow updates ({spend, budget_reset_at})
via db.batch_, and move the counter invalidation out of
_reset_budget_common so it only fires after the DB write commits. On
DB-write failure the counter is left untouched, enforcement continues
to block, and the next scheduler tick can retry without leaving a
bypass window.

Fixes #27730.

* fix(reset_budget): address Greptile review on #29358

- Strengthen the bypass-half regression test: replace the for-loop over
  call_args_list (vacuously true when empty) with assert_not_called(),
  so the test would actually flag a re-introduction of counter-zeroing
  via any code path.
- Add the same explanatory docstring on _write_user_reset_updates and
  _write_team_reset_updates that _write_key_reset_updates already has,
  so all three helpers point future maintainers at #27730.

* test(reset_budget): update test_proxy_budget_reset for new batch-write path

Same shape as the previous test_reset_budget_job.py update: keys/users/teams
now write through prisma.db.batch_().<table>.update, not update_data, so the
tests need a batcher mock and updated assertions. Adds:

- _wire_batcher_for_test helper that returns a list which accumulates per-row
  batch updates captured from prisma_client.db.batch_().
- _attrify helper that wraps dict fixtures so getattr(item, "token") works
  alongside the dict item-access the fake_reset_* mocks rely on. The new
  narrow-write helpers use getattr to pull out the row's id, and would
  silently skip plain dicts otherwise.
- Updates 3 partial_failure tests to assert against the batch-call list
  (rows by id, payload contains only {spend, budget_reset_at}) instead of
  update_data.assert_awaited_once + data_list inspection.
- Updates test_reset_budget_continues_other_categories_on_failure: only
  budget + enduser still flow through update_data; key/user/team go through
  the batch path now.
- Wires the batcher mock into 3 service_logger_*_success tests so commit()
  is actually awaitable and the success hook fires.

These tests were silently passing locally only because the editable install
in .venv pointed at the main repo, not the worktree — running pytest with
PYTHONPATH overridden to the worktree (matching CI) reproduces the failures.

* test(e2e): cover PROXY_LOGOUT_URL redirect on Logout

Env-gated spec mirroring the existing serverRootPathRedirect pattern:
when the proxy is booted with PROXY_LOGOUT_URL set, clicking Logout in
the navbar must navigate to that external URL. The standard run_e2e.sh
exports an empty value so the rest of the suite is unaffected; this
spec self-skips unless the env var is populated.

* test(e2e): run PROXY_LOGOUT_URL spec in the suite + harden logout assertions

Boot the e2e proxy with PROXY_LOGOUT_URL set (job-level env in CircleCI and
run_e2e.sh) so proxyLogoutUrl.spec.ts actually runs instead of self-skipping.
Nothing else in the suite performs a logout, so this only affects the behavior
under test.

Harden the spec to verify the logout flow rather than a URL substring:
- wait for /sso/get/ui_settings before clicking so logoutUrl is populated
  (otherwise window.location.href = "" silently reloads same-origin)
- assert a token cookie exists first, and is cleared after logout
- locate the dropdown via getByRole instead of internal antd CSS classes
- stub the external destination and assert on URL origin + path prefix

* test(e2e): assert exact PROXY_LOGOUT_URL on logout redirect

Replace the origin + startsWith(pathname) checks with a single normalized
href comparison. With PROXY_LOGOUT_URL=https://www.example.com the path was
"/", so startsWith("/") matched any path and left path/query/hash
unchecked. Comparing normalized hrefs pins scheme, host, port, path, query
and hash while still tolerating the browser's trailing-slash/default-port
normalization.

When the user has visited both the dev UI (e.g. localhost:3000) and the
proxy UI (e.g. localhost:4000) in the same tab, logging out from the dev
origin produced an infinite logout/login redirect.

The proxy-side LoginPage's "is the user still authenticated?" check
was reading getCookie("token"), which falls back to sessionStorage when
document.cookie has no token. The cross-origin clearTokenCookies() call
from the dev origin can clear cookies on the shared hostname, but cannot
reach sessionStorage on the proxy origin (sessionStorage is per-origin),
so the fallback returned a stale token and LoginPage interpreted the
user as logged in, redirecting back to the dev origin. Dev origin then
saw no cookie and redirected to LoginPage, repeating ~20x per second.

This change introduces getCookieFromDocument(), a cookie-only read with
no sessionStorage fallback, and uses it in LoginPage's already-logged-in
check. The HttpOnly-reverse-proxy defense from PR #23532 is unaffected:
storeLoginToken still writes both the JS cookie at /ui and the
sessionStorage backup, and getCookie still falls back for callers that
want the full read path.

…tcher (#27324)

….20.2) (#29366)

…imit (#28805)

…29369)

The Next.js admin UI is exported with trailingSlash: true, so the proxy
serves /ui/login at /ui/login/index.html and 308s /ui/login → /ui/login/.
The waitForURL predicate used endsWith("/ui/login"), which never matched
the canonicalized URL and timed out after 15s.

This was masked until the build artifacts were regenerated against the
AuthContext fix: the prior bundles still hit the racy redirect path that
fired before proxyBaseUrl was populated, producing /ui/login (no prefix,
no proxy round-trip, no trailing slash) which fortuitously satisfied the
predicate. The first PR to ship the corrected bundle exposed the
assertion bug.

Switch the predicate to includes("/ui/login"); the prefix assertion below
still validates the SERVER_ROOT_PATH preservation that is the actual
contract under test.