…olSpec (BerriAI#29814)

* feat(bedrock): forward strict and additionalProperties to Converse toolSpec

Bedrock Converse supports strict in toolSpec since 2026-02, but
_bedrock_tools_pt only whitelisted type/properties/required/name/description,
so strict: true was silently dropped and Claude-on-Bedrock ignored enum
constraints that GPT and direct-Anthropic honored. Forward strict from the
OpenAI function and additionalProperties from the schema (Bedrock requires
the latter alongside strict), passing each only when present.

https://claude.ai/code/session_01WQjWd8NfUB3vxERwudbHkv

* fix(bedrock): only forward strict tool schemas to Claude on Converse

Nova, Llama and GPT-OSS on Bedrock reject the strict field
(BedrockException 'This model doesn't support the strict field'), and the
GPT-OSS request-body test asserts strict/additionalProperties are stripped.
Forwarding them to every model broke the llm_translation suite, so gate the
forwarding on the anthropic base model since only Claude honours strict
tool schemas on Bedrock.

…per-user env vars (BerriAI#29856)

* fix(mcp): flag missing per-user env vars on the card for every accessible server

The dashboard MCP card grid lists servers via the registry-backed manager
(get_all_mcp_servers_unfiltered for admins in view_all mode, the allowed-context
aggregation otherwise), but the per-user env-var status endpoint that drives the
red "user fields missing" highlight resolved servers through the much narrower
get_all_mcp_servers_for_user, which only returns servers explicitly granted on
the calling key. An admin's dashboard session key carries no per-server MCP
grant, so the status feed came back empty and the card never turned red even
when the logged-in user had not filled in their required variables.

Both surfaces now share a single _resolve_accessible_mcp_servers helper, so the
status feed is computed over exactly the cards the user sees. The helper returns
servers unredacted; the status endpoint needs the raw env_vars and still only
ever reports is_set booleans, never the stored secret values.

* test(mcp): drop dead get_all_mcp_servers_for_user patch from view_all regression test

The bulk status endpoint resolves servers through _resolve_accessible_mcp_servers
now, so the old get_all_mcp_servers_for_user patch in the admin view_all
regression test is never hit. Removing it keeps the test honest about which code
path it exercises.

* feat(ui): add budget duration to edit team member form

Editing a team member created a member budget with no duration, so the
budget never reset. This threads a budget reset period through the edit
flow end to end and reuses the shared duration dropdown so the options
stay in sync with the rest of the UI.

Resolves LIT-2651

* fix(proxy): validate member budget_duration and persist clears

Reject budget_duration values that can't be parsed, are non-positive, or overflow date math before any write, so a bad value can't be persisted and later crash the budget reset job.

Clearing the budget duration in the edit-member form now sends null and clears the column end to end, so the dropdown's clear control reflects a real change instead of being a no-op

* chore(ui): regenerate schema.d.ts for member budget_duration

Adds budget_duration to TeamMemberUpdateRequest/Response in the generated dashboard types so the Check UI API Types Sync gate passes

The Workflow Runs page rendered its table at roughly a quarter of the
available width. Its root container is a flex child of the dashboard
content row but set only padding, min-height and background, so with no
width it shrank to the table's natural content size. Sibling pages
(logs, memory) fill the area with a full-width root; mirror that by
setting width 100% on the container.

Fixes LIT-3636

…odel, and llm_provider fields (BerriAI#27687)

* feat(exceptions): add RateLimitErrorCategory + headers/detail fields on RateLimitError

LiteLLM previously surfaced rate-limit conditions through several unrelated
error classes (RateLimitError, FastAPI HTTPException(429), BaseLLMException).
This commit adds the data model needed to consolidate them under a single
class:

* RateLimitErrorCategory enum exposing four categorical values
  (vendor_rate_limit, vendor_batch_rate_limit, litellm_rate_limit,
  litellm_batch_rate_limit) so callers can switch on the rate-limit source.
* New optional fields on RateLimitError:
  - category (defaults to vendor_rate_limit, preserving today's behavior for
    every existing call site in exception_mapping_utils);
  - headers (preserves retry-after / rate_limit_type / reset_at across the
    proxy boundary instead of dropping them on the floor);
  - detail (mirrors FastAPI HTTPException.detail so the same instance can be
    serialized through both paths).

litellm.RateLimitErrorCategory is re-exported at the package root to match
the existing exception-export pattern.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* feat(proxy): add ProxyRateLimitError unifying RateLimitError + HTTPException

Adds a single proxy-side error class that subclasses BOTH
litellm.exceptions.RateLimitError AND fastapi.HTTPException via cooperative
multiple inheritance.

Why both bases:
* Subclassing RateLimitError lets user code catch every rate-limit source
  with one 'except RateLimitError' and switch on the new .category field.
* Subclassing HTTPException keeps every existing FastAPI plumbing path (the
  isinstance(e, HTTPException) branches in proxy_server.py route handlers,
  FastAPI's own dispatcher, and tests asserting pytest.raises(HTTPException))
  working without modification, and preserves retry-after / rate_limit_type /
  reset_at headers on the wire.

The class declaration order is (HTTPException, RateLimitError) so the MRO
puts HTTPException's no-super-call __init__ ahead of openai's cooperative
__init__ chain — preventing openai.APIError.super().__init__(message) from
landing in HTTPException.__init__(status_code=message).

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* refactor(proxy/hooks): raise ProxyRateLimitError from budget + iteration limiters

Replaces three bare HTTPException(status_code=429, ...) call sites with
ProxyRateLimitError, which is both a RateLimitError (catchable by category)
and an HTTPException (preserves existing FastAPI serialization). Drops the
now-unused HTTPException import in the iteration / per-session limiters.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* refactor(proxy/hooks): raise ProxyRateLimitError from parallel-request limiters

Replaces HTTPException(status_code=429, ...) call sites in the v1 and v3
parallel-request limiters (key/team/user/model/customer rate limits) with
ProxyRateLimitError. Updates the raise_rate_limit_error helper's return type
annotation accordingly.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* refactor(proxy/hooks): raise ProxyRateLimitError from dynamic rate limiters

Replaces HTTPException(status_code=429, ...) call sites in the v1 and v3
dynamic rate limiters (project-level TPM/RPM allocation, model-saturation
checks, priority-based limits, fail-closed guards) with ProxyRateLimitError.
The v3 limiter still imports HTTPException for an unrelated bare 'except
HTTPException:' branch.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* refactor(proxy/hooks): raise ProxyRateLimitError from batch rate limiter

Replaces HTTPException(status_code=429, ...) in batch_rate_limiter._raise_rate_limit_error
with ProxyRateLimitError tagged as RateLimitErrorCategory.LITELLM_BATCH_RATE_LIMIT
so users can distinguish batch-level throttling (which counts requests/tokens
across an uploaded batch input file before submission) from the generic
key/team/user RPM/TPM limiter.

The HTTPException import is retained because the same module raises
HTTPException for unrelated 403/IO error paths.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* test(rate-limit): pin down unified rate-limit error contract

Adds a dedicated test module covering the new RateLimitErrorCategory enum,
RateLimitError.category default + override behavior, ProxyRateLimitError's
dual nature (RateLimitError + HTTPException), and a parametrized regression
guard that asserts every proxy hook module imports the unified class.

The regression guard catches the failure mode the refactor is designed to
prevent: someone re-introducing a bare HTTPException(status_code=429, ...)
in one of the hook modules instead of going through ProxyRateLimitError.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* feat(logging): expose rate-limit category via StandardLoggingPayload

Adds an optional 'error_rate_limit_category' field to
StandardLoggingPayloadErrorInformation, populated from the unified
RateLimitError.category attribute (introduced in the previous commits on
this branch).

Why: the .category attribute is reachable off the raw exception today via
getattr(e, 'category', None), but the structured contract that downstream
custom callbacks / loggers / spend log writers consume is the
StandardLoggingPayload. Without this field, a user building custom
rate-limit metrics on top of callback data has to special-case the raw
exception object — which defeats the purpose of the StandardLoggingPayload
abstraction.

The field is None for non-rate-limit exceptions (so consumers can read it
unconditionally without isinstance checks) and is one of the
RateLimitErrorCategory string values otherwise.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* test(rate-limit): assert StandardLoggingPayload carries the category

Five tests covering: vendor default, explicit litellm_rate_limit and
litellm_batch_rate_limit values, None for non-rate-limit exceptions, and
None when no exception is provided. Pins down the contract that custom
callbacks can read 'error_information.error_rate_limit_category' off the
StandardLoggingPayload to drive custom rate-limit metrics without ever
reaching for the raw exception.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* fix(types): silence mypy [misc] on intentional dual-base attr overlap

mypy emits two [misc] errors on the ProxyRateLimitError class line because
its two bases declare overlapping attributes with related-but-not-identical
annotations:

* status_code: int on starlette HTTPException vs. Literal[429] on openai's
  RateLimitError (every openai status-error subclass narrows it the same
  way and silences pyright with the same convention).
* headers: Mapping[str, str] | None on HTTPException vs. our Optional[
  Dict[str, str]] (the proxy hooks always carry a stringified dict).

Both narrowings are intentional and enforced at construction time. Add a
type: ignore[misc] with an inline explanation rather than relax the
annotations on the parent or change the wire-format guarantees.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* test(rate-limit): add direct hook-invocation tests to lift patch coverage

Adds six end-to-end tests that drive each refactored hook past its
limit and assert the unified ProxyRateLimitError is raised with the
correct category and dual-base shape. Complements the
import-shape-only parametrized guard above by actually executing the
new 'raise ProxyRateLimitError(...)' lines so codecov's patch coverage
sees them as hit.

Hooks covered (one test each):
* parallel_request_limiter v1 — direct call to raise_rate_limit_error()
* parallel_request_limiter v3 — direct call to _handle_rate_limit_error
  with a fabricated OVER_LIMIT response
* max_iterations_limiter — full async_pre_call_hook with mocked agent
  registry, second call exceeds budget=1
* max_budget_limiter — async_pre_call_hook with mocked get_current_spend
* dynamic_rate_limiter v1 — async_pre_call_hook with mocked
  check_available_usage forcing available_tpm == 0
* batch_rate_limiter — direct _raise_rate_limit_error call, asserts
  category is the batch-specific LITELLM_BATCH_RATE_LIMIT (not the
  generic LITELLM_RATE_LIMIT)

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* fix: guard rate_limit_category extraction with isinstance check

* test(rate-limit): cover remaining hook raise sites for codecov

Adds five more direct hook-invocation tests so every PR-touched line
in the proxy hooks is exercised by tests in tests/test_litellm/, which
codecov measures:

* parallel_request_limiter v1 — check_key_in_limits inline raise
  (the second raise site, separate from the raise_rate_limit_error
  helper covered earlier)
* dynamic_rate_limiter v1 — RPM raise branch (TPM branch was already
  covered)
* dynamic_rate_limiter v3 — parametrized over all three raise sites:
  model_saturation_check, priority_model, and the fail-closed
  fallback for an unrecognized descriptor_key
* max_budget_per_session_limiter — full async_pre_call_hook with a
  mocked agent registry and over-budget cached spend

All 42 tests in test_rate_limit_error_unification.py now pass and
together exercise every changed import + raise line across the eight
refactored proxy hooks.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* fix: use computed error_message in ProxyRateLimitError detail

* fix(parallel-request-limiter): drop None from detail; annotate raise_rate_limit_error as NoReturn

The v1 ' raise_rate_limit_error' helper built an unused 'error_message'
variable and then assembled the actual ' detail' via an f-string that
interpolated 'additional_details' verbatim — producing
'Max parallel request limit reached None' when invoked without
arguments (flagged by code review).

Fix the helper to:
- use the constructed 'error_message' as the detail
- annotate the helper as NoReturn since it always raises
- drop the redundant 'raise'/'return' at the two call sites

Add two regression tests covering both the with- and without-
additional_details paths.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* fix(proxy/hooks): drop literal 'None' from raise_rate_limit_error detail

The v1 parallel_request_limiter's raise_rate_limit_error helper has a
long-standing bug: it computes a None-guarded 'error_message' string but
then ignores it and emits an f-string that interpolates the raw
'additional_details' arg. Callers that pass no argument get
'Max parallel request limit reached None' as the user-facing detail.

This commit:
* wires error_message into the detail kwarg so the None-guard actually
  applies and operators see a clean message;
* changes the return-type annotation from ProxyRateLimitError to NoReturn
  (the function always raises) so type-checkers know callers after this
  invocation are unreachable.

Greptile P1 + P2 review feedback on PR BerriAI#27687.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* fix(types): demote TypedDict floating string to a # comment

A string literal placed after a field declaration in a TypedDict body is
not a per-field docstring — it's an orphaned string expression Python
discards. Tools like mypy / pyright that inspect TypedDict fields won't
surface that text either.

Move the documentation for error_rate_limit_category to a real comment
so the intent is visible to readers and type-checker tooling without
the misleading docstring framing.

Greptile P2 review feedback on PR BerriAI#27687.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* security(exceptions): do not auto-copy vendor response headers to e.headers

A vendor 429 response can set arbitrary headers (Set-Cookie, CORS
overrides, …). Previously, when RateLimitError was constructed with only
a 'response=' (no explicit 'headers=' kwarg), self.headers fell back to
a copy of response.headers. If a downstream proxy serializer ever
forwarded e.headers to the client, a malicious upstream could inject
browser-interpreted headers for the proxy origin.

Drop the fallback. Only headers passed explicitly via the headers= kwarg
make it onto self.headers (proxy hooks pass retry-after etc. — they
control what's surfaced). Vendor response headers stay reachable on
e.response.headers for callers that explicitly want them.

Today's proxy_server.py route handlers don't actually forward e.headers
on the wire (they construct ProxyException without passing headers), so
no current behavior changes — this is a defensive narrowing so the
fallback can never be turned into a vector when someone wires
e.headers through later.

Veria-AI security review feedback on PR BerriAI#27687.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* test(rate-limit): regression guards for review-pass fixes

Pins down the three review-pass fixes:

* test_parallel_request_limiter_v1_helper_no_additional_details — calls
  raise_rate_limit_error() with no args and asserts the detail does NOT
  contain the literal string 'None'. Pre-fix, callers got 'Max parallel
  request limit reached None'.
* test_rate_limit_error_does_not_auto_copy_response_headers — passes a
  vendor httpx.Response with a Set-Cookie header to RateLimitError
  WITHOUT an explicit headers= kwarg, asserts self.headers stays None
  (no leak), then re-checks that an explicit headers= kwarg DOES
  populate self.headers. Vendor headers remain reachable on
  e.response.headers for callers that explicitly want them.
* The existing v1-helper test now also asserts the additional_details
  string makes it through to the detail.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* feat(rate-limit): add orthogonal RateLimitType (requests/tokens/concurrent_requests/budget/max_iterations)

trho's last ask in the LIT-2968 thread: distinguish rate-limit failures by
the dimension that was exceeded, not just by who rate-limited (vendor vs.
litellm). Adds:

- RateLimitType str-enum exposed at `litellm.RateLimitType` with values
  requests / tokens / concurrent_requests / budget / max_iterations.
- `rate_limit_type` kwarg on litellm.RateLimitError + ProxyRateLimitError;
  None default so existing callers (vendor-429 path in exception_mapping_utils)
  remain a no-op.
- StandardLoggingPayloadErrorInformation.error_rate_limit_type so custom
  callbacks can split rate-limit failures by cause without parsing free-text
  error messages. Mirror to error_rate_limit_category extraction in
  get_error_information(); single isinstance(RateLimitError) check covers both.
- map_v3_rate_limit_type() helper to collapse the v3 limiter's internal labels
  ("requests", "tokens", "max_parallel_requests") onto the public enum so
  the v3 limiter and dynamic_rate_limiter_v3 share one mapping. Defensive
  None on unknown values rather than silently picking a wrong dimension.

Co-authored-by: Mateo Wang <[email protected]>

* feat(proxy/hooks): wire rate_limit_type onto every limiter raise site

Each refactored proxy hook now populates rate_limit_type with the dimension
that actually tripped the limit, so downstream consumers (custom callbacks,
prometheus exporters via the StandardLoggingPayload) can split key/team/user
rate-limit failures by cause:

- parallel_request_limiter (v1): detect dimension from current vs. limit in
  the post-cache branch (concurrent_requests > tokens > requests, matches the
  boolean condition order). Base case (current is None, one limit set to 0)
  picks the most-specific zero. raise_rate_limit_error() helper accepts an
  explicit rate_limit_type kwarg with CONCURRENT_REQUESTS default (matches
  every existing internal call site, including the global-limit branch).
- parallel_request_limiter (v3): forward status["rate_limit_type"] through
  map_v3_rate_limit_type() so "max_parallel_requests" → CONCURRENT_REQUESTS
  for the public field while the raw v3 jargon stays on the HTTP header for
  wire-format backward compat.
- dynamic_rate_limiter (v1): TPM-zero → TOKENS, RPM-zero → REQUESTS. Pass
  data["model"] through so callbacks see the model that hit the limit
  (addresses the secondary "provider missing" complaint in the original
  Slack thread, partially — the model is what dashboards typically split on).
- dynamic_rate_limiter (v3): forward status["rate_limit_type"] via
  map_v3_rate_limit_type() at every raise site (model_saturation_check,
  priority_model, fail-closed unknown-descriptor guard). Also pass model.
- batch_rate_limiter: limit_type is hard-typed "requests"|"tokens" — map
  directly without going through the helper's None branch.
- max_budget_limiter, max_budget_per_session_limiter: BUDGET.
- max_iterations_limiter: MAX_ITERATIONS.

Co-authored-by: Mateo Wang <[email protected]>

* test(rate-limit): cover RateLimitType enum, hook wiring, and StandardLoggingPayload propagation

27 new tests across five new test classes:

- TestRateLimitType: enum exposed at litellm.RateLimitType, all five values
  defined, RateLimitError default is None (vendor 429 path makes no claim
  about which dimension), accepts both string and enum forms with
  str-coercion guarantee for downstream JSON serializers.
- TestProxyRateLimitErrorType: ProxyRateLimitError default is None, accepts
  string or enum, doesn't break existing callers that pass nothing.
- TestMapV3RateLimitType: pins each v3-internal → public-enum mapping
  (tokens, requests, max_parallel_requests → concurrent_requests, unknown
  → None) so a future v3 refactor can't silently swap dimensions.
- TestStandardLoggingPayloadCarriesType: the new error_rate_limit_type
  field reaches the structured payload for both ProxyRateLimitError and
  plain RateLimitError, is None when unspecified, and is None for
  non-rate-limit exceptions (symmetric with error_rate_limit_category).
- TestProxyHooksWireTypeCorrectly: drives the actual raise sites in the
  v1 parallel_request_limiter helper, the v3 _handle_rate_limit_error
  (both "tokens" and "max_parallel_requests" paths), and the batch
  limiter (both tokens and requests paths) — coverage tools see the new
  rate_limit_type= kwargs as exercised, not just the import shape.

Co-authored-by: Mateo Wang <[email protected]>

* test(rate-limit): cover _coerce_message branches and v1 dimension detection

Drives the patch coverage on the new orthogonal RateLimitType wiring up
to (or close to) 100% on the touched files.

ProxyRateLimitError._coerce_message — was 22% covered, now 100%:
* nested {error: {message}} dict
* nested {message: {message}} dict (alt key)
* dict without 'error'/'message' keys → JSON dump fallback
* non-JSON-serializable dict value → str() fallback
* non-string non-mapping detail (int) → str() coercion

v1 parallel_request_limiter dimension detection — was 0% covered, now
exercised across 6 parametrized cases:
* check_key_in_limits else-branch: current at concurrent / TPM / RPM cap
  → asserts rate_limit_type is concurrent_requests / tokens / requests.
* check_key_in_limits base case (current is None): max_parallel_requests
  / tpm_limit / rpm_limit set to 0 → asserts the most-specific zero
  attribution wins per the helper's order.

LIT-2968

Co-authored-by: Mateo Wang <[email protected]>

* feat(proxy/hooks): add ProxyHTTPRateLimitError + provider resolver

Introduces a small helper layer used by every proxy-side rate-limit
hook so that the 429 they raise carries a populated llm_provider /
model — instead of an empty exception.llm_provider that downstream
loggers (Prometheus failure metric, observability callbacks) read as
'no provider attribution'.

ProxyHTTPRateLimitError inherits from both fastapi.HTTPException
(so the proxy server still renders it as a 429) and
litellm.exceptions.RateLimitError (so isinstance checks and
PrometheusLogger._get_exception_class_name pick up llm_provider).
We deliberately don't call RateLimitError.__init__ — it constructs
an httpx.Response we don't need and would just add failure surface;
attribute parity is what downstream consumers care about.

resolve_llm_provider_for_rate_limit() wraps litellm.get_llm_provider
defensively. Internal limiter hooks fire from async_pre_call_hook —
well before get_llm_provider runs anywhere else in the request
lifecycle — so we have to call it ourselves at raise time. If the
model is missing or unparseable (alias, router-only model) we fall
back to llm_provider='litellm_proxy' rather than letting a second
exception leak out and break the request path.

Co-authored-by: Mateo Wang <[email protected]>

* fix(proxy/hooks): populate llm_provider on parallel-request 429s

Both v1 and v3 parallel-request limiters fired bare HTTPException(429)
from inside async_pre_call_hook. The downstream Prometheus failure
metric reads exception.llm_provider via _get_exception_class_name —
the empty value showed up as exception_class='HTTPException' and
left model_id='None' on the time series.

Threads requested_model through every raise site in:

* parallel_request_limiter.py:
  - check_key_in_limits (the per-key/per-model/per-user/per-team/
    per-customer over-limit path)
  - raise_rate_limit_error (zero-limit + global_max_parallel_requests
    paths) — now takes an optional requested_model kwarg
* parallel_request_limiter_v3.py:
  - _handle_rate_limit_error (the OVER_LIMIT translator), called
    from both the should_rate_limit pre-check and the TPM
    reservation path

Resolved via resolve_llm_provider_for_rate_limit so unknown / missing
models silently fall back to llm_provider='litellm_proxy' instead of
breaking the request path with a second exception.

Co-authored-by: Mateo Wang <[email protected]>

* fix(proxy/hooks): populate llm_provider on dynamic-rate-limit 429s

Same plumbing change as the parallel limiters, applied to both
dynamic_rate_limiter (v1) and dynamic_rate_limiter_v3:

* v1: TPM-zero and RPM-zero paths in async_pre_call_hook now resolve
  data['model'] -> (model, llm_provider) once and pass it into both
  raises.
* v3: All three raise sites in _check_rate_limits — the
  model_saturation_check enforced raise, the priority_model
  enforced raise, and the fail-closed unknown-descriptor branch —
  now attribute the 429 to the actual provider.

Falls back to llm_provider='litellm_proxy' when the model can't be
resolved.

Co-authored-by: Mateo Wang <[email protected]>

* fix(proxy/hooks): populate llm_provider on batch-rate-limit 429s

batch_rate_limiter._raise_rate_limit_error now takes a
requested_model kwarg threaded from data['model'] in
_check_and_increment_batch_counters. The batch-creation 429 is what
gets raised when the input file's tokens/requests count would push
the per-key TPM/RPM window over its limit.

Co-authored-by: Mateo Wang <[email protected]>

* fix(proxy/hooks): populate llm_provider on budget/iterations 429s

Final batch of internal raise sites — the user/session-budget and
max-iterations hooks. Same pattern: resolve data['model'] once at
raise time, attach to ProxyHTTPRateLimitError so Prometheus and
observability callbacks can attribute the 429.

Hooks updated:
* max_budget_limiter (per-user max_budget exceeded)
* max_iterations_limiter (per-session agent iteration cap)
* max_budget_per_session_limiter (per-session dollar cap)

All three fall back to llm_provider='litellm_proxy' when data['model']
is missing or unparseable. Drops the now-unused HTTPException import
from each module.

Co-authored-by: Mateo Wang <[email protected]>

* test(proxy/hooks): pin provider field on internal rate-limit 429s

Regression coverage for the 'provider field missing' bug across every
proxy-side rate-limit hook + the helper layer:

* ProxyHTTPRateLimitError class shape (HTTPException + RateLimitError,
  dict-detail stringification, None-provider normalization).
* resolve_llm_provider_for_rate_limit happy paths
  (gpt-4o-mini, anthropic/..., bedrock/...) plus all three fallback
  branches (None, '', unknown name) plus a 'get_llm_provider raises'
  case that asserts we swallow the secondary exception.
* For each limiter (parallel v1/v3, dynamic v1/v3, batch,
  max_budget, max_iterations, max_budget_per_session): assert the
  raised exception is a RateLimitError carrying the resolved
  model + llm_provider, and a sibling test that asserts the
  fallback path returns 'litellm_proxy' without leaking a second
  exception.
* Two PrometheusLogger._get_exception_class_name pins so the
  Prometheus failure metric label flips from 'HTTPException' to
  'Openai.ProxyHTTPRateLimitError' (or 'Litellm_proxy.*' on
  fallback) — that's what dashboards consume.

Co-authored-by: Mateo Wang <[email protected]>

* perf(proxy/hooks): defer provider resolution to over-limit branches

* fix: use error_message in raise_rate_limit_error to avoid literal 'None' in detail

* Consolidate rate_limiter_utils imports in dynamic_rate_limiter

* fix(proxy): set num_retries/max_retries on ProxyHTTPRateLimitError

ProxyHTTPRateLimitError inherits from RateLimitError but did not call
RateLimitError.__init__, so num_retries/max_retries were never set.
When Starlette's HTTPException lacks __str__, MRO falls through to
RateLimitError.__str__, which unconditionally reads these attributes
and raises AttributeError during logging/traceback formatting.
Initialize them to None defensively.

* fix(mypy): silence base-class status_code conflict on ProxyHTTPRateLimitError

HTTPException declares 'status_code: int' while openai.RateLimitError
(via APIStatusError) declares 'status_code: Literal[429] = 429'. Mypy
flags the multi-base override as [misc] in CI lint. The runtime semantics
are fine (we set self.status_code in __init__), so silence the
class-level annotation conflict with a targeted ignore.

Co-authored-by: Mateo Wang <[email protected]>

* fix: annotate batch limiter _raise_rate_limit_error as NoReturn

* feat(prometheus): rate-limit category/type labels + exception_class back-compat (follow-up to BerriAI#27687) (BerriAI#27706)

* feat(prometheus): add rate_limit_category and rate_limit_type labels

Adds two new labels to litellm_proxy_failed_requests_metric so dashboards
can split 429s by rate-limit source (vendor vs. litellm-internal) and by
the dimension that was exceeded (requests/tokens/concurrent_requests/
budget/max_iterations) without parsing free-text error messages.

Closes the Prometheus side of LIT-2718. The unified RateLimitError.category
and .rate_limit_type fields landed in PR BerriAI#27687 but were only surfaced on
StandardLoggingPayload (custom-callback channel); this exposes them on
the metric label set as well.

Both labels are populated only when the underlying exception is a
litellm.RateLimitError; non-rate-limit failures keep them empty.

Co-authored-by: Mateo Wang <[email protected]>

* feat(prometheus): populate rate-limit labels + preserve exception_class back-compat

Two coupled changes in the Prometheus integration:

1. async_post_call_failure_hook now extracts the new RateLimitError
   .category / .rate_limit_type fields (added in PR BerriAI#27687) via a
   _extract_rate_limit_labels helper and forwards them through
   UserAPIKeyLabelValues onto litellm_proxy_failed_requests_metric.
   Empty for non-rate-limit failures.

2. _get_exception_class_name special-cases ProxyRateLimitError and
   keeps emitting 'HTTPException' for the exception_class label.
   Without this shim, ProxyRateLimitError (which multi-inherits from
   HTTPException + RateLimitError) would silently flip the label
   from 'HTTPException' (the historical value for proxy-side 429s)
   to 'ProxyRateLimitError', breaking existing dashboards / alerts
   that key off exception_class='HTTPException'. Distinguishing
   vendor vs. litellm 429s is now the job of the new
   rate_limit_category label.

Co-authored-by: Mateo Wang <[email protected]>

* test(prometheus): cover rate-limit labels and exception_class back-compat

Adds 19 tests across:
- enum / label-list registration
- _extract_rate_limit_labels for vendor RateLimitError, ProxyRateLimitError,
  non-rate-limit and None inputs (incl. parametrized over every
  RateLimitErrorCategory x RateLimitType combo)
- _get_exception_class_name back-compat: ProxyRateLimitError keeps the
  legacy 'HTTPException' string while vendor RateLimitError keeps the
  historical 'Provider.ClassName' format
- end-to-end through async_post_call_failure_hook with both
  ProxyRateLimitError and vendor RateLimitError, asserting both new
  labels populate and exception_class stays back-compat

Co-authored-by: Mateo Wang <[email protected]>

* fix(prometheus): tolerate missing fastapi in lazy ProxyRateLimitError import

Address greptile feedback:
- async_post_call_failure_hook docstring: drop the stale labelnames listing
  and reference PrometheusMetricLabels.litellm_proxy_failed_requests_metric
  as the source of truth so the doc cannot drift from the actual labelset.
- _get_exception_class_name: guard the lazy ProxyRateLimitError import with
  ImportError so router-side fallback callsites don't blow up in non-proxy
  installs that don't have fastapi (a transitive dep of
  proxy.common_utils.proxy_rate_limit_error). Behavior is unchanged when
  fastapi is available.

Also fix the existing enterprise callback test that asserted the old
labelset on litellm_proxy_failed_requests_metric — it now expects the new
rate_limit_category / rate_limit_type labels populated for vendor 429s.

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Mateo Wang <[email protected]>

* fix(bugbot): simplify rate-limit label coercion + guard None detail

- prometheus.py _extract_rate_limit_labels: RateLimitError.__init__ already
  normalizes category/rate_limit_type to plain str, so the getattr(.value)
  + isinstance dance was dead code. Reduce to str(value) if not None.
- proxy_rate_limit_error.py _coerce_message: short-circuit None to ''
  instead of falling through to str(None) = 'None', which produced the
  literal message 'litellm.RateLimitError: None'.

* fix(rate-limit): surface unified category/type fields on BudgetExceededError

The most common budget cap (virtual-key max_budget enforcement in
auth_checks.py) raises litellm.BudgetExceededError, a bare Exception
subclass that bypassed the unified rate-limit error class introduced
by PR BerriAI#27687. Custom callbacks reading
StandardLoggingPayload.error_information saw category=None and
rate_limit_type=None for these 429s, missing the most common budget
case (team / org / end-user budgets all hit the same code path).

Surface the fields off BudgetExceededError as plain attributes:
- category = RateLimitErrorCategory.LITELLM_RATE_LIMIT
- rate_limit_type = RateLimitType.BUDGET
- llm_provider = "" (or caller-supplied)

Switch get_error_information and _extract_rate_limit_labels from
isinstance(RateLimitError) gating to duck-typed attribute reads,
guarded by membership in the rate-limit enums so unrelated third-party
exceptions exposing a .category attribute can't leak garbage values
into the payload.

This is strictly additive: BudgetExceededError keeps its bare-Exception
base class, so `except BudgetExceededError:` handlers keep firing and
`except RateLimitError:` does not start catching budget errors.

* fix(rate-limit): validate enum membership at duck-typed read sites + enrich BudgetExceededError llm_provider

Two follow-ups uncovered during the second QA pass on PR BerriAI#27687:

1. Guard third-party `.category` / `.rate_limit_type` attribute leakage.
   The duck-typed read in `get_error_information` and
   `_extract_rate_limit_labels` would forward any string attribute named
   `category` / `rate_limit_type` on an unrelated third-party exception
   into the StandardLoggingPayload and Prometheus labels — silently
   mislabeling custom-callback payloads and blowing out Prometheus label
   cardinality. Add `validate_rate_limit_category` /
   `validate_rate_limit_type` helpers that gate on the documented enum
   value sets; non-matching values are dropped to None.

2. Enrich BudgetExceededError.llm_provider from request_data.
   Budget checks live in tenant-scoped helpers (key / team / org / tag /
   end-user / project) that don't see the request model, so the
   BudgetExceededError they raise carried llm_provider="" — leaving
   custom-metrics consumers without provider attribution for the most
   common 429 case. Resolve it once at the central
   UserAPIKeyAuthExceptionHandler seam, before post_call_failure_hook
   fires, so the StandardLoggingPayload the callback sees has the same
   provider attribution as RPM/TPM 429s.

Regression tests pin both: 4 leakage tests + 4 enrichment tests. The
leakage tests would fail under the pre-validation version of either read
site; the enrichment tests would fail if the handler skipped the
resolver call.

* fix(rate-limit): resolve router model_name aliases to real provider (BerriAI#27914)

* fix(rate-limit): resolve router model_name aliases to real provider

For nearly every real LiteLLM proxy deployment the request model is a
router model_name alias (e.g. 'tpm-locked' -> litellm_params.model:
openai/gpt-4o-mini), and 'litellm.get_llm_provider' doesn't know about
router aliases — it raises 'LLMProviderNotProvidedError'. The resolver
then fell through to the defensive 'litellm_proxy' fallback, so the
'llm_provider' field this PR adds was effectively always
'litellm_proxy' in the field, defeating its purpose for the most common
proxy configuration.

Add a router-alias fallback step: when 'get_llm_provider' raises, scan
the active 'llm_router.model_list' for a deployment whose 'model_name'
matches the request model and resolve from its 'litellm_params.model'
instead. If multiple deployments share the same alias (load-balancing
case) the first one wins — every deployment under one alias should
agree on provider in any sensible config, and 'first' is deterministic
so the Prometheus label stays stable.

Defensive throughout: an uninitialized router, a malformed deployment,
a 'litellm_params.model' that itself fails 'get_llm_provider' — every
branch falls through to the existing 'litellm_proxy' fallback rather
than letting a secondary exception escape and mask the rate-limit
error we're trying to surface.

Tests:
  - test_router_alias_resolves_to_underlying_provider: alias
    'tpm-locked' -> 'openai/gpt-4o-mini' produces provider='openai',
    model='gpt-4o-mini'.
  - test_router_alias_with_multiple_deployments_uses_first.
  - test_router_alias_unknown_falls_back.
  - test_router_alias_with_malformed_deployment_falls_back.
  - Existing fallback test updated to also stub
    'litellm.proxy.proxy_server.llm_router' so it exercises the
    full 'no resolution anywhere' path.

Co-authored-by: Mateo Wang <[email protected]>

* fix(rate-limit): harden router alias resolver + test isolation

- Wrap _resolve_provider_from_router_alias loop in top-level try/except so
  a non-iterable model_list / unexpected deployment shape can't escape and
  mask the 429 with a 500.
- Type-check litellm_params before .get() to handle non-dict truthy values.
- Patch llm_router=None in the parametrized fallback test so a router left
  by another test in the session can't redirect the unknown-model path.

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Mateo Wang <[email protected]>

* fix(bugbot): preserve "BudgetExceededError" Prometheus label

Adding llm_provider to BudgetExceededError (so callbacks get provider
attribution from StandardLoggingPayload) made the provider-prefix step in
_get_exception_class_name silently flip the label from "BudgetExceededError"
to e.g. "Openai.BudgetExceededError", breaking dashboards keyed on the
historical value.

Short-circuit BudgetExceededError in _get_exception_class_name the same way
ProxyRateLimitError already is. Provider/category attribution still lands on
the new rate_limit_category / rate_limit_type labels.

* test: fix invalid 'rpm' rate_limit_type in v3 limiter test mocks

The v3 rate limiter only emits 'requests', 'tokens', or
'max_parallel_requests'. Using 'rpm' caused map_v3_rate_limit_type to
return None, leaving the expected RateLimitType.REQUESTS untested.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(bugbot): hoist provider resolver + opt-in prom rate-limit labels

- dynamic_rate_limiter.py: hoist resolve_llm_provider_for_rate_limit
  above the TPM/RPM if/elif so the lookup runs once per request, matching
  the pattern in dynamic_rate_limiter_v3.py.
- prometheus.py: gate the new rate_limit_category / rate_limit_type
  labels on litellm_proxy_failed_requests_metric behind
  litellm.prometheus_emit_rate_limit_labels (default False). Mirrors the
  existing prometheus_emit_stream_label opt-in. Preserves the metric's
  pre-unification label set so existing dashboards / recording rules
  keep matching after upgrade; operators can enable the new labels once
  downstream consumers include them.
- Tests updated: default-off back-compat case, opt-in path enables the
  flag before asserting label presence.

* fix: stabilize prometheus label sets and drop redundant model normalization

- Cache PrometheusLogger.get_labels_for_metric per metric_name so that
  the label set used to construct counters at __init__ time stays in
  sync with the label set used at increment time, even if module-level
  toggles like prometheus_emit_rate_limit_labels or
  prometheus_emit_stream_label are flipped at runtime. Without this,
  toggling these flags after the logger was created would cause
  ValueError from prometheus_client because the runtime labels would
  not match the counter's declared labelnames.
- Drop redundant 'model or ""' guard in ProxyRateLimitError.__init__
  where model is already normalized one step earlier.

Co-authored-by: Yassin Kortam <[email protected]>

* perf(dynamic_rate_limiter): only resolve provider when rate limit hit

Co-authored-by: Yassin Kortam <[email protected]>

* test(prometheus): clear cached metric labels after toggling rate-limit flag

The PrometheusLogger caches each metric's label set at construction
time so that labels used at counter.labels(...) time stay consistent
with the labels the metric was registered with. The enterprise
async_post_call_failure_hook test toggles
litellm.prometheus_emit_rate_limit_labels = True AFTER the fixture
has already built the logger, so without invalidating the cache the
rate_limit_category / rate_limit_type labels never reach the mocked
counter and the assert_called_once_with check fails.

Co-authored-by: Yassin Kortam <[email protected]>

* test: fix CI failures from prom label cache + flaky time-window assertion

PrometheusLogger.get_labels_for_metric now caches the per-metric label
set at first read so the labels passed to counter.labels(...) stay in
lock step with the labels the counter was registered with. This broke
two existing test patterns:

- test_prometheus_labels.py: tests bind the real method onto a
  MagicMock, but MagicMock auto-creates a Mock for _cached_metric_labels
  whose .get(...) returns a truthy Mock — treated as a populated cache
  and returned as the label set, producing empty filtered labels and
  KeyError on labels["requested_model"] / ["route"]. Seed real {}
  containers for _cached_metric_labels and label_filters before binding.

- test_prometheus_logging_callbacks.py::test_set_team_budget_metrics_with_custom_labels:
  the fixture builds the logger before the test monkeypatches
  litellm.custom_prometheus_metadata_labels, so the cached label set
  never picks up the new metadata labels. Clear the cache after the
  monkeypatch (same pattern already used for the rate-limit toggle in
  test_async_post_call_failure_hook).

UI: view_logs/index.test.tsx "Last Minute" window assertion is off by
one at the minute boundary. start_date is floored to the minute, so the
dropped sub-minute fraction can push the truncated-seconds diff up to
(minMinutes+1)*60 exactly when the click lands near a minute rollover.
Switch the upper bound to toBeLessThanOrEqual.

* feat(otel-v2): surface rate_limit_category + rate_limit_type on failed LLM-call spans

PR BerriAI#28909 introduced the typed v2 OTel engine that builds spans from
StandardLoggingPayload, with SpanError carrying error_type + message and
the genai mapper stamping error.type onto every failed LLM-call span.
This PR's earlier commits added error_rate_limit_category and
error_rate_limit_type to the same StandardLoggingPayload.error_information
the v2 engine reads — but neither field reached a span attribute, so v2
OTel traces stayed opaque about *why* a 429 fired (vendor vs litellm,
RPM vs TPM vs concurrent vs budget vs max_iterations) even after the
custom-callback and prometheus surfaces gained that decomposition.

Three coupled changes:

1. semconv.py: add LiteLLM.ERROR_RATE_LIMIT_CATEGORY /
   LiteLLM.ERROR_RATE_LIMIT_TYPE under the litellm.* vendor namespace
   (no GenAI semconv equivalent exists for who-rate-limited /
   which-dimension).

2. payloads.py: extend SpanError with rate_limit_category +
   rate_limit_type, populated by _parse_error() from the same
   error_information.error_rate_limit_* fields the custom-callback
   channel and prometheus rate_limit_category / rate_limit_type labels
   read. Single source of truth across all three observability surfaces.

3. mappers/genai.py: stamp the two attributes on the LLM-call span when
   present. drop_none guarantees they stay absent (not 'None') for
   non-rate-limit failures so trace consumers can read them
   unconditionally.

Three regression tests in test_otel_v2_emitter.py pin: a vendor /
litellm-internal RateLimitError lands category=litellm_rate_limit +
rate_limit_type=requests on the span; a BudgetExceededError lands
rate_limit_type=budget; a non-rate-limit failure (BadRequestError)
keeps the rate_limit_* attributes absent. Mutation-tested against
reverting either the SpanError extension or the _parse_error read site
— both new tests fail under either mutation.

Co-authored-by: Mateo Wang <[email protected]>

* test: align prometheus user-budget + logs quick-select tests with merged code

The merge into this branch left two test patterns out of step with the code
they exercise.

test_set_user_budget_metrics_includes_user_email_and_alias_labels_when_opted_in
flipped litellm.prometheus_user_budget_label_include_email_alias after the
fixture had already built the PrometheusLogger. get_labels_for_metric now
snapshots each metric's label set at construction time, so the runtime flip
no longer reached the cached labels. Enable the flag before constructing the
logger, matching how the proxy applies config at startup.

view_logs/index.test.tsx referenced uiSpendLogsCall and moment without
importing them, and the merged index.tsx now fetches through
useLogFilterLogic (the hook the file stubs out) rather than calling
uiSpendLogsCall directly. Add the imports and restore the real hook for the
Quick Select window assertions so the call is actually observed.

* refactor(otel/v2): drop rate-limit decomposition from the LLM-call span

Proxy-side rate limits (litellm_rate_limit, budget, max_iterations) are
rejected at the gate before any upstream call, so async_post_call_failure_hook
tags the synthetic failure log with LITELLM_LOGGING_NO_UPSTREAM_LLM_CALL and the
v2 OTel logger never opens an LLM-call span for them; the
litellm.error.rate_limit_category / litellm.error.rate_limit_type attributes
were dead for exactly the cases they were meant to surface. The only failure
that does open an LLM-call span carrying a RateLimitError is a vendor 429, where
rate_limit_type is always None and the category just restates
error.type=RateLimitError.

The decomposition still reaches downstream consumers through
StandardLoggingPayload.error_information.error_rate_limit_* and the prometheus
rate_limit_category / rate_limit_type labels, both unchanged.

Removes the SpanError fields, the _parse_error reads, the genai mapper
attributes, the semconv keys, and the three span tests that asserted a scenario
that never reaches the mapper in production.

* fix(batch_rate_limiter): map max_parallel_requests to concurrent_requests

* refactor(prometheus): drop transitive fastapi import from _get_exception_class_name

Read the legacy exception_class label from a prometheus_exception_class_name
marker on ProxyRateLimitError instead of importing the proxy module, keeping
the integrations layer free of a transitive fastapi dependency.

* chore(ui): sync schema.d.ts with unified rate-limit error spec

The ProxyRateLimitError docstring flows into the proxy OpenAPI spec's 429
response description, so the generated dashboard types were out of sync.
Regenerated via npm run gen:api (Check UI API Types Sync).

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Mateo Wang <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>

…ed (BerriAI#29872)

The Guardrails page hardcoded defaultActiveKey="submitted", so admins
landed on the "Submitted Guardrails" tab (the last of their four tabs)
instead of the primary view. The original intent was for non-admins,
whose only tab is Submitted Guardrails, to default there; admins should
open on their first tab.

Make the default role-aware: admins default to the first tab (Guardrail
Garden), non-admins keep Submitted Guardrails.

… subclass (BerriAI#29869)

…I types (BerriAI#29885)

The dashboard calls UI-internal proxy routes that the public /openapi.json hides with include_in_schema=False, so they never reached schema.d.ts and could not be typed. The type generator now force-includes those routes when it dumps the spec for openapi-typescript; this mutates a throwaway interpreter only, so the spec the proxy actually serves is unchanged.

Regenerates schema.d.ts so 86 internal route families (for example /v2/model/info, /global/spend/*, /config/*, /v2/login, /sso/*) are now typed, with no public route removed. This unblocks migrating the dashboard's data fetching onto the typed $api client.

Branch CI note: schema.d.ts is generated; CI regenerates and diffs it via the same gen:api script.

…29900)

* feat(proxy): publish /v2/model/info in Swagger OpenAPI spec

Expose the v2 model info endpoint in /docs by removing include_in_schema=False
and documenting query parameters used by the admin UI and proxy CLI consumers.

Co-authored-by: Cursor <[email protected]>

* chore(ui): regenerate schema.d.ts for /v2/model/info OpenAPI docs

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>

…AI#29949)

Consolidate the three hand-synced copies of the migrated-pages map
(LEGACY_REDIRECTS in app/page.tsx, MIGRATED_PAGES in the dashboard layout,
and MIGRATED_PAGES in leftnav) into one shared module,
src/utils/migratedPages.ts, which also owns the migratedHref helper. Delete
the unused, incomplete Sidebar2 prototype.

No runtime behavior change: the map is still empty and Sidebar2 had no
importers, so this is pure deduplication ahead of the per-page App Router
migration. Follow-up work will unify the remaining base-URL builders
(layout's withBase and page.tsx's redirect) onto migratedHref.

…riAI#29958)

The provider logo base path was relative ("../ui/assets/logos/"). With
trailingSlash enabled, the public model hub is served at /ui/model_hub_table/,
so the browser resolved the base to /ui/ui/assets/logos/ (a doubled /ui/), which
404s every icon. The authenticated hub renders inside the single-level /ui/ SPA
route where the relative path resolves correctly, so only the public hub broke.
Make the base root-absolute so it resolves at any route depth.

…9871)

The create guardrail modal used antd's default maskClosable, so clicking
outside it dismissed the modal and reset every field the user had entered.
Setting maskClosable={false} keeps the modal open; it now closes only via
the explicit close button or Cancel, matching the other form modals in the
dashboard

…rriAI#29870)

The key edit page showed the default key type (no allowed_routes restriction)
as "Default", while the key creation form already labels the same value
"Full Access". Align the edit page to the create form so the two surfaces agree
on both the label and its description.

…BerriAI#29953)

* fix(ui): unify migrated-route URLs and cut the API Reference page over to path routing

Route all migrated-page navigation through one /ui-prefixed, serverRootPath-aware
builder in migratedPages.ts (migratedHref/legacyPageHref/legacyKeyForPathname),
replacing the three divergent base-URL constructions that lived in the dashboard
layout's withBase, leftnav, and the page.tsx redirect. The previous migratedHref
read NEXT_PUBLIC_BASE_URL, which no build sets, so it produced URLs without the
/ui prefix the app is served under; every other internal link hardcodes /ui and
this now matches that convention.

Remove the sidebar's own pushState navigation so the parent (legacy root page or
dashboard layout) is the single owner of navigation, fixing the double-navigate
that fired when moving between a path route and a legacy ?page= route.

Cut API Reference over to its path route: add api_ref -> api-reference to
MIGRATED_PAGES and delete its arm from the legacy switch. Visiting
/ui/?page=api_ref redirects to /ui/api-reference, the sidebar links to and
highlights it, and navigating away returns to the legacy switch.

* fix(ui): address review on migrated-page routing

Keep the legacy hyphenated ?page=api-reference form working by mapping it to the
api-reference route alongside api_ref; the old switch matched both, so a bookmark
using the hyphen would otherwise fall through to the Usage default.

Add legacyKeyForPathname coverage: a migrated path (with and without trailing
slash) resolves to the api_ref sidebar key rather than the alias, a non-migrated
path returns null, and a non-root serverRootPath prefix is stripped before matching.

* fix(ui): populate serverRootPath from getUiConfig so migrated nav links keep the root path

getUiConfig updated proxyBaseUrl but never called updateServerRootPath, so the
module-level serverRootPath stayed at its "/" default. Under a custom
server_root_path the unified migratedHref/legacyPageHref builders then dropped
the prefix and the sidebar produced /ui/api-reference (404) instead of
/<root>/ui/api-reference. Adds the missing updateServerRootPath call plus a
regression test asserting getUiConfig sets serverRootPath and that migratedHref
carries the prefix

…the Tools page (BerriAI#29867)

* fix(ui): let non-creator users OAuth into OBO-mode MCP servers from the Tools page

* fix(ui): clear OBO Tools-tab one-shot on navigate-back and gate on credential-status errors

* feat(bedrock_mantle): add SigV4/IAM auth to Responses API route (fixes BerriAI#29665) (BerriAI#29788)

* feat(responses): add default no-op sign_request to BaseResponsesAPIConfig

* feat(responses): call sign_request after body is final, send signed bytes when signed

* feat(bedrock_mantle): add SigV4 sign_request via composed BaseAWSLLM (bearer path)

* test(bedrock_mantle): cover SigV4 access-key, AssumeRole, body bytes, region/auth consistency

* feat(bedrock_mantle): defer auth to sign_request; validate_environment no longer requires bearer

* docs(bedrock_mantle): document SigV4 + Bearer auth on Responses route

* test(responses): cover fake-stream signing order and mantle bearer arg/env precedence

* fix(bedrock_mantle): wrap all botocore credential errors with both-paths guidance

* fix(bedrock_mantle): catch specific credential errors, not all BotoCoreError, so STS transport failures are not masked

* fix(bedrock_mantle): sign the compact Responses route too, not just create

* fix(github-copilot): route per-model on /v1/responses based on model info (BerriAI#29747)

* feat(focus): add GCS destination for FOCUS export (BerriAI#29751)

* test: add failing tests for FocusGCSDestination

* feat: add FocusGCSDestination reusing GCSBucketBase auth

* feat: register FocusGCSDestination in factory; export from __init__

* fix(focus): preserve GCS_PATH_SERVICE_ACCOUNT when service_account_json not in config

* style: apply Black formatting to gcs_destination and tests

* style: apply Black formatting to factory.py

* fix(bedrock): omit empty additionalModelRequestFields and system from Converse API payload (BerriAI#29565)

Amazon Nova Pro (and other strict Bedrock models) return 400 Malformed input
request when additionalModelRequestFields: {} or system: [] are present in the
payload. Both fields are optional in CommonRequestObject (total=False) and must
be omitted rather than sent as empty structures.

Co-authored-by: shin-berri <[email protected]>
Co-authored-by: yuneng-jiang <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible in pass-through cost tracking (BerriAI#29730)

* fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible

Azure OpenAI resources created via the newer "Azure AI Foundry" /
Cognitive Services pathway live on `*.cognitiveservices.azure.com`
subdomains, not the older `openai.azure.com`. Both are valid Azure
OpenAI surfaces in production today.

The OpenAI pass-through cost-tracking handler hard-codes only the older
hostname in five places (four `is_openai_*_route` methods on
OpenAIPassthroughLoggingHandler, plus is_openai_route on
PassThroughEndpointLogging). As a result, calls from newer Azure
deployments are silently classified as "not an OpenAI route", the
dispatch into the cost-tracking handler is skipped, and tokens/cost
never get extracted into LiteLLM_SpendLogs — the row gets written with
prompt_tokens=0, completion_tokens=0, spend=0, model='unknown'.

Reproduced 2026-06-04 against a real Azure OpenAI deployment on
`*.cognitiveservices.azure.com` proxied through LiteLLM v1.88.0.

Fix: factor the hostname check into a single helper
`_is_openai_compatible_host` listing all three recognized surfaces
(api.openai.com, openai.azure.com, cognitiveservices.azure.com), and
have all five call sites delegate to it. Purely additive — never
weakens recognition for the originally-supported hostnames.

Adds a test
`test_is_openai_route_recognizes_cognitiveservices_azure_com` that
exercises all four `is_openai_*_route` static methods against
`*.cognitiveservices.azure.com` URLs (positive cases per route + a
small cross-route negative to confirm route-specific path matching
still works on the new hostname).

Out of scope for this PR (separate followup):
  - `openai_passthrough_handler` calls chat/completions
    `transform_response` on Responses API payloads (`output:` not
    `choices:`), which throws inside the dispatch and drops the
    SpendLogs row entirely. Recognized + tracked separately.

* ci: trigger fresh run

Empty commit to re-run checks. The previous auth-and-jwt failure was
a transient HuggingFace Hub 429 rate-limit hitting tokenizer downloads
in tests/proxy_unit_tests/test_custom_tokenizer_bug.py — unrelated to
this PR's scope (hostname recognition in pass-through cost tracking).
No code change.

---------

Co-authored-by: shin-berri <[email protected]>
Co-authored-by: yuneng-jiang <[email protected]>

* fix(responses): preserve forced-function tool_choice name in Responses to Chat transform (BerriAI#29812)

The Responses API forces a specific function with a top-level name
({"type": "function", "name": "X"}), but _transform_tool_choice only handled the
nested Chat Completions shape and fell through to returning "required" for the flat
form, silently dropping the function name and degrading a forced function call to
force-any-tool. Map the flat Responses shape to the nested Chat shape, keeping the
"required" fallback when no name is present.

* Preserve x-anthropic-billing-header system blocks for first-party Anthropic (BerriAI#29584)

* Preserve x-anthropic-billing-header system blocks for first-party Anthropic

PR BerriAI#20951 strips system blocks beginning with "x-anthropic-billing-header:" for
every Anthropic target. That block is how the first-party Anthropic API recognizes
Claude Code subscription (OAuth) traffic, so dropping it makes requests that carry
only that block, such as the auto-mode tool-safety classifier, fail with a
misleading 429 rate_limit_error; normal turns still work because they also carry
the "You are Claude Code" identity block.

Gate the strip behind should_strip_billing_metadata(), defaulting to False on the
first-party AnthropicConfig and AnthropicMessagesConfig so the block is kept, and
overridden to True on the providers that reach these transforms and reject the
block (Bedrock platform, Vertex, Azure for the chat path; Minimax, Azure, DeepSeek
for the messages path). Behavior for those providers is unchanged.

* Strip billing header on Bedrock invoke and Vertex messages pass-through

Two more subclasses reach the gated strip but inherited keep-by-default.
AmazonAnthropicClaudeConfig (Bedrock invoke) calls AnthropicConfig.transform_request,
which calls translate_system_message, and VertexAIPartnerModelsAnthropicMessagesConfig
(Vertex messages pass-through) calls super().transform_anthropic_messages_request.
Override should_strip_billing_metadata() to True on both.

Add a parametrized test asserting the flag for every first-party base (False) and
provider subclass (True), covering all overrides, plus a translate_system_message
regression test for the Bedrock invoke path.

* fix(cache): log hashed cache keys (BerriAI#29890)

* fix(ui): save routing groups as list (BerriAI#29889)

* Revert "fix(ui): save routing groups as list (BerriAI#29889)" (BerriAI#29928)

This reverts commit 9b1f78f.

* feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider (BerriAI#29842)

* feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider

Registers parasail in the openai_like JSON provider loader with both
/v1/chat/completions and /v1/responses support. Parasail's Responses API
rejects store:true and any request that omits store, so the loader gains a
force_store_false special_handling flag; the parasail entry sets it and
the generated Responses config overrides store=false on every call. This
keeps callers from hitting "State storage not supported" and matches what
Parasail's docs require.

Adds the PARASAIL enum value, listing under openai_compatible_providers,
provider documentation at docs/my-website/docs/providers/parasail.md, and
a focused unit test file under tests/test_litellm/llms/parasail/ that
covers JSON registration, chat URL construction, Responses URL
construction with PARASAIL_API_BASE override, and the force_store_false
regression in both the caller-sent-store=true and caller-omitted cases.

* fix(parasail): register in provider_endpoints_support, drop in-repo docs

Greptile review feedback. The provider doc belongs in the litellm-docs
repo, not this one's docs/my-website tree; removing it here. Adds the
parasail entry to provider_endpoints_support.json so the
check_provider_folders_documented.py CI check passes (chat_completions
and responses true; others false).

* fix: normalize Anthropic passthrough server tool usage (BerriAI#29827)

* test(anthropic): cover server_tool_use dict cost tracking

* fix: normalize Anthropic server tool usage

(cherry picked from commit 982f726)

* fix: keep server tool usage subscriptable

(cherry picked from commit 70280b9)

---------

Co-authored-by: Genmin <[email protected]>

* fix(proxy): fix typo generic_role_mappoings -> generic_role_mappings in ui_sso.py (BerriAI#29753)

Co-authored-by: shin-berri <[email protected]>
Co-authored-by: yuneng-jiang <[email protected]>

* feat(proxy): add disable_budget_reservation general setting (BerriAI#27639) (BerriAI#29493)

* feat(proxy): add disable_budget_reservation general setting (BerriAI#27639)

* feat(proxy): register disable_budget_reservation in ConfigGeneralSettings (BerriAI#27639)

* docs(proxy): document disable_budget_reservation concurrency tradeoff (BerriAI#27639)

* ci: re-trigger flaky docker build (prisma generate ECONNRESET)

* fix(proxy): warn and document budget enforcement tradeoff when disable_budget_reservation is set (BerriAI#27639)

* feat(gemini_tts): adding support to Gemini TTS languageCode parameters (BerriAI#29623)

* Adding support to Gemini TTS Language Code parameters

* Mapping Gemini TTS languageCode param in Docstring

* Use snake_case for language_code input keyMapping Gemini TTS languageCode param in Docstring

* Restoring files modified under enterprise/litellm_enterprise due to lint/formatting checks

---------

Co-authored-by: João Garrido <[email protected]>

* feat(guardrails): capture user and model metadata in CrowdStrike AIDR (BerriAI#29517)

* fix(proxy): require OpenAI path segment for shared Azure Cognitive Services domains

Address Greptile review: the `*.cognitiveservices.azure.com` /
`*.openai.azure.com` domains are shared by every Azure Cognitive Service
(Speech, Vision, Language, ...), so a hostname-only substring match
misclassified non-OpenAI Azure traffic as OpenAI routes.

- Replace the substring host test with suffix matching (rejects look-alike
  domains like cognitiveservices.azure.com.attacker.example).
- Add `_is_openai_compatible_url` that requires an OpenAI-style path marker
  (`/openai/` or `/v1/`) on the shared Azure domains, and use it in
  PassThroughEndpointLogging.is_openai_route (previously hostname-only).
- Add negative tests for Azure Speech/Vision paths and look-alike domains.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* fix: support Responses input in Redis semantic cache (BerriAI#29581)

* fix: support responses input in redis semantic cache

* test: cover redis semantic prompt extraction

* test: handle blank redis semantic text fallbacks

* chore: remove async cache dead statement

* test: cover redis semantic cache miss paths

* fix: filter sensitive cache lookup kwargs

* chore: rerun ci after huggingface rate limit

* chore(ui): regenerate dashboard API types (npm run gen:api)

Sync src/lib/http/schema.d.ts with the proxy OpenAPI spec: adds the
disable_budget_reservation general-settings field and picks up the
RateLimitError docstring reindent. Fixes the gen:api CI drift check.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* test(bedrock): assert empty additionalModelRequestFields is omitted

The Converse transformer now drops an empty additionalModelRequestFields
block instead of sending it as `{}`. Update test_bedrock_top_k_param so
models without top_k support (llama3) assert the key is absent rather than
equal to an empty dict.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

---------

Co-authored-by: Kent <[email protected]>
Co-authored-by: codgician <[email protected]>
Co-authored-by: Praveen Ghuge <[email protected]>
Co-authored-by: Roi <[email protected]>
Co-authored-by: shin-berri <[email protected]>
Co-authored-by: yuneng-jiang <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Liam Scott <[email protected]>
Co-authored-by: abhay23-AI <[email protected]>
Co-authored-by: Ceder Dens <[email protected]>
Co-authored-by: 冯基魁 <[email protected]>
Co-authored-by: Kai Huang <[email protected]>
Co-authored-by: rinto <[email protected]>
Co-authored-by: Genmin <[email protected]>
Co-authored-by: Arnav Bhilwariya <[email protected]>
Co-authored-by: Armaan Sandhu <[email protected]>
Co-authored-by: João Garrido <[email protected]>
Co-authored-by: João Garrido <[email protected]>
Co-authored-by: Kenan Yildirim <[email protected]>
Co-authored-by: Dávid Balatoni <[email protected]>

…#29908)

* feat(galileo): add health check support for UI callback test

Register galileo in /health/services so the proxy UI callback connection test works.

Co-authored-by: Cursor <[email protected]>

* feat(galileo): verify API key via /current_user health check

Call Galileo's current_user endpoint so the UI callback test validates credentials against the provider.

Co-authored-by: Cursor <[email protected]>

* chore(ui): regenerate schema.d.ts for galileo health service

Co-authored-by: Cursor <[email protected]>

* fix(galileo): return IntegrationHealthCheckStatus from async_health_check

Fixes mypy assignment error in health_services_endpoint where response was
narrowed to IntegrationHealthCheckStatus from earlier branches.

Co-authored-by: Cursor <[email protected]>

* Fix Galileo logging to match Langfuse across all endpoint types.

Stop skipping ingest when output is empty and log embeddings with a placeholder so embedding, speech, and other non-text responses are recorded like Langfuse.

Co-authored-by: Cursor <[email protected]>

* fix(galileo): remove unreachable health-check guard and None output sentinel

The use_v2_api flag is derived from bool(api_key), so the inner
GALILEO_API_KEY check inside the v2 branch could never run; collapse the
credential validation into the username/password path with a combined
message. _serialize_galileo_output now returns an empty string for None,
so _get_galileo_input_output_content always yields a str and the
post-call None coalescing guard is no longer needed.

* test(galileo): cover async_health_check failure paths and empty model response

Add regression tests for the Galileo health check unhealthy branches
(missing project id, missing base url, missing credentials, auth
failure, and request exception) and for logging a model response with
no choices, which now queues an empty output instead of being skipped.

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

…deleted (BerriAI#29875)

* fix(model-management): allow deleting a BYOK model after its team is deleted

A team BYOK model (model_info.team_id set) became undeletable once its team
was deleted: POST /model/delete ran can_user_make_model_call, which looked the
team up and raised 400 "Team id=... does not exist in db" before the delete
could run, so the model lingered on the Models + Endpoints page with no way to
remove it.

Drop the team-existence prerequisite from the delete path. When the model's
team still exists the normal auth check runs unchanged; when it is gone a proxy
admin may delete the orphan and any other caller gets a 403. The check is
fail-closed, so a missing or errored team lookup can only block the delete or
require an admin, never grant a non-admin access. Add/update/health keep their
team-existence validation.

* refactor(model-management): drop redundant team lookup on model delete

Move the orphaned-team handling into can_user_make_model_call behind an
allow_missing_team flag instead of pre-checking team existence in delete_model.
The endpoint no longer issues its own litellm_teamtable lookup, so deleting a
model whose team still exists hits the team table once instead of twice. The
auth behavior is unchanged: a proxy admin can delete a model whose team was
deleted, any other caller gets a 403, and add/update/health keep the strict
"team must exist" validation.

…erriAI#28913)

* fix(jwt-auth): defer to single-team DB fallback on claim mismatch

Extends the single-team DB fallback introduced in BerriAI#26418 to two more
cases where it previously could not run:

* `find_and_validate_specific_team_id`: when `team_id_jwt_field` is
  configured and a claim value is present in the token but the team
  does not exist in the LiteLLM DB (HTTPException 404 from
  `get_team_object`), return `(None, None)` instead of raising — the
  auth_builder fallback then attributes the request to the user's
  single DB team. Only HTTPException is caught; other errors (e.g.
  "No DB Connected") still propagate.

* `find_team_with_model_access`: when none of the `team_ids_jwt_field`
  groups resolve to a real LiteLLM team, return `(None, None)` instead
  of raising 403 so the same fallback path runs. If at least one group
  DID resolve to a team but none granted the requested model, the
  original 403 is preserved (legitimate access denial — not a claim
  mismatch). Tracked via the new `any_claim_team_resolved` flag.

The strict `is_required_team_id` raise and `enforce_team_based_model_access`
raise remain unchanged. Unit tests cover both new soft-fail paths and
guard each preserved path (strict required, enforce_team_based, the
preserved 403, and the non-HTTPException propagation).

Co-authored-by: Cursor <[email protected]>

* fix(jwt-auth): narrow HTTPException catch to 404 (greptile review)

Address Greptile review comments on BerriAI#28913:

* `find_and_validate_specific_team_id`: re-raise HTTPException when
  `status_code != 404`, pinning the catch to the "team doesn't exist
  in db" path documented for `get_team_object`. A future change that
  introduces a different status code (e.g. 403 for a blocked team)
  will now propagate instead of silently falling through to the
  single-team DB fallback.

* Add `test_find_and_validate_specific_team_id_non_404_http_exception_propagates`
  parametrised over 400 / 403 / 500 to lock in the contract.

Co-authored-by: Cursor <[email protected]>

* fix(jwt-auth): gate claim-mismatch fallback behind opt-in flag

The unresolved-team-claim fallback added in the previous commit
weakened the strict claim-based authorization contract by default —
an authenticated user whose JWT carries a stale or invalid team
claim could still consume their single DB team's models/quota via
the fallback.

Gate both soft-fail paths in `find_and_validate_specific_team_id`
and `find_team_with_model_access` behind a new opt-in flag
`team_claim_fallback` on `LiteLLM_JWTAuth` (default False).

Default-off preserves the pre-existing strict behavior. Operators
who intentionally treat IdP team claims as advisory (e.g. machine
tokens whose group claims live in a separate namespace from
LiteLLM team_ids) opt in via config.

Adds two regression tests guarding the default-off behavior.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>

BerriAI#29525)

On /team/update for a standalone (no-org) team, _check_user_team_limits()
compared the request max_budget against the caller's personal max_budget
whenever max_budget was present in the payload. A team admin whose personal
budget is lower than the team's budget could not edit any field (tpm_limit,
team name, etc.) because the UI re-sends the unchanged max_budget on every
update, tripping the personal-budget check.

Pass the team's current max_budget into _check_user_team_limits() and skip the
personal-budget comparison when the incoming value is unchanged or lower than
the team's current budget. Only genuine increases above the team's current
budget are still validated against the caller's personal limit, so no
over-relaxation. Proxy admins and the org-scoped path are unaffected.

Adds two regression tests for the standalone update path (unchanged budget +
tpm_limit change, and lowering the budget), both for a caller whose personal
budget is below the team budget.

Co-authored-by: Cursor <[email protected]>

…rriAI#29697)

glm-5p1 supports native tools on Fireworks; explicit false flags caused
drop_params to strip tools and tool_choice before the provider request.

Co-authored-by: Cursor <[email protected]>

…cks (BerriAI#29899)

* fix(vertex): propagate Vertex AI metadata in streaming success callbacks

Streaming calls assembled via stream_chunk_builder were missing
vertex_ai_grounding_metadata and vertex_ai_url_context_metadata in
standard_logging_object.response. Merge metadata from chunks into the
assembled response and mirror non-streaming hidden_params on Gemini chunks.

Co-authored-by: Cursor <[email protected]>

* refactor(vertex): move streaming metadata merge into provider config hook

Address review feedback by delegating assembled-stream metadata propagation
to VertexGeminiConfig via BaseConfig.apply_assembled_streaming_response_metadata,
and only write chunk hidden_params when metadata is non-empty.

Co-authored-by: Cursor <[email protected]>

* fix(redaction): scrub Vertex provider metadata when message logging is off

Clear vertex_ai_grounding_metadata and related fields from standard
logging responses and assembled streaming ModelResponse objects so
turn_off_message_logging cannot leak prompt-derived web search queries.

Co-authored-by: Cursor <[email protected]>

* Use assembled model for streaming metadata hook

* Fix Vertex metadata redaction bypass in logging callbacks.

Scrub Vertex provider fields from litellm_params.metadata.hidden_params during perform_redaction so streaming success_handler merges do not leak prompt-derived metadata when message logging is disabled.

Co-authored-by: Cursor <[email protected]>

* Fix Vertex streaming metadata from hidden params

* fix(vertex): mirror vertex_ai_safety_results on assembled streaming responses

The non-streaming transform_response stores safety data under
vertex_ai_safety_results, but the streaming path only wrote
vertex_ai_safety_ratings. Assembled streaming responses therefore never
carried vertex_ai_safety_results, so any consumer reading that field saw
a silent difference between streaming and non-streaming calls.

Set vertex_ai_safety_results alongside vertex_ai_safety_ratings in the
shared stream metadata setter and add it to the assembled metadata field
list so it propagates through stream_chunk_builder.

* fix(streaming): log provider streaming metadata hook failures instead of swallowing them

* refactor(vertex): share single Vertex metadata field tuple across redaction and streaming

* refactor(vertex): move Vertex metadata redaction helpers into llms/vertex_ai

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

Allow internal users to fetch their backend-scoped project list so the key creation project dropdown can populate for selected teams.

…29982)

Raise the PyJWT floor in pyproject (>=2.13.0,<3.0) and re-resolve uv.lock so
the proxy installs 2.13.0 instead of 2.12.0. Bump the ws transitive-version
override in the dashboard from 8.19.0 to 8.20.1 and regenerate package-lock;
jsdom and openai both dedupe onto the single 8.20.1 copy.

Both are routine dependency maintenance bumps to keep pinned versions current.

…leted (BerriAI#29977)

A team's BYOK models (rows in LiteLLM_ProxyModelTable with model_info.team_id set)
were left orphaned when the team was deleted; they lingered in the database and kept
showing on the Models + Endpoints page. delete_team now removes them via a new
delete_team_models helper that deletes the rows in one transaction and syncs the
in-memory router only after that transaction commits, run before the team rows are
deleted so a mid-flight failure never leaves the team gone with its models orphaned

…rriAI#28184)

* feat(vantage): include organization metadata in FOCUS Tags export

Join LiteLLM_OrganizationTable when building Vantage/FOCUS export rows so
organization_id and organization_alias appear in Tags for org-level filtering.

Co-authored-by: Cursor <[email protected]>

* test(focus): include api_requests in organization Tags tests

FocusTransformer now requires api_requests after staging merge; add the
column to test fixtures so integrations CI can run the Tags assertions.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>

…gs (BerriAI#29991)

Capture user_id and extra_info from metadata or litellm_metadata. The single-bag read dropped identity whenever a request carried a present litellm_metadata field (null or a user-supplied dict), since /chat/completions routes the authenticated identity into metadata while the guardrail read litellm_metadata first

…AI#29951)

…e streaming (BerriAI#30270)

* fix(google_genai): preserve complete SSE events in image streaming

Use iter_lines/aiter_lines instead of byte chunking so large inlineData
base64 payloads from Vertex/Gemini streamGenerateContent are not split
across events, which caused truncated JSON and SDK parse failures.

Co-authored-by: Cursor <[email protected]>

* fix(google_genai): buffer SSE lines until event delimiter

Assemble multi-field SSE events on blank-line boundaries instead of
terminating each field line individually.

Co-authored-by: Cursor <[email protected]>

* fix(tests): update google_ai_studio mocks from aiter_bytes to aiter_lines

Streaming iterator was changed to use iter_lines/aiter_lines instead of
iter_bytes/aiter_bytes. Update the two mocked streaming responses in
test_google_ai_studio.py to match.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>

…0274)

* fix(proxy): populate access_via_team_ids on /v1/model/info

Team metadata enrichment previously only ran on /v2/model/info with
include_team_models=true, leaving /v1/model/info without
access_via_team_ids for project model-picker flows.

Co-authored-by: Cursor <[email protected]>

* docs(dashboard): sync OpenAPI schema for /v1/model/info query params

Add include_team_models and teamId to the generated schema for /model/info
and /v1/model/info after the proxy endpoint gained team-access filtering.

Co-authored-by: Cursor <[email protected]>

* fix(proxy): always return direct_access on /v1/model/info

Set direct_access to true or false on every enriched model so clients
can filter without treating a missing field as ambiguous.

Co-authored-by: Cursor <[email protected]>

* perf(proxy): fail fast when teamId is set without a connected DB on /v1/model/info

Raise the db_not_connected error before building, enriching, and translating the model list instead of after, so a teamId query against a proxy with no database no longer wastes the full enrichment pipeline.

* fix(proxy): fail fast when include_team_models is set without a database

include_team_models=True relies on _populate_team_access_on_models to set
direct_access/access_via_team_ids, which only runs when a database is connected.
Without one, _filter_models_to_user_accessible discarded every model and the
endpoint returned an empty list with HTTP 200. Mirror the teamId guard so the
request fails fast with a clear db_not_connected error before any model-list work.

* fix(proxy): populate direct_access on single-model /model/info lookup

The /v1/model/info list path populates model_info.direct_access (and
access_via_team_ids) when a database is connected, but the
litellm_model_id single-model lookup returned early without it. This
made the two endpoints disagree, breaking the parity assertion in
test_get_specific_model. Run the same population on the single-model
path so both responses match.

* fix(proxy): apply no-DB fast-fail before litellm_model_id branch

The teamId/include_team_models no-DB guard sat after the litellm_model_id
early return, so ?litellm_model_id=X&teamId=Y with no DB returned 200 with
unpopulated access fields instead of the 500 raised on every other path.
Move the guard ahead of the branch so the fast-fail is uniform.

* fix(proxy): apply teamId/include_team_models filters on single-model lookup

The litellm_model_id early-return branch in model_info_v1 populated the
team access fields but returned before the teamId and include_team_models
filters ran, so a single-model lookup surfaced the deployment regardless
of team access when the DB was connected. Run both filters on the
single-model list before returning so the documented query params behave
the same with and without litellm_model_id.

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: mateo-berri <[email protected]>

* feat(bedrock): add bedrock mantle gemma 4 models (BerriAI#30264)

* feat(bedrock): add bedrock mantle gemma 4 models

* test(bedrock): harden mantle local cost fixture

* feat(responses): enable the responses API for the Tensormesh provider (BerriAI#30209)

* feat(responses): enable the responses API for the Tensormesh provider

* Update litellm/llms/openai_like/providers.json

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(langfuse_otel): mark LLM spans as generations (BerriAI#30250)

* fix(bedrock): stop stream_chunk_size leaking into invoke request bodies (BerriAI#30240)

stream_chunk_size is a LiteLLM-internal knob for re-chunking the HTTP
response stream. The invoke transformations splat optional_params into the
provider request body without dropping it, and Bedrock rejects unknown
fields, so any bedrock/invoke request that sets the parameter fails with
ValidationException: stream_chunk_size: Extra inputs are not permitted.
Drop it in the invoke dispatcher (covers cohere, titan, mistral, meta,
ai21) and in the Claude messages-format request builder (the route used
for bedrock/invoke Anthropic models)

* fix(bedrock): stop buffering streamed tool-call argument deltas (BerriAI#30231)

* fix(bedrock): stop buffering streamed tool-call argument deltas

Two issues made Bedrock tool-use streaming arrive as a single end-of-stream
burst through LiteLLM while plain text streamed fine.

First, the anthropic-beta allowlist mapped fine-grained-tool-streaming-2025-05-14
to null for bedrock and bedrock_converse, so the header was silently stripped.
Without that beta, Anthropic models on Bedrock buffer tool input server-side and
emit all toolUse.input deltas at once (verified against converse-stream and
invoke-with-response-stream directly). Bedrock accepts the beta via
additionalModelRequestFields.anthropic_beta, so it is now forwarded.

Second, the streaming reads re-chunked the AWS event stream with
iter_bytes(chunk_size=1024). httpx's ByteChunker only releases full 1024-byte
blocks, so the small early events (messageStart, contentBlockStart, first
deltas) sat in the buffer until enough bytes accumulated, pushing
time-to-first-byte from ~1.4s to ~8.5s on buffered tool-use streams. The
default is now no re-chunking; an explicit stream_chunk_size is still honored.

* test(bedrock): cover explicit stream_chunk_size on sync invoke path

* test(bedrock): cover stream_chunk_size plumbing through converse completion

* test(bedrock): cover stream_chunk_size default in legacy BedrockLLM streaming

* test(bedrock): merge converse handler tests into existing mapped test file

pytest imports test modules by basename in non-package test dirs, so the new
tests/test_litellm/llms/bedrock/chat/test_converse_handler.py collided with
the pre-existing tests/test_litellm/llms/chat/test_converse_handler.py and
broke collection in CI. Move the new tests into the existing file

* feat(otel): emit v2 cost breakdown + stamp tracer scope version (BerriAI#30156)

Read the StandardLoggingPayload cost_breakdown into a typed LLMCost on
LLMCallSpanData and emit each component under litellm.cost.* (absent
components omitted, so spans stay sparse). Stamp litellm.__version__ as
the instrumentation scope version so every v2 span carries a
deterministic scope.version.

Tests under tests/test_litellm/integrations/otel/.

* fix(proxy): cancel in-flight upstream LLM request on client disconnect (opt-in) (BerriAI#30223)

* fix(proxy): cancel in-flight upstream LLM request on client disconnect (opt-in)

On the non-streaming path, base_process_llm_request awaited the LLM call
with no disconnect monitoring; when the HTTP client went away the
upstream request kept running until completion or request_timeout (6000s
default), holding a backend slot (e.g. a vLLM GPU slot) for output
nobody would read

Add an opt-in general_settings.cancel_on_disconnect flag, default off,
so the default code path is unchanged. When enabled, a receive-based
watcher task observes http.disconnect and cancels the asyncio.gather
driving the upstream call. The resulting CancelledError is converted to
HTTPException 499 only when the disconnect event is set, so
server-initiated cancellations still propagate as-is. The 499 then flows
through _handle_llm_api_exception like any other failure, meaning
post_call_failure_hook still releases max_parallel_requests slots and
fires spend and alerting callbacks; it is logged at info level instead
of a full traceback

Also removes the dead check_request_disconnection helper in
proxy_server.py (zero call sites) along with its behavior-pin tests

Builds on the receive-based design from BerriAI#25776

Addresses BerriAI#13774. Re-fixes BerriAI#22805 (regressed after the BerriAI#14295 revert)

Co-authored-by: CreateRandom <[email protected]>

* fix(proxy): scope 499 quiet logging to disconnects and harden watcher

Address the two P2 findings from the Greptile review on BerriAI#30223. The
info-level logging in _log_llm_api_exception now applies only to the
disconnect-specific HTTPException (status 499 plus the shared
_CLIENT_DISCONNECT_DETAIL message), so any other 499 raised by hooks or
guardrails keeps its full traceback. The disconnect watcher now catches
exceptions from request.receive() (e.g. a transport reset) and logs a
warning instead of dying silently, making the degradation to no-op
visible; a test pins that the LLM call is not cancelled in that case

---------

Co-authored-by: kursad <[email protected]>
Co-authored-by: CreateRandom <[email protected]>

* fix(bedrock): grant aws-external-anthropic:* in OIDC session policy for claude_platform (BerriAI#30200) (BerriAI#30205)

The inline STS session policy passed to assume_role_with_web_identity
acts as an IAM PERMISSION CEILING — effective permissions are the
intersection of the role's identity policies and this policy. Any
action not listed is silently denied even when the IAM role grants it.

BerriAI#27678 added the bedrock/claude_platform/<model> route but its
service-side action namespace is aws-external-anthropic:*, not
bedrock:*. Without a matching statement here, every claude_platform
request via OIDC (GCP federation, EKS Pod Identity webhook, etc.) 403s
with 'no session policy allows the aws-external-anthropic:CreateInference
action' — even with a fully permissive identity policy.

Add a second ClaudePlatformLiteLLM statement covering CreateInference,
CreateBatchInference, CancelBatchInference, DeleteBatchInference,
CountTokens, Get*, List*. Keep aws:SecureTransport=true parity with the
bedrock statement.

Static creds + IRSA flow through different code paths and are not
affected.

Fixes BerriAI#30200

* fix(proxy): set Retry-After header on RouterRateLimitError 429 responses (BerriAI#30098)

* Set Retry-After header on RouterRateLimitError responses

When all deployments for a model are in cooldown, the proxy returns a
429 whose cooldown timing is only available by parsing the error
message string. RouterRateLimitError already carries cooldown_time, so
expose it as a standard retry-after header in
_handle_llm_api_exception. The value is rounded up so clients never
retry before the cooldown window ends.

Fixes BerriAI#27823.

* Set Retry-After after response-headers hook so cooldown wins

The cooldown-derived retry-after was assigned before the
post_call_response_headers_hook merge, so a callback returning a
retry-after key (including a stale or empty value) silently clobbered
it. Move the RouterRateLimitError block after the callback merge so the
cooldown value is authoritative for this error type.

* fix(router): route aspeech through async_function_with_fallbacks (BerriAI#30104)

* fix(router): route aspeech through async_function_with_fallbacks

Router.aspeech selected a deployment and awaited litellm.aspeech
directly, so TTS requests got no retry on failure and no failover to
backup deployments; the except block only fired an exception alert and
re-raised. Every other router endpoint (acompletion, aembedding,
atranscription, arerank) already delegates to
async_function_with_fallbacks

Mirror the atranscription pattern: move deployment selection and the
litellm.aspeech call into a private _aspeech method, then have the
public aspeech set kwargs["original_function"] = self._aspeech and
await self.async_function_with_fallbacks(**kwargs). _aspeech also picks
up the shared _get_async_openai_model_client helper and the same
total/success/fail call accounting the sibling endpoints use

Fixes BerriAI#27778.

* fix(router): apply deployment kwargs and rpm semaphore in _aspeech

Bring _aspeech fully in line with _atranscription: call
_update_kwargs_with_deployment so deployment metadata, model_info,
timeout, and default litellm params flow into the request, and wrap
the litellm.aspeech call with the max_parallel_requests semaphore plus
async_routing_strategy_pre_call_checks so TTS respects rpm limits the
same way the other router endpoints do

Also add a unit test that exercises _aspeech directly and asserts the
deployment metadata reaches the underlying call

* fix(slack_alerting): stop false-positive hanging request alerts for requests below the alerting threshold (BerriAI#30106)

* fix(slack_alerting): skip hanging request alerts below the threshold

The hanging request check alerted on any cached request whose
completion status was not yet recorded, with no minimum age check.
Since the background loop runs every alerting_threshold / 2 seconds,
any request that happened to be in flight at a check fired a
"hanging - Ns+ request time" alert even if it was only seconds old,
producing a steady stream of false positives.

Add a created_at timestamp to HangingRequestData, stamped when the
request enters the hanging request cache, and skip requests younger
than alerting_threshold without evicting them, so a later check can
still alert if they never complete. Extend the cache TTL from
threshold + 60s to 1.5x threshold + 60s; with the age check, entries
only become alertable after threshold seconds, and the check period
is threshold / 2, so the old TTL could evict a genuinely hanging
request before any check saw it cross the threshold.

Fixes BerriAI#27855.

* fix(slack_alerting): alert once per hanging request

The min-age gate stops false positives for young in-flight requests, but
a genuinely hanging request still re-alerted on every checker tick within
the cache TTL. With the wider TTL (1.5x threshold + 60s) that is 1-2 extra
Slack notifications per stuck request at the default 600s threshold.

Flag a HangingRequestData entry as alerted once its alert fires and skip
flagged entries on later ticks, so each hang produces exactly one alert.
The cache reference is mutated in place, so the TTL is untouched and still
handles cleanup. Adds a regression test asserting one alert across multiple
ticks.

Fixes BerriAI#27855.

* fix(health): treat all-proxy-models keys as unrestricted in /health (BerriAI#30087)

* fix(health): treat all-proxy-models keys as unrestricted in /health

A key granted all model permissions stores the literal
"all-proxy-models" marker in its models list. The /health access
filter compared that marker against real model_names, so the model
list filtered down to nothing and the WebUI health check returned
healthy_count=0, unhealthy_count=0 with HTTP 503. Skip the filter
(both the live path and the background-cache model_id scoping) when
the marker is present, matching how auth_checks treats
SpecialModelNames.all_proxy_models.

Fixes BerriAI#29744.

* fix(health): resolve all-team-models sentinel to the team allowlist

Same failure shape as the all-proxy-models case: a key carrying the
literal "all-team-models" entry matches no real model_name, so the
/health access filter would zero out the model list. Resolve the
sentinel to the key's team models when team_id is set, matching
get_key_models in model_checks.py. Without a team_id the sentinel
stays unresolved and matches nothing, denying rather than widening
access, mirroring _resolve_key_models_for_auth_check.

* feat(proxy): auto-enable drop_params for Claude Code requests (BerriAI#30218)

* feat(proxy): auto-enable drop_params for Claude Code requests

Claude Code identifies itself with a claude-cli/<version> user agent and
sends Anthropic-specific params (top_k, thinking, etc.) on every request.
When the proxy routes those requests to a non-Anthropic provider, the
unsupported params fail the call unless drop_params is configured. Detect
the Claude Code user agent in add_litellm_data_to_request and default
drop_params to true for those requests, without overriding an explicit
drop_params value sent by the caller.

* feat(proxy): respect operator litellm_settings drop_params over Claude Code default

An explicit drop_params in the operator's litellm_settings (true or false)
now suppresses the Claude Code user agent default, so an operator who
deliberately configured drop_params: false keeps strict param validation
for Claude Code clients too. The auto-default only fills the gap when
neither the request body nor the config sets a value.

* fix(snowflake): migrate to native endpoints with auto-routing for Claude models (BerriAI#29964)

* fix(snowflake): migrate to native Cortex REST API endpoints

Replaces the legacy /api/v2/cortex/inference:complete endpoint with the
native OpenAI-compatible /api/v2/cortex/v1/chat/completions endpoint,
fixing error 390142 (Incoming request does not contain a valid payload)
when using model: snowflake/<model> in LiteLLM proxy.

Changes:
- litellm/llms/snowflake/chat/transformation.py: route to native
  /cortex/v1/chat/completions, remove Snowflake-specific tool_spec
  payload transformation, remove content_list response handling,
  add stream to supported params
- litellm/llms/snowflake/anthropic/transformation.py (new):
  SnowflakeCortexAnthropicConfig routes Claude models to /cortex/v1/messages
  with anthropic-version header and Anthropic->OpenAI response transform
- tests: 29 unit tests covering URL routing, auth headers, payload
  format, and response parsing

* fix(snowflake): map max_tokens to max_completion_tokens for native endpoint

* fix: handle multi-turn tool conversations and OpenAI→Anthropic tool format conversion

- _extract_system_and_messages now preserves tool_calls from assistant messages
  and converts them to Anthropic tool_use content blocks
- tool role messages are converted to user role with tool_result content blocks
  (as required by Anthropic Messages API)
- Added _transform_tools_to_anthropic() to convert OpenAI tool format
  (type/function/parameters) to Anthropic format (name/input_schema)
- Added comprehensive tests for multi-turn tool conversations

Addresses review feedback on PR BerriAI#29964

* test: add coverage for malformed JSON and non-string tool arguments

* fix(tests): update chat transformation tests for native OpenAI-compatible endpoint

* style: apply black formatting

* fix: resolve mypy type errors in anthropic transformation

* fix: correct mypy type: ignore error codes (attr-defined)

* fix: use max_tokens instead of max_completion_tokens for Snowflake endpoint compatibility

* refactor: merge Anthropic config into unified SnowflakeConfig with auto-routing

- Remove separate SnowflakeCortexAnthropicConfig and anthropic/ directory
- SnowflakeConfig now auto-routes based on model name:
  - Claude models → /messages endpoint (Anthropic format)
  - All others → /chat/completions endpoint (OpenAI format)
- No new provider needed (stays as SNOWFLAKE = 'snowflake')
- Tool message transformation for Claude: tool_calls → tool_use blocks,
  tool role → user with tool_result
- OpenAI → Anthropic tool format conversion (parameters → input_schema)
- Addresses Greptile feedback about unwired SnowflakeCortexAnthropicConfig

* fix: use max_completion_tokens for /chat/completions (Snowflake deprecated max_tokens on this endpoint)

* fix(tests): update assertions for Claude auto-routing to /messages endpoint

* fix(snowflake): add tool_choice conversion and preserve max_completion_tokens in Anthropic path

* fix(snowflake): use ChatCompletionMessageToolCall objects and strip model prefix on OpenAI path

* fix(snowflake): collect multiple system messages to prevent guardrail override

* chore: remove committed .pyc files and add __pycache__ to .gitignore

* fix: remove unused Union import

* fix: restore original .gitignore (accidentally replaced in earlier commit)

* feat(snowflake): add streaming response handler for both Anthropic and OpenAI SSE formats

* fix: remove unused AsyncIterator and Iterator imports

* fix: add missing total_tokens to ChatCompletionUsageBlock

* fix(snowflake): coalesce consecutive tool results into single user message for Anthropic

* fix(snowflake): handle message_start event for streaming input_tokens tracking

* fix: evict last deleted model in multi-instance deployments (BerriAI#28608)

* fix: evict last deleted model in multi-instance deployments

_delete_deployment had an early return when db_models was empty,
preventing eviction of the last deleted model during reconciliation.

- Remove len(db_models)==0 early return from _delete_deployment
- Return None (not []) from _get_models_from_db on DB failure so
  callers can distinguish a transient failure from a genuinely empty DB
- Guard _update_llm_router against None to skip updates on DB failure

Fixes BerriAI#28443

* test: remove dead MagicMock assignment in type_mismatch test

* fix: update test to pass [] not None to _update_llm_router

test_ProxyConfig__update_llm_router_bad_proxy_logging_raises was passing
None as new_models to get through to the proxy_logging_obj check, but
the None guard we added now returns early before reaching that path.
Pass [] instead so the test exercises the intended AttributeError case.

Signed-off-by: Rudra Dudhat <[email protected]>

* chore: regenerate API types to sync schema.d.ts with proxy OpenAPI spec

Signed-off-by: Rudra Dudhat <[email protected]>

---------

Signed-off-by: Rudra Dudhat <[email protected]>

* fix: invalidate Redis spend counter on /key/reset_spend (BerriAI#29694)

* fix: set Redis spend counter to reset_to value on /key/reset_spend

Previously, the Redis spend counter was always set to 0.0 after a reset,
even when reset_to was a non-zero value (partial reset). This caused
the budget to be under-enforced for up to 60 seconds until the counter
expired and fell through to the DB.

Now the counter is set to the actual reset_to value, so partial resets
are reflected correctly and budget enforcement is consistent.

* test: update reset_key_spend test to match direct cache set

The implementation now sets spend_counter_cache directly instead of
calling _invalidate_spend_counter. Update the test to verify the
in_memory_cache.set_cache call with the correct key, value, and ttl.

---------

Co-authored-by: michaelxer <[email protected]>

* fix: add scaleway models pricing (BerriAI#27659)

* fix: Add embeddings support for Scaleway provider

* fix: resolve merge conflicts

* fix(main): clarify backend route handling for Swagger static assets (BerriAI#30196)

* fix(main): clarify backend route handling for Swagger static assets

* fix(allowlist): add BACKEND_MOUNT_PATHS for Swagger static assets

* fix(voyage): route multimodal embeddings to correct endpoint (BerriAI#30193)

* fix(voyage): route multimodal embeddings to correct endpoint

* test(voyage): cover multimodal embedding edge cases

* test(voyage): cover api key fallback

* fix(voyage): raise early on missing api key and malformed image url

* test(voyage): cover utils routing and helper

* fix(voyage): route supported openai params for multimodal models

* style: apply black formatting

* fix(ui): infer Azure API version from API base (BerriAI#30204)

* fix(ui): infer Azure API version from API base

* fix(ui): address Azure API version feedback

* Update litellm/llms/snowflake/chat/transformation.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* feat(datadog): add team-scoped Datadog callback support (BerriAI#29947)

Enable teams to configure their own Datadog credentials via
POST /team/{team_id}/callback, following the same pattern as Langfuse.

* Merge pull request BerriAI#29528 from aanchal22/litellm_byok-alias-merge

fix(proxy): atomic merge for team model aliases and team.models on BYOK create

* feat: add EmpirioLabs as an OpenAI-compatible provider (BerriAI#30278)

Co-authored-by: Adam Dalloul <[email protected]>

* fix: resolve failing tests and lint in snowflake/team endpoints

- Black-format snowflake/chat/transformation.py to fix lint failure
- Update Anthropic config test to expect default max_tokens of 4096 (matches implementation)
- Add AsyncMock + execute_raw mock to team_model_add cache-refresh pin test
- Add model_dump mock and patch cache/logging in test_uses_atomic_array_append_with_dedup

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(test): update test_db_error_new_model_check for new _delete_deployment logic

_delete_deployment no longer short-circuits on empty db_models — it now
treats [] as a valid empty-DB state and proceeds to check config models.
Mock get_config to return the two router deployments so they appear in
combined_id_list and are protected, which matches the real-world scenario
where a DB error occurs but the models are config-backed.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* feat(proxy): register cancel_on_disconnect in ConfigGeneralSettings and config list (BerriAI#30295)

* feat(proxy): register cancel_on_disconnect in ConfigGeneralSettings and config list

Follow-up to BerriAI#30223 per maintainer review: documents the flag in
ConfigGeneralSettings with a short description and adds it to
allowed_args in get_config_list so the UI and /config/list expose it.
A test pins that /config/list returns the field with type Boolean,
which requires both registrations to be present

* chore(ui): regenerate schema.d.ts for cancel_on_disconnect

---------

Co-authored-by: kursad <[email protected]>

* fix(datadog): never fall back to env DD_API_KEY for caller-supplied destinations

Team/key-scoped Datadog loggers could be pointed at an arbitrary dd_agent_host or
dd_site while omitting dd_api_key, causing the proxy's global DD_API_KEY to be sent
as the DD-API-KEY header to that destination. Gate the env-var fallback behind an
allow_env_credentials flag, set to False when the destination is caller-supplied,
mirroring the existing langfuse/langsmith pattern.

---------

Signed-off-by: Rudra Dudhat <[email protected]>
Co-authored-by: Emerson Gomes <[email protected]>
Co-authored-by: daitran-tensormesh <[email protected]>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Muspi Merol <[email protected]>
Co-authored-by: fangkang <[email protected]>
Co-authored-by: Chris Hoogeboom <[email protected]>
Co-authored-by: kursadlacin <[email protected]>
Co-authored-by: kursad <[email protected]>
Co-authored-by: CreateRandom <[email protected]>
Co-authored-by: hcl <[email protected]>
Co-authored-by: Filippo Menghi <[email protected]>
Co-authored-by: Mateo Wang <[email protected]>
Co-authored-by: sfc-gh-nashukla <[email protected]>
Co-authored-by: Rudra Dudhat <[email protected]>
Co-authored-by: Michael <[email protected]>
Co-authored-by: michaelxer <[email protected]>
Co-authored-by: Quentin Champenois <[email protected]>
Co-authored-by: mauriceberentsen <[email protected]>
Co-authored-by: lost9999 <[email protected]>
Co-authored-by: GaetanVDB07 <[email protected]>
Co-authored-by: Aanchal Khandelwal <[email protected]>
Co-authored-by: Adam Dalloul <[email protected]>
Co-authored-by: Adam Dalloul <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>

…kills to path routes (BerriAI#30263)

* feat(ui): cut policies, guardrails, prompts, tool-policies, and skills over to path routes

Continues the page-by-page App Router migration. All five legacy switch
arms passed only accessToken/userRole, so each route wrapper is a thin
useAuthorized() + render. skills keeps a claude-code-plugins alias in
MIGRATED_PAGES because the old switch matched both page ids, mirroring
the api_ref/api-reference precedent.

* refactor(ui): colocate the prompts panel under its route

The new route wrapper was its only importer, so the 32-file folder
moves wholesale into (dashboard)/prompts/components; tree-escaping
relative imports (networking, molecules, common_components) become
@/components aliases and the suppressions baseline is re-keyed.
policies, guardrails, claude_code_plugins, and ToolPoliciesView stay
at src/components: each has consumers on other pages (playground
selectors, AI Hub, public model hub), so their shared/page splits go
in the colocation follow-up.

* fix(ui): move the PromptsPanel file along with its folder

@/components/prompts resolved to the prompts.tsx FILE next to the
prompts/ folder, not the folder itself; the colocation moved only the
folder, so the wrapper's ./components import and the panel's
./prompts/* imports both broke and next build failed. Move the panel
in as components/index.tsx and fix its now-escaping relative imports.
Caught by next build; tsc --noEmit missed it because incremental mode
reused a stale tsbuildinfo.

* test(ui): lock skills alias resolution in legacyKeyForPathname

Both skills and claude-code-plugins map to the skills segment, and
sidebar highlighting depends on first-match-wins returning the sidebar
key; assert it so a future reorder of MIGRATED_PAGES cannot silently
break highlighting. Mirrors the api_ref/api-reference assertion.
Flagged by Greptile.

…, and logs to path routes (BerriAI#30267)

* feat(ui): cut caching, cost-tracking, transform-request, ui-theme, and logs over to path routes

Completes the simple-leaf portion of the page-by-page App Router
migration. All five legacy switch arms passed only identity props
(accessToken/userRole/userID, plus token/premiumUser for caching and
logs), all of which useAuthorized() provides, so each route wrapper is
a thin useAuthorized() + render. MIGRATED_PAGES routes the sidebar and
redirects the legacy ?page= URLs; the e2e fixture picks all five up in
the migration smoke and sidebar specs automatically.

* refactor(ui): colocate caching, cost-tracking, transform-request, and ui-theme components

Each had the legacy switch as its only importer. caching takes its
whole closure (cache_dashboard, cache_health, cache_settings,
response_time_indicator); CostTrackingSettings moves as the
cost-tracking components folder; the transform-request and ui-theme
single-file panels move under their routes. view_logs stays at
src/components: six other pages (guardrails monitor, tool policies,
pass-through, MCP toolsets, usage) import it. Suppressions re-keyed.

* chore: retrigger ci

e2e_ui_testing failed on three specs unrelated to this PR's pages
(team-info tabs, MCP create form) and local_testing_part1 on
test_batch_completions; all pass on the pre-merge commit and none
touch files in this diff.

…nder SERVER_ROOT_PATH (BerriAI#30312)

* fix(ui): gate dashboard layout on ui config load so deep links work under SERVER_ROOT_PATH

* test(ui): create ui config deferred per test so the pending state stays repeatable

…sage over to path routes (BerriAI#30268)

admin-panel pulls proxySettings from the shared useProxySettings query
hook (dropping the last reader of the legacy page's copy), the model
hub wrapper keeps the admin-vs-public branch as an early return, and
the usage wrapper feeds NewUsagePage from the useTeams and
useOrganizations query hooks instead of the lifted switch state.
new_usage maps to the /usage segment while the old ?page=usage report
keeps its legacy arm, asserted in the unit test so the two cannot be
confused.

BerriAI#30257)

* fix(otel): cap metric attribute cardinality with include/exclude lists

OTEL metrics stamped every per-request hidden_params and metadata.* field
onto each gen_ai.client.* sample, so near-unique values created one metric
time series per request and backends like Splunk Observability Cloud throttled
and dropped the data.

Add an attributes block under callback_settings.otel with mutually-exclusive
include_list (allowlist) and exclude_list (denylist), validated against the
known attribute names at startup and applied once to the metric attributes in
_record_metrics. Spans are untouched, and with no config every attribute is
still emitted so existing setups are unaffected.

Resolves LIT-3600

* fix(otel): resolve metric attribute filter from callback_settings

The proxy usually constructs the OpenTelemetry logger without forwarding the
attributes kwarg, while the filter lives under
litellm.callback_settings["otel"]["attributes"]. __init__ only read the kwarg,
so the recording instance kept config.attributes=None and shipped metrics at
full cardinality even when the filter was configured; a live proxy run exposed
this. Fall back to the global at init for the base otel logger, and add a
regression test that drives the real success hook through the callback_settings
path (the unit tests passed before because they injected the config directly).

* fix(otel): reject gen_ai.token.type from metric attribute filter lists

gen_ai.token.type was a member of VALID_METRIC_ATTRIBUTE_NAMES, so an
operator could list it in include_list or exclude_list and pass startup
validation. The attribute is injected into the input/output token series
after _filter_metric_attributes runs, so the filter never sees it and the
request silently has no effect.

Reject it loudly from either list instead, matching the contract that a
non-actionable attribute name fails fast rather than falling through to a
no-op. It stays a structural discriminator on the token-usage histogram.

* fix(otel): resolve metric attribute filter lazily at record time

The proxy constructs the OpenTelemetry logger before it populates
litellm.callback_settings["otel"]["attributes"], so resolving the filter at
__init__ left config.attributes None and shipped metrics at full cardinality. A
live proxy run confirmed the leak. Resolve the filter on the first metric record
instead, when callback_settings is populated, while still validating an explicit
config eagerly so a bad SDK config fails at startup. The regression test now
constructs the logger before populating callback_settings to mirror that
ordering, so it fails if the filter is resolved too early.

* fix(otel): don't cache invalid filter on lazy callback_settings path

On the lazy callback_settings resolution path, _ensure_metric_attribute_filter
wrote self.config.attributes before validating it. When validation then failed,
_metric_attr_filter_resolved stayed False while config.attributes held the bad
filter, so the next record skipped the callback_settings re-read and re-raised
the stale error indefinitely; fixing the misconfiguration required a restart.

Drop the premature write and resolve from the local value. A subsequent record
now re-reads callback_settings, so a corrected config takes effect without a
restart. The write was dead on the success path anyway, since the resolved
frozensets are what the filter reads.

…combined view (BerriAI#30327)

The grace-period branch assigned the recursive get_data result (a
finished LiteLLM_VerificationTokenView) back into the variable that the
combined-view dict normalization then subscripts, raising TypeError on
every request made with a rotated key inside its grace window; auth
surfaced that as a 401. Return the recursive result directly instead.

Regression test drives the full get_data flow: old hash misses the view,
deprecated table resolves to the active token, and the call must return
the view object

…#30220)

* chore(deps): bump aiohttp to 3.14.1 and vitest to 3.2.6

Lockfile-only bump for aiohttp (3.13.5 -> 3.14.1, within the existing
pyproject constraint) and dashboard devDependency bumps for vitest,
@vitest/coverage-v8, @vitest/ui (3.2.4 -> 3.2.6) plus transitive
brace-expansion (5.0.5 -> 5.0.6). Clears the currently published
advisories flagged by osv.dev against uv.lock and the dashboard
lockfile. Verified: 154 custom_httpx unit tests and all 3943 dashboard
vitest tests pass; live proxy completion and streaming calls succeed on
the bumped venv

* chore(deps): raise aiohttp floor to 3.14.0

The lockfile bump alone only protects environments built from uv.lock.
Raising the pyproject floor extends the same minimum to package
consumers installing litellm from PyPI, and prevents a future lockfile
regeneration from resolving below 3.14.0

* Revert "chore(deps): raise aiohttp floor to 3.14.0"

This reverts commit d6c1c9d.

* revert(deps): roll back aiohttp to 3.13.5

vcrpy is incompatible with aiohttp >= 3.14 (the aiohttp_stubs module
imports a symbol removed in 3.14) and the upstream fix is merged but
unreleased, so every cassette-based test suite fails on 3.14. Hold
aiohttp at 3.13.5 until a vcrpy release ships; the vitest and
brace-expansion bumps stay

* chore(deps): bump pypdf to 6.13.1 and tornado to 6.5.7

Lockfile-only bumps clearing the advisories published for both since
this branch was opened

* chore(deps): add regression guards for the bumped versions

Raise the pypdf floor to 6.12.0 (direct dependency, applies to package
consumers too) and add uv constraint-dependencies for the transitive
pins: tornado >= 6.5.6, and aiohttp held in [3.13.5, 3.14) so a lockfile
regeneration can neither fall back below the current version nor move
onto 3.14 while vcrpy is incompatible. Constraints live in [tool.uv]
and only affect this repo's resolution, not published metadata.
Verified: uv lock -P with each out-of-range version fails to resolve;
in-range resolutions unchanged (pypdf 6.13.1, tornado 6.5.7,
aiohttp 3.13.5)

The /ui/chat route is not linked from anywhere: no sidebar entry, no
redirect, and no backend reference. It is only reachable by typing the
URL by hand. Delete the route (src/app/chat) and its components
(src/components/chat), which nothing else imports, and drop the deleted
files' entries from the eslint suppressions baseline.

…30323)

* feat(ui): cut agents and router-settings over to path routes

Both pages depended on a slice of the legacy shell's lifted state, now
replaced with React Query hooks in their route wrappers: agents pulls
teams from useTeams, and router-settings feeds the Fallbacks model
dropdown from useAllProxyModels. The shell's modelData copy only
populated after visiting the Models page in the same session, so the
dropdown was empty on a fresh load of router-settings; the hook fixes
that as a side effect of the cutover.

* refactor(ui): delete the dead modelData prop chain

AddFallbacks fetches its own model list when its modal opens and never
reads the models prop, so the whole shell modelData -> GeneralSettings
-> Fallbacks -> AddFallbacks chain fed a prop nobody consumed;
RouterSettings declared it without using it at all. Remove the chain
and the router-settings wrapper's useAllProxyModels adaptation that
was feeding it. Also corrects this PR's earlier claim: the Fallbacks
dropdown was never broken by the empty shell state, because the
component self-fetches.

* feat: strengthen coding conventions in CLAUDE.md

* fix: make hand-roll rule clearer

…I#30334)

ViewUserDashboard renders from app/(dashboard)/users/page.tsx via useAuthorized
and useTeams instead of the legacy ?page=users switch arm. The keys and setKeys
props were declared but never destructured by the component, so they are removed
from the interface rather than wired into the wrapper.

* feat: add ruff strict-rule suppressions baseline gate

Introduce a stricter ruff rule set (typed params, no Any, complexity and
arg-count caps, mutable-default and global-rebinding checks) grandfathered
against the current tree and enforced as a budget rather than zero-tolerance

ruff-strict.toml defines the 9 rules separately from ruff.toml so the existing
ruff check stays green. scripts/ruff_suppressions.py builds the per-file,
per-rule baseline in ruff-suppressions.json and gates CI by failing when the
total grows past the baseline plus a 0.5% slack margin. The baseline ratchets
down via `make lint-suppressions-update` after fixes

* fix: surface per-file drift as a warning on a passing suppressions check

Greptile flagged that cmd_check computed per-file regressions but only printed
them on failure, so violations shifted between files (or a brand-new file under
the slack) passed with a silent OK. Print them as a non-fatal warning on the
pass path too; pass/fail behavior is unchanged

* refactor: gate strict ruff rules on the delta vs base, not a frozen baseline

The committed total-count baseline went stale against a moving base. CI lints the
PR merged with the current staging tip, so violations merged by other PRs counted
against this PR and tripped the budget even though nothing here touched them

Replace it with a drift-proof gate. scripts/ruff_strict_gate.py runs ruff on the
head, keeps only violations on lines this change adds relative to the merge-base,
and fails when a rule exceeds its per-rule allowance in ruff-strict-budget.json
(all 0 today). Because the base is measured live, base drift cancels out and only
what the change introduces is gated. Drops ruff-suppressions.json and the old
suppressions script

* chore: allow 5 new ANN001/ANN003/ANN401 per change

Give the three annotation-completeness rules a small per-change allowance so a
large new module is not blocked over a few untyped params or kwargs, while the
correctness and structural rules (B006, C901, PLR0913, PLW0603, RUF012, ANN002)
stay at 0

* feat: add TID251 typing.Any/Dict import ban and widen annotation budgets

Add TID251 (flake8-tidy-imports banned-api) to ruff-strict.toml, banning new
imports of typing.Any and typing.Dict and steering new code toward structured
types. It counts the import site, about one per file, so it is set non-blocking
at 50 as a forward-looking signal

Widen the annotation-completeness budgets so they nudge rather than block:
ANN001 50, ANN401 50, ANN003 25. Correctness and structural rules stay at 0

* refactor: make the strict gate a drift-safe per-rule total ceiling

Switch the gate from a per-change allowance to a hard ceiling on each rule's
total count across the codebase. The ceiling is baseline + slack in
ruff-strict-budget.json, with baseline captured from today's tree

To stay drift-safe, the gate counts each rule on the head and on the merge-base
(via a throwaway git worktree) and fails a rule only when its head total is over
the ceiling and higher than the base, so base drift never blames a change that
did not add to that rule. Annotation rules keep generous slack (ANN001 and
ANN401 50, ANN003 25, TID251 50); structural and correctness rules are frozen at
today's count. Add make lint-strict-budget-update to re-capture baselines

* chore: give the structural strict rules a cushion of 3

To be liberal to start, B006, C901, PLR0913, PLW0603, RUF012, and ANN002 each get
a slack of 3 instead of 0, so an occasional legitimate case is not hard-blocked.
The annotation budgets are unchanged, and these ratchet down later

* feat: ban more typing collection aliases and tighten annotation slack to 10

Add typing.List, typing.Set, typing.MutableSequence, and typing.MutableMapping to
the TID251 banned-api list, steering new code toward tuple, Sequence, Mapping,
frozenset, and frozen dataclasses. This raises TID251's baseline to 2404

Bring the three rules that were at slack 50 (ANN001, ANN401, TID251) down to 10

* docs: document the strict-gate ratchet and Any-avoidance in CLAUDE.md

Add a line on running make lint-strict-budget-update to knock baselines down
after fixes, and a line on validating untyped inputs in the caller rather than
spending the Any budget

* feat: make it a bit more strict

---------

Co-authored-by: mateo-berri <[email protected]>

…erriAI#30338)

…I#30340)

These components are not reachable from any page: WebRTCTester, the agents agent_card/agent_card_grid pair (the agents route renders a Table directly), and view_logs ErrorViewer/RequestResponsePanel (logs uses inline equivalents in columns.tsx). Each was imported only by its own test, so the tests go too. Also drops the now-stale agent_card_grid vi.mock in agents.test.tsx and the WebRTCTester eslint-suppressions entry.

* ci: add osv-scanner lockfile scan workflow

Daily scheduled scan plus a pull_request scan scoped to uv.lock and the
dashboard package-lock.json. The osv-scanner v2.3.8 binary is fetched by
full release URL and verified against its official SHA-256 before use;
the job needs no credentials and runs with contents: read only.
osv-scanner.toml carries the single suppression for the diskcache
advisory, which has no fixed release published

* ci: temporary push trigger for runtime verification (will be dropped)

* ci: harden osv-scan per review (RUNNER_TEMP, job-scoped permissions, suppression expiry)

* ci: drop temporary push trigger after runtime verification

* ci: suppress aiohttp advisories while vcrpy blocks the 3.14 bump

Time-boxed like the diskcache entry: ignoreUntil forces a dated
re-triage if no vcrpy release has shipped by then

…tel v2 (BerriAI#30380)

The v2 span engine only stamped error.type and stuffed the message into the
span status description; it never recorded the standard OTel exception event.
Backends that dynamic-map unknown string fields (e.g. Elasticsearch) index the
message as a keyword capped at ignore_above:1024, truncating it. Emit the full
message under the recognized exception.message semconv field via a span event so
it is mapped as full text instead.

Co-authored-by: Claude <[email protected]>

…ls (BerriAI#30391)

TestFireworksAIAudioTranscription inherited two tests from
BaseLLMAudioTranscriptionTest that made real calls to
audio-prod.api.fireworks.ai. The audio host is a separately-entitled
product, and the CI key gets a 401 there, so both tests fail without
signalling anything about litellm.

Override them in the subclass with a dependency-injected mock OpenAI
client (the openai-compatible path uses the openai SDK, which is why the
failure surfaced as openai.AuthenticationError). The real routing,
model-prefix stripping, and response parsing still run; only the network
call is mocked. The sync path mocks audio.transcriptions.create and the
async path mocks audio.transcriptions.with_raw_response.create.

This matches the existing direction in this file, where the
document-inlining and global-disable tests were already converted off
live Fireworks calls.

esbuild reaches the dashboard tree transitively through vite (the vitest
stack), and vite 7.x still requests ^0.27.0, so it never advances past
0.27.x on its own. An overrides entry forces 0.28.1, which the osv-scan
job reports as the fixed version for the dev dependency. Keeping it in
package.json (not just the lockfile) means a lockfile regeneration cannot
silently roll it back.

…th route (BerriAI#30336)

* feat(ui): cut the organizations page over to the /ui/organizations path route

OrganizationsTable now owns its data through React Query instead of lifted
shell state: useOrganizations (extended with optional org_id/org_alias
filters) replaces the organizations/setOrganizations prop pair, and a new
useUserModels hook replaces the userModels prop. Create and delete
invalidate the organizations list queries rather than refetching into a
parent setter. The dead currentOrg and guardrailsList props are removed.

The shell keeps its own fetchOrganizations call because the teams and
api-keys arms still read the lifted organizations state; the userModels
state had no remaining readers and is deleted.

* fix(ui): seed organization detail initialData from any cached list, not just the unfiltered one

useOrganization's initialData only read organizationKeys.list({}), so on a
session that only ever fetched a filtered organization list the detail view
fell back to a loading state and a redundant info call. Scan every cached
list variant via the lists() prefix instead, with regression tests covering
the filtered-cache hit and the no-cache fallthrough.

* fix: add smtp ssl (BerriAI#30248)

* add smtp ssl

* fix comments'

* fix(proxy): verify SMTP server certificate on starttls

* dont read ssl from env

* test(proxy): restore regression test for SMTP_TLS=False starttls skip

---------

Co-authored-by: mubashir1osmani <[email protected]>

…AI#30396)

Linear is deprecating the SSE transport at https://mcp.linear.app/sse in
favor of streamable HTTP at https://mcp.linear.app/mcp. Update the curated
discovery registry so the Linear preset uses transport "http" and the /mcp
URL.