Cherry-pick #29311 and #29343 onto patch/v1.84.3 by mateo-berri · Pull Request #29352 · BerriAI/litellm

mateo-berri · 2026-05-30T21:39:50Z

Cherry-picks two commits into patch/v1.84.3

Commits

[internal copy of fix: duplicate claude code traces #29089] fix: duplicate claude code traces ([internal copy of #29089] fix: duplicate claude code traces #29311) (628a65c)
- Routes pass-through streaming success logging through dispatch_success_handlers(prefer_async_handlers=True) instead of async_success_handler + a separate executor.submit sync call, eliminating duplicate traces.
- Resolved a merge conflict in litellm/proxy/pass_through_endpoints/streaming_handler.py: kept this branch's per-endpoint result-building block (ANTHROPIC/VERTEX_AI/OPENAI + None fallback) and adopted the new dispatch_success_handlers call.
refactor(proxy/auth): normalize Bearer prefix in safe-hash helper (refactor(proxy/auth): normalize Bearer prefix in safe-hash helper #29343) (23d9f1f)
- UserAPIKeyAuth._safe_hash_litellm_api_key strips a case-insensitive leading Bearer prefix before its sk-/JWT classification, so the hashed output is identical whether or not the caller stripped the Authorization header prefix.

Test plan

make test-unit

…9343) * refactor(proxy/auth): normalize Bearer prefix in safe-hash helper UserAPIKeyAuth._safe_hash_litellm_api_key now strips a leading "Bearer "/"bearer " prefix before its existing sk-/JWT classification, so the helper produces the same hashed output regardless of whether the caller stripped the Authorization header prefix or passed the header value through unchanged. * refactor(proxy/auth): make Bearer-prefix strip case-insensitive Per RFC 7235 the HTTP authorization scheme token is case-insensitive. Replace the two-prefix loop with a single case-insensitive check so the helper normalizes "Bearer ", "bearer ", "BEARER ", and any mixed-case variant before classifying the remainder as sk- or JWT. The contract test gains coverage of "BEARER " and "BeArEr ". * test(mcp): align auth-handler test expectations with safe-hash helper The two MCP auth tests asserted that UserAPIKeyAuth(api_key="Bearer ...") retained the raw header bytes on the api_key field. _safe_hash_litellm_api_key now normalizes that input — stripping the Bearer prefix and hashing the resulting sk- key — so the expectations move to the normalized form: the bare token in the parametrize case, and hash_token("sk-...") in the backward-compat assertion. This matches what the real auth flow produces (the builder strips Bearer and the DB stores the hashed token), so the mocks now line up with production rather than with the un-normalized validator output.

greptile-apps · 2026-05-30T21:45:56Z

Greptile Summary

This cherry-pick backport onto patch/v1.84.3 contains two independent fixes: (1) eliminates duplicate Claude Code traces in pass-through streaming by consolidating all success-logging dispatch into a new dispatch_success_handlers method with a prefer_async_handlers flag and a has_dispatched_final_stream_success dedup guard; and (2) normalises the Bearer prefix in UserAPIKeyAuth._safe_hash_litellm_api_key so that keys are consistently hashed whether or not the caller has already stripped the Authorization header prefix.

Duplicate-trace fix (#29311): introduces dispatch_success_handlers in Logging, which internally routes to async_success_handler and/or executor.submit(success_handler) based on call type and the new prefer_async_handlers flag. All proxy/pass-through streaming paths migrate from the previous async_success_handler + executor.submit pair to this single entry-point, removing the duplicate-log paths.
Bearer normalisation fix (#29343): _safe_hash_litellm_api_key now strips a case-insensitive \"Bearer \" prefix before classifying the token; sk- keys that previously fell through unmatched (stored raw) are now correctly hashed. The MCP auth test is updated to assert the hashed value rather than the unhashed Bearer string.

Confidence Score: 4/5

Safe to merge; the logging refactor is well-tested and the auth fix corrects a pre-existing bug where Bearer-prefixed sk- keys were stored unhashed.

The dispatch_success_handlers consolidation is logically sound — should_run_logging never sets the has_logged flag for streaming, so the new assembled-stream bypass in async_success_handler is redundant rather than risky. The _safe_hash_litellm_api_key Bearer-strip change silently alters the fallthrough return value for non-sk, non-JWT tokens (from "Bearer token" to "token"); any downstream cache lookup keyed on the raw header form would miss, though this path is unlikely to be hit in practice. Both changes carry thorough unit tests.

litellm/proxy/_types.py — verify no downstream cache or comparison relies on the Bearer-prefixed form of non-sk, non-JWT tokens in the _safe_hash_litellm_api_key fallthrough path.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/litellm_logging.py	Adds `dispatch_success_handlers`, `_is_assembled_stream_success`, and `_is_sync_litellm_request`; refactors `async_success_handler` dedup guard for assembled streaming responses.
litellm/proxy/_types.py	Strips case-insensitive "Bearer " prefix before sk-/JWT classification in `_safe_hash_litellm_api_key`. Fixes existing bug where "Bearer sk-…" keys were returned raw (unhashed). Fallthrough `return normalized` subtly changes behavior for non-sk, non-JWT tokens with a Bearer prefix.
litellm/litellm_core_utils/streaming_handler.py	Per-chunk sync logging now correctly gates on `_is_sync_litellm_request`; final assembled-stream logging migrated from paired `async_success_handler` + `executor.submit` to single `dispatch_success_handlers(prefer_async_handlers=True)`.
litellm/proxy/common_request_processing.py	Replaces paired async/sync handler calls with `dispatch_success_handlers(prefer_async_handlers=True)` in the orphaned-stream and deferred-guardrail paths; removes the now-redundant `executor.submit(success_handler)` calls.
litellm/proxy/pass_through_endpoints/streaming_handler.py	Migrates `_route_streaming_logging_to_handler` from dual `async_success_handler`/`executor.submit` pattern to `dispatch_success_handlers(prefer_async_handlers=True)`, removing the duplicate trace source for SDK pass-through streams.
litellm/proxy/pass_through_endpoints/success_handler.py	Consolidates `_handle_logging` from two separate handler dispatches into a single `dispatch_success_handlers(prefer_async_handlers=True)` call.
tests/test_litellm/litellm_core_utils/test_litellm_logging.py	Adds comprehensive unit tests for `dispatch_success_handlers`: dedup guard, sync/async routing, `prefer_async_handlers` legacy callback behaviour, and pass-through path.
tests/test_litellm/proxy/guardrails/test_deferred_guardrail_logging.py	Mock setup updated from direct `async_success_handler` assignment to `_attach_mock_success_dispatch` helper to align with the new `dispatch_success_handlers` call in production; existing assertions unchanged.
tests/test_litellm/proxy/_experimental/mcp_server/auth/test_user_api_key_auth_mcp.py	Test assertions updated to reflect the Bearer-strip fix: `api_key` is now expected to be `hash_token("sk-litellm-valid-key")` rather than the raw unhashed Bearer-prefixed string.
tests/pass_through_unit_tests/test_unit_test_streaming.py	Two new regression tests verify async success handler fires (and sync does not) for SDK pass-through streaming and non-streaming paths.
tests/proxy_unit_tests/test_proxy_reject_logging.py	Adds `_register_proxy_test_logger` helper and changes embedding test model to `fake-model`.
tests/test_litellm/litellm_core_utils/test_streaming_handler.py	Minor style clean-ups only: removes a stray debug `print` and reformats long assert expressions.
tests/test_litellm/proxy/test_proxy_types.py	Adds test verifying all case variants of the "Bearer " prefix produce the same hash as the bare sk- key.

_{Reviews (1): Last reviewed commit: "refactor(proxy/auth): normalize Bearer p..." | Re-trigger Greptile}

greptile-apps · 2026-05-30T21:46:00Z

+        if JWTHandler.is_jwt(token=normalized):
+            return f"hashed-jwt-{hash_token(token=normalized)}"
+        return normalized


The fallthrough branch now returns normalized (the Bearer-stripped value) instead of the original api_key. For sk- and JWT tokens this is the correct fix, but for any other token type that happens to carry a "Bearer " prefix (e.g. an opaque OAuth2 access token passed verbatim from an Authorization header), the stored api_key changes silently from "Bearer <token>" to "<token>". Any downstream cache lookup or comparison that was built against the raw header value would miss. Consider adding a comment explicitly documenting the new contract.

Suggested change

if JWTHandler.is_jwt(token=normalized):

return f"hashed-jwt-{hash_token(token=normalized)}"

return normalized

if JWTHandler.is_jwt(token=normalized):

return f"hashed-jwt-{hash_token(token=normalized)}"

# For unrecognised token types (neither sk- nor JWT) return the

# normalized (Bearer-stripped) form so the caller always receives

# a prefix-free value regardless of how the header was forwarded.

return normalized

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

greptile-apps · 2026-05-30T21:46:01Z

+    @staticmethod
+    def _is_sync_litellm_request(litellm_params: dict) -> bool:
+        """True for sync SDK entrypoints (``completion``), false for async (``acompletion``, etc.)."""
+        return (
+            litellm_params.get(CallTypes.acompletion.value, False) is not True
+            and litellm_params.get(CallTypes.aresponses.value, False) is not True
+            and litellm_params.get(CallTypes.aembedding.value, False) is not True
+            and litellm_params.get(CallTypes.aimage_generation.value, False) is not True
+            and litellm_params.get(CallTypes.atranscription.value, False) is not True
+        )


_is_sync_litellm_request checks five specific async CallTypes flags but misses several others (e.g. atext_completion, aspeech, acreate_batch, aretrieve_batch, and thread-related async types). This is a direct copy of the pre-existing inline check, so it is not a regression here. Now that the logic is centralised it is worth documenting the incomplete coverage so future callers know to use prefer_async_handlers=True for any async path not covered by this heuristic.

Suggested change

@staticmethod

def _is_sync_litellm_request(litellm_params: dict) -> bool:

"""True for sync SDK entrypoints (``completion``), false for async (``acompletion``, etc.)."""

return (

litellm_params.get(CallTypes.acompletion.value, False) is not True

and litellm_params.get(CallTypes.aresponses.value, False) is not True

and litellm_params.get(CallTypes.aembedding.value, False) is not True

and litellm_params.get(CallTypes.aimage_generation.value, False) is not True

and litellm_params.get(CallTypes.atranscription.value, False) is not True

)

@staticmethod

def _is_sync_litellm_request(litellm_params: dict) -> bool:

"""True for sync SDK entrypoints (``completion``), false for async (``acompletion``, etc.).

Note: only a subset of async call types are checked here; this mirrors

the pre-existing inline logic. Pass-through and proxy streaming paths

use ``prefer_async_handlers=True`` via ``dispatch_success_handlers`` to

force async dispatch regardless of this heuristic.

"""

return (

litellm_params.get(CallTypes.acompletion.value, False) is not True

and litellm_params.get(CallTypes.aresponses.value, False) is not True

and litellm_params.get(CallTypes.aembedding.value, False) is not True

and litellm_params.get(CallTypes.aimage_generation.value, False) is not True

and litellm_params.get(CallTypes.atranscription.value, False) is not True

)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

mateo-berri and others added 2 commits May 30, 2026 14:36

[internal copy of #29089] fix: duplicate claude code traces (#29311)

628a65c

mateo-berri requested a review from yuneng-berri May 30, 2026 21:39

greptile-apps Bot reviewed May 30, 2026

View reviewed changes

yuneng-berri merged commit a0bb8e6 into patch/v1.84.3 May 31, 2026
65 of 75 checks passed

This was referenced Jun 1, 2026

Cherry-pick #29311, #29343, #29358, #27913, and #29447 onto stable/1.85.x #29458

Closed

Cherry-pick #29311, #29343, #29358, #27913, and #29447 onto stable/1.85.x #29460

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Cherry-pick #29311 and #29343 onto patch/v1.84.3#29352

Cherry-pick #29311 and #29343 onto patch/v1.84.3#29352
yuneng-berri merged 2 commits into
patch/v1.84.3from
litellm_cherrypick_29311_29343

mateo-berri commented May 30, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented May 30, 2026

Important Files Changed

Uh oh!

greptile-apps Bot May 30, 2026

Uh oh!

greptile-apps Bot May 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

mateo-berri commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Commits

Test plan

Uh oh!

greptile-apps Bot commented May 30, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

greptile-apps Bot May 30, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mateo-berri commented May 30, 2026 •

edited

Loading