fix: duplicate claude code traces by mubashir1osmani · Pull Request #29089 · BerriAI/litellm

mubashir1osmani · 2026-05-27T23:38:10Z

Relevant issues

Linear ticket

fixes LIT-3315
LIT-3414
LIT-3415

proposing a long term solution for duplicate claude code traces in datadog and arize:

Fixes duplicate observability exports (Datadog logs, OTLP traces to Arize Phoenix / Langfuse, etc.) caused by calling both async_success_handler and success_handler on the same completed stream.
Introduces dispatch_success_handlers() as the single orchestration entry for post-request success logging on proxy/streaming paths.
Adds a final-stream idempotency guard so multiple code paths that finish the same stream (CSW stream end, deferred guardrail closure, pass-through streaming) only emit one full success export per request.

Problem

For async proxy traffic (e.g. Anthropic /v1/messages streaming), several places fired success logging twice:

asyncio.create_task(async_success_handler(...))
executor.submit(success_handler, ...)

CustomLogger integrations normally skip sync export on async requests, but streaming deliberately disables has_logged_async_success so per-chunk hooks can run. That left final assembled-response logging vulnerable to duplicate calls from different orchestration paths—showing up as duplicate rows in Datadog Logs Explorer and duplicate spans in OTLP backends.

Callback-list dedupe alone does not fix this; the bug is when handlers run, not duplicate entries in the callback array.

Solution

add a helper function to differentiate sync and async endpoints then dispatch success handlers. this acts a single orchestartinon point that routes to the correct handler.

async calls hits acompletion -> async_success_handler()
sync (litellm sdk) hits completion -> success_handler()
pass through endpoints -> async success handler()

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

50-55 passing tests: main is stable with minor issues.

45-49 passing tests: acceptable but needs attention

<= 40 passing tests: unstable; be careful with your merges and assess the risk.

Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:

Screenshots / Proof of Fix

Before:

After:

Datadog before:

after:

Type

🐛 Bug Fix

Changes

greptile-apps · 2026-05-27T23:43:56Z

Greptile Summary

This PR fixes duplicate observability exports (Datadog, Arize, Langfuse, etc.) on async proxy streaming paths by introducing dispatch_success_handlers as the single orchestration entry point for post-request success logging, backed by a has_dispatched_final_stream_success idempotency flag on model_call_details.

Core fix: dispatch_success_handlers routes to success_handler (sync SDK), async_success_handler + optional executor.submit (async/pass-through), and guards final-stream calls with an idempotency flag that prevents any second dispatch from re-running either handler.
Per-chunk sync logging in streaming_handler.py is now gated behind _is_sync_litellm_request, eliminating the path that previously emitted duplicate sync callbacks on async proxy streams.
All call sites in common_request_processing.py, pass_through_endpoints/streaming_handler.py, and pass_through_endpoints/success_handler.py are consolidated to use dispatch_success_handlers, removing the scattered manual dual-call pairs that were the root cause.

Confidence Score: 4/5

The core fix is sound and the idempotency guard covers both handler paths, but the change touches central logging infrastructure across streaming, pass-through, and deferred guardrail paths.

The guard in dispatch_success_handlers correctly fires before both the async and sync handlers. The _is_assembled_stream_success heuristic (result is not None) is safe today because per-chunk calls never route through dispatch_success_handlers, but it creates a false contract that could silently suppress final exports in future refactors. The guardrail test mock omits the idempotency guard, reducing regression sensitivity for the deferred path.

litellm/litellm_core_utils/litellm_logging.py for the _is_assembled_stream_success heuristic; tests/test_litellm/proxy/guardrails/test_deferred_guardrail_logging.py for the mock missing the idempotency guard

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/litellm_logging.py	Adds dispatch_success_handlers, _is_sync_litellm_request, and _is_assembled_stream_success; moves idempotency guard to dispatch level; changes async_success_handler to bypass should_run_logging for streaming results
litellm/litellm_core_utils/streaming_handler.py	Per-chunk sync logging gated behind _is_sync_litellm_request; final stream dispatches through dispatch_success_handlers instead of separate async+sync calls
litellm/proxy/common_request_processing.py	Deferred guardrail closure and orphaned streaming paths consolidated to use dispatch_success_handlers; removes duplicate executor.submit calls
tests/test_litellm/litellm_core_utils/test_litellm_logging.py	Adds four new tests covering deduplication for async, sync, prefer_async_handlers, and pass-through dispatch paths using real callbacks
tests/test_litellm/proxy/guardrails/test_deferred_guardrail_logging.py	Updates async_success_handler stubs to _attach_mock_success_dispatch; mock helper forwards to tracked function but omits the idempotency guard

_{Reviews (5): Last reviewed commit: "changes" | Re-trigger Greptile}

codecov · 2026-05-27T23:46:03Z

Codecov Report

❌ Patch coverage is 91.89189% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/litellm_core_utils/streaming_handler.py	66.66%	2 Missing ⚠️
litellm/litellm_core_utils/litellm_logging.py	96.42%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

mubashir1osmani · 2026-05-28T00:01:31Z

@greptileai review it

mubashir1osmani · 2026-05-28T00:28:53Z

@greptileai fixed, review again

mubashir1osmani · 2026-05-28T00:40:55Z

this pr is buns let me find a better way to do thsi

mubashir1osmani · 2026-05-28T21:52:35Z

@greptileai - review it, is this code quality better?

mubashir1osmani · 2026-05-29T17:21:49Z

@greptile-apps review this

* [internal copy of #29089] fix: duplicate claude code traces (#29311) * refactor(proxy/auth): normalize Bearer prefix in safe-hash helper (#29343) * refactor(proxy/auth): normalize Bearer prefix in safe-hash helper UserAPIKeyAuth._safe_hash_litellm_api_key now strips a leading "Bearer "/"bearer " prefix before its existing sk-/JWT classification, so the helper produces the same hashed output regardless of whether the caller stripped the Authorization header prefix or passed the header value through unchanged. * refactor(proxy/auth): make Bearer-prefix strip case-insensitive Per RFC 7235 the HTTP authorization scheme token is case-insensitive. Replace the two-prefix loop with a single case-insensitive check so the helper normalizes "Bearer ", "bearer ", "BEARER ", and any mixed-case variant before classifying the remainder as sk- or JWT. The contract test gains coverage of "BEARER " and "BeArEr ". * test(mcp): align auth-handler test expectations with safe-hash helper The two MCP auth tests asserted that UserAPIKeyAuth(api_key="Bearer ...") retained the raw header bytes on the api_key field. _safe_hash_litellm_api_key now normalizes that input — stripping the Bearer prefix and hashing the resulting sk- key — so the expectations move to the normalized form: the bare token in the parametrize case, and hash_token("sk-...") in the backward-compat assertion. This matches what the real auth flow produces (the builder strips Bearer and the DB stores the hashed token), so the mocks now line up with production rather than with the un-normalized validator output. --------- Co-authored-by: yuneng-jiang <[email protected]>

The #29311 cherry-pick onto stable/1.85.x carried the test test_route_streaming_logging_runs_async_handler_for_sdk_passthrough, which patches PassThroughStreamingHandler._build_passthrough_logging_result to verify the SDK-passthrough dispatch contract. The manual conflict resolution kept the per-endpoint if/elif/elif chain inline in _route_streaming_logging_to_handler (matching v1.84.4's resolution), so the patched attribute did not exist and the test errored at collection with AttributeError. Extract the chain into the static _build_passthrough_logging_result helper as #29089 originally designed it. _route_streaming_logging_to_handler now resolves (standard_logging_response_object, kwargs) through the helper and dispatches via dispatch_success_handlers; the helper itself is synchronous and CPU-bound, suitable for the unit test's patch target. Verified locally: tests/pass_through_unit_tests/test_unit_test_streaming.py passes (5/5) and tests/test_litellm/litellm_core_utils/test_litellm_logging.py passes (83/83). v1.84.4 ships with the same broken test; this strictly improves on that resolution.

…erriAI#29311)

add

90889c4

greptile-apps Bot reviewed May 27, 2026

View reviewed changes

Comment thread litellm/litellm_core_utils/litellm_logging.py Outdated

Comment thread litellm/litellm_core_utils/litellm_logging.py Outdated

Comment thread tests/test_litellm/litellm_core_utils/test_litellm_logging.py

fix test

d78060d

mubashir1osmani added 2 commits May 27, 2026 20:21

fix greptile

40cccfa

add test

4be0de3

code quality

144680a

greptile-apps Bot reviewed May 28, 2026

View reviewed changes

Comment thread litellm/litellm_core_utils/streaming_handler.py

changes

98b7b18

fix ci

f770d8e

This was referenced May 29, 2026

[internal copy of #29089] fix: duplicate claude code traces #29311

Merged

fix(passthrough): run async success handler for SDK pass-through streaming #29312

Merged

mateo-berri added a commit that referenced this pull request May 30, 2026

[internal copy of #29089] fix: duplicate claude code traces (#29311)

581c30f

This was referenced May 30, 2026

Cherry-pick #29311 and #29343 onto patch/v1.84.3 #29351

Closed

Cherry-pick #29311 and #29343 onto patch/v1.84.3 #29352

Merged

This was referenced Jun 1, 2026

Cherry-pick #29311, #29343, #29358, #27913, and #29447 onto stable/1.85.x #29458

Closed

Cherry-pick #29311, #29343, #29358, #27913, and #29447 onto stable/1.85.x #29460

Merged

This was referenced Jun 2, 2026

bump: version 1.85.2 → 1.85.3 #29464

Merged

chore(release): backport pending 1.84.x/1.85.x fixes into stable/1.86.x and cut 1.86.3 #29541

Merged

mubashir1osmani closed this Jun 6, 2026

fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026

[internal copy of BerriAI#29089] fix: duplicate claude code traces (B…

cb4c2cd

…erriAI#29311)

yassin-berriai mentioned this pull request Jul 6, 2026

fix(responses): stop scheduling sync success_handler concurrently with async_success_handler #32239

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix: duplicate claude code traces#29089

fix: duplicate claude code traces#29089
mubashir1osmani wants to merge 7 commits into
BerriAI:litellm_internal_stagingfrom
mubashir1osmani:litellm__duplicate_cc_trce

mubashir1osmani commented May 27, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented May 27, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented May 27, 2026 •

edited

Loading

Uh oh!

mubashir1osmani commented May 28, 2026

Uh oh!

mubashir1osmani commented May 28, 2026

Uh oh!

mubashir1osmani commented May 28, 2026

Uh oh!

mubashir1osmani commented May 28, 2026

Uh oh!

Uh oh!

mubashir1osmani commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

mubashir1osmani commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Relevant issues

Linear ticket

Problem

Pre-Submission checklist

Delays in PR merge?

CI (LiteLLM team)

Screenshots / Proof of Fix

Type

Changes

Uh oh!

greptile-apps Bot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mubashir1osmani commented May 28, 2026

Uh oh!

mubashir1osmani commented May 28, 2026

Uh oh!

mubashir1osmani commented May 28, 2026

Uh oh!

mubashir1osmani commented May 28, 2026

Uh oh!

Uh oh!

mubashir1osmani commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mubashir1osmani commented May 27, 2026 •

edited

Loading

greptile-apps Bot commented May 27, 2026 •

edited

Loading

codecov Bot commented May 27, 2026 •

edited

Loading