Relevant issues

Fixes zero spend on successful router-retried calls when intermediate failures pin response_cost to 0 (regression interaction with #21844).

Root cause: failure_handler sets response_cost = 0 on each failed attempt (#4604). PR #21844 added a preserve branch in _process_hidden_params_and_response_cost that kept any non-None response_cost, including 0 / 0.0, so a later success with full usage never recalculated cost.

How the fix helps the customer case: Router retries share one Logging object. Failed attempts set response_cost to 0; the successful Bedrock/passthrough response has real usage. We no longer treat that stale 0 as a pre-calculated cost—we recalculate from the success response so spend logs and cost_breakdown match token usage (~$0.03 for ~9.7k + 30 tokens). Pass-through handlers that set a non-zero cost (or response_cost on result._hidden_params) are unchanged.

Linear ticket

Resolves LIT-3261

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit (CI on PR)
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link: (fill after PR opened)

• CI run for the last commit
  Link: (fill after PR opened)

• Merge / cherry-pick CI run
  Links: (fill after PR opened)

Screenshots / Proof of Fix

Before (on litellm_internal_staging without this PR)

Simulate two failed attempts (failure_handler → response_cost = 0), then process a successful ModelResponse with usage (same shared Logging object as router retries):

cd berriai/litellm
source ../litellm-proxy/venv_berriai/bin/activate  # or your dev venv

python <<'PY'
from datetime import datetime
import litellm
from litellm.litellm_core_utils.litellm_logging import Logging
from litellm.types.utils import ModelResponse, Usage

logging_obj = Logging(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "hi"}],
    stream=False,
    call_type="acompletion",
    start_time=datetime.now(),
    litellm_call_id="repro",
    function_id="repro",
)
logging_obj.model_call_details["litellm_params"] = {"model": "openai/gpt-4o-mini"}
logging_obj.optional_params = {}

err = litellm.RateLimitError(message="rate limit", llm_provider="openai", model="openai/gpt-4o-mini")
for _ in range(2):
    logging_obj._failure_handler_helper_fn(
        exception=err, traceback_exception="", start_time=datetime.now(), end_time=datetime.now()
    )

resp = ModelResponse(
    id="ok",
    choices=[{"message": {"role": "assistant", "content": "ok"}}],
    usage=Usage(prompt_tokens=9698, completion_tokens=30, total_tokens=9728),
)
logging_obj._process_hidden_params_and_response_cost(resp, datetime.now(), datetime.now())
print("response_cost:", logging_obj.model_call_details.get("response_cost"))
print("slo response_cost:", (logging_obj.model_call_details.get("standard_logging_object") or {}).get("response_cost"))
PY

Expected before fix: response_cost: 0 and slo response_cost: 0.0 despite 9728 tokens (matches customer report on v1.85.0).

After (this branch)

Run all regression tests:

pytest tests/test_litellm/litellm_core_utils/test_litellm_logging.py::test_process_hidden_params_recalculates_cost_after_failure_handler_zero \
  tests/test_litellm/litellm_core_utils/test_litellm_logging.py::test_process_hidden_params_preserves_zero_cost_in_hidden_params \
  tests/test_litellm/litellm_core_utils/test_litellm_logging.py::test_process_hidden_params_uses_hidden_params_cost_after_failure_handler_zero \
  tests/test_litellm/litellm_core_utils/test_litellm_logging.py::test_async_success_handler_preserves_response_cost_for_pass_through_endpoints \
  -q

Expected after fix:

....                                                                     [100%]
4 passed

Manual check on this branch (same script as above) prints non-zero cost (~0.0014727 for gpt-4o-mini at 9698+30 tokens).

Customer symptom (context)

• Bedrock allm_passthrough_route, attempted_retries: 2, ~9.7k prompt + 30 completion tokens, cost_breakdown all zeros on success row.
• Not reproduced end-to-end on live proxy + HTTP 429 mock (OpenAI SDK internal retries / set_response_headers recalc differ from Bedrock passthrough path). Unit repro above matches the logging bug mechanism.

Greptile P2 (!= 0 vs > 0)

Greptile suggested existing_cost != 0 instead of > 0. For 0 and 0.0, == 0 is true in Python, so both guards recalculate the same way. We use != 0 per review; legitimate zero cost on result._hidden_params is still preserved via the earlier branch ("response_cost" in hidden_params).

Type

🐛 Bug Fix

Changes

• litellm/litellm_core_utils/litellm_logging.py: In _process_hidden_params_and_response_cost, only preserve existing response_cost when it is != 0 (pass-through pre-calculated non-zero). Do not preserve 0 / 0.0 from failure_handler on intermediate router retries; recalculate from usage on success.
• tests/test_litellm/litellm_core_utils/test_litellm_logging.py:
  • test_process_hidden_params_recalculates_cost_after_failure_handler_zero
  • test_process_hidden_params_preserves_zero_cost_in_hidden_params
  • test_process_hidden_params_uses_hidden_params_cost_after_failure_handler_zero