test(streaming): tolerate Vertex 429 wrapped in MidStreamFallbackError#28669
Conversation
Streaming 429s are wrapped in MidStreamFallbackError so the Router can fall back; the existing 'except litellm.RateLimitError: pass' in test_vertex_ai_stream no longer matches, causing the generic pytest.fail branch to fire when upstream Vertex returns 429. Add a sibling except for MidStreamFallbackError that only swallows it when e.original_exception is a RateLimitError, so unrelated streaming failures still fail the test.
Greptile SummaryThis PR adds a targeted
Confidence Score: 5/5Safe to merge — single-file change to a test helper that only widens transient-error tolerance. The change touches only one test file and one exception-handling block. The MidStreamFallbackError.original_exception attribute is defined and always set (or None, in which case the guard correctly routes to pytest.fail). No production code is altered, and the test's ability to surface real streaming errors is fully preserved. No files require special attention.
|
| Filename | Overview |
|---|---|
| tests/local_testing/test_streaming.py | Adds a MidStreamFallbackError catch clause to test_vertex_ai_stream that silently passes only when the wrapped original_exception is a RateLimitError, matching the existing 429-tolerance pattern. |
Reviews (1): Last reviewed commit: "test(streaming): tolerate Vertex 429 wra..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
yuneng-berri
merged commit 574ee75
into
litellm_internal_staging
BerriAI#28669) Streaming 429s are wrapped in MidStreamFallbackError so the Router can fall back; the existing 'except litellm.RateLimitError: pass' in test_vertex_ai_stream no longer matches, causing the generic pytest.fail branch to fire when upstream Vertex returns 429. Add a sibling except for MidStreamFallbackError that only swallows it when e.original_exception is a RateLimitError, so unrelated streaming failures still fail the test.
2 participants
What
tests/local_testing/test_streaming.py::test_vertex_ai_stream[vertex_ai_beta]is currently blocking CI when the live Vertex AI endpoint returns 429.The test has a tolerance for rate limits:
…but streaming 429s are now (intentionally) wrapped in
MidStreamFallbackErrorbyCustomStreamWrapper._handle_stream_fallback_errorso the Router can trigger a fallback. The bareRateLimitErrorcatch no longer matches; the genericexcept Exception → pytest.failfires instead.Change
Add a sibling
except litellm.exceptions.MidStreamFallbackErrorclause that swallows the error only whene.original_exceptionis aRateLimitError, so the test still fails on unrelated streaming errors. Same pattern already used attest_streaming.py:2925.Test plan
make test-unitpasses locally; the targeted test no longer fails on an upstream 429.