Repository navigation
Commit 0598bab
fix(retry): retry on httpx.TimeoutException with HttpRetryOptions (#2345)
The retry predicate in `retry_args` only matched `errors.APIError` subclasses,
so lower-level httpx transport exceptions (`httpx.TimeoutException`,
`httpx.ConnectError`, and their subclasses such as `ReadTimeout` /
`ConnectTimeout`) were silently dropped and never retried, even when
`HttpRetryOptions` was explicitly configured.
Long-running LLM inference calls regularly hit transient transport errors
from load balancers, NAT gateways, and idle TCP timeouts. Vertex AI's
retry strategy guidance lists socket timeouts and TCP disconnects as
retryable, so honoring those here matches user expectations.
Fixes #2337.
Co-authored-by: Amy Wu <[email protected]>1 parent f8a2e7e commit 0598bab
2 files changed
Lines changed: 19 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
524 | 524 | | |
525 | 525 | | |
526 | 526 | | |
527 | | - | |
| 527 | + | |
| 528 | + | |
528 | 529 | | |
529 | 530 | | |
530 | 531 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
187 | 187 | | |
188 | 188 | | |
189 | 189 | | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
190 | 207 | | |
191 | 208 | | |
192 | 209 | | |
| |||
0 commit comments