You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Task][AI Proxy] Harden configurable retry and fallback policy in AiProxyEngine #7667
Follow up on #7532 to make retry and fallback decisions explicit, configurable, and independently testable. Retry and fallback remain orchestration responsibilities of AiProxyEngine; Transport only reports classified failures.
Scope
Extract HTTP status and transport failure decisions into AiProxyRetryPolicy.
Define explicit retry and fallback eligibility for upstream HTTP statuses.
Define explicit retry and fallback eligibility for classified transport failures.
Configure retry limits independently for non-streaming and streaming requests.
Support configurable exponential backoff for non-streaming retries.
Retry the primary Provider before attempting fallback, and attempt fallback at most once.
Rebuild fallback requests using the fallback Provider configuration.
Prevent retry or fallback after response data has been emitted.
Add unit tests for decision matrices, configuration binding, backoff, fallback, and streaming boundaries.
Out of Scope
Connection pool, timeout, TLS verification, HTTP proxy, and transport lifecycle configuration.
Health-based multi-provider load balancing or automatic Provider selection.
Mapping terminal transport failures to gateway HTTP 502/503/504 responses.
Protocol and Provider request/response adaptation.
Acceptance Criteria
Retry and fallback decisions are centralized and independently testable.
Only explicitly eligible statuses and transport failures are retried or switched to fallback.
Retry counts and initial backoff can be configured through Spring Boot configuration.
Streaming retries happen only before the first response chunk is emitted.
No retry or fallback occurs after downstream response emission starts.
Activity