fix(provider-health): relax probe timeout for remote local-provider URLs#3146
Merged
Conversation
The local-provider probe used 1 s connect / 2 s total timeouts tuned for a loopback ollama on `localhost:11434`. When the same provider points at a remote reverse-proxy (e.g. Open WebUI under `https://host/ollama/v1`), TLS handshake plus WAN latency routinely exceeds 2 s on cold connections, flipping the catalog to `LocalOffline` and producing periodic `Configured local provider offline` WARNs even though the upstream ollama is healthy. Pick the timeout pair from the base URL: loopback hosts keep the original 1 s / 2 s budget so a dead local daemon still surfaces fast; everything else gets 3 s / 8 s — comfortable for cold TLS plus several WAN round-trips. Unparseable URLs fall back to the loopback budget rather than silently slowing every probe. Also flatten reqwest's error cause chain in the WARN line so future operators can see whether the failure was a timeout, DNS, TLS handshake, or HTTP status — the default Display impl drops the inner cause.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
reqwest renders timeouts as 'timed out' (no 'timeout' substring), so the
fallback's contains("timeout") check never matched and always appended
a redundant marker.
4 tasks
probe_provider built a fresh reqwest::Client on every call, throwing away the connection pool and TLS sessions between cycles. With the relaxed remote timeouts in place, every 60-second probe to an HTTPS reverse-proxy front (Open WebUI, remote ollama) was paying a full ~1-2 s cold-start TLS handshake — about 29 minutes of avoidable per-day latency per provider, plus extra connection log noise upstream. Hold the client in a process-wide OnceLock so the pool persists across probe cycles. The client is built with the looser remote timeouts; loopback callers tighten the total budget via per-request `RequestBuilder::timeout(\`PROBE_TIMEOUT_SECS\`)`. connect_timeout collapses to the remote 3 s value (reqwest cannot override it per-request), but a localhost daemon that cannot accept TCP within 3 s is already broken. Limitation: cache is not invalidated on `[proxy]` config reload. The motivating case — reverse-proxy URLs in `[provider_urls]` — does not involve forward proxies, so this is acceptable until someone hits it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Local-provider probes (ollama / vLLM / LM Studio / lemonade) used hard-coded 1 s connect / 2 s total timeouts tuned for a loopback daemon. When the same provider is fronted by a remote reverse-proxy — Open WebUI exposing ollama under
https://host/ollama/v1, a remote vLLM, etc. — TLS handshake plus WAN latency routinely exceeds 2 s on cold connections, so the probe intermittently times out and the catalog flips toLocalOffline. The visible symptom is periodiceven when the upstream ollama is healthy and curl from the same host returns 200 in ~1 s.
Fix
base_url:127.0.0.1,localhost,::1) keep the original1 s / 2 sbudget so a dead local daemon still surfaces quickly.3 s / 8 s— comfortable for a cold TLS handshake plus several WAN round-trips.Displaychain into the WARN'serror=...field. Previously every transport failure rendered as the genericerror sending request for url (...); now operators see whether it was a timeout, DNS, TLS handshake, or status code without flipping log level.Test plan
provider_healthunit tests added foris_loopback_base_urlcovering loopback, remote, LAN-mDNS, and unparseable inputs.test_probe_timeout_valueextended to assert the new constants and the invariantremote > local.[provider_urls].ollama = "https://webui-host/ollama/v1"andOLLAMA_API_KEYset, no more periodicConfigured local provider offlineWARNs over a 5-minute window.