Skip to content

fix(provider-health): relax probe timeout for remote local-provider URLs#3146

Merged
houko merged 6 commits into
mainfrom
fix/probe-remote-url-timeout
Apr 25, 2026
Merged

fix(provider-health): relax probe timeout for remote local-provider URLs#3146
houko merged 6 commits into
mainfrom
fix/probe-remote-url-timeout

Conversation

@houko

@houko houko commented Apr 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Local-provider probes (ollama / vLLM / LM Studio / lemonade) used hard-coded 1 s connect / 2 s total timeouts tuned for a loopback daemon. When the same provider is fronted by a remote reverse-proxy — Open WebUI exposing ollama under https://host/ollama/v1, a remote vLLM, etc. — TLS handshake plus WAN latency routinely exceeds 2 s on cold connections, so the probe intermittently times out and the catalog flips to LocalOffline. The visible symptom is periodic

WARN Configured local provider offline provider=ollama error="error sending request for url (https://host/ollama/api/tags)"

even when the upstream ollama is healthy and curl from the same host returns 200 in ~1 s.

Fix

  • Pick the timeout pair from base_url:
    • Loopback hosts (127.0.0.1, localhost, ::1) keep the original 1 s / 2 s budget so a dead local daemon still surfaces quickly.
    • Remote hosts get 3 s / 8 s — comfortable for a cold TLS handshake plus several WAN round-trips.
  • Unparseable URLs fall back to the loopback budget so a malformed config doesn't silently slow every probe.
  • Flatten reqwest's Display chain into the WARN's error=... field. Previously every transport failure rendered as the generic error sending request for url (...); now operators see whether it was a timeout, DNS, TLS handshake, or status code without flipping log level.

Test plan

  • provider_health unit tests added for is_loopback_base_url covering loopback, remote, LAN-mDNS, and unparseable inputs.
  • Existing test_probe_timeout_value extended to assert the new constants and the invariant remote > local.
  • CI green
  • Manual: with [provider_urls].ollama = "https://webui-host/ollama/v1" and OLLAMA_API_KEY set, no more periodic Configured local provider offline WARNs over a 5-minute window.

The local-provider probe used 1 s connect / 2 s total timeouts tuned for a
loopback ollama on `localhost:11434`. When the same provider points at a
remote reverse-proxy (e.g. Open WebUI under `https://host/ollama/v1`),
TLS handshake plus WAN latency routinely exceeds 2 s on cold connections,
flipping the catalog to `LocalOffline` and producing periodic
`Configured local provider offline` WARNs even though the upstream
ollama is healthy.

Pick the timeout pair from the base URL: loopback hosts keep the original
1 s / 2 s budget so a dead local daemon still surfaces fast; everything
else gets 3 s / 8 s — comfortable for cold TLS plus several WAN
round-trips. Unparseable URLs fall back to the loopback budget rather
than silently slowing every probe.

Also flatten reqwest's error cause chain in the WARN line so future
operators can see whether the failure was a timeout, DNS, TLS handshake,
or HTTP status — the default Display impl drops the inner cause.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions github-actions Bot added the ready-for-review PR is ready for maintainer review label Apr 25, 2026
@github-actions github-actions Bot added area/runtime Agent loop, LLM drivers, WASM sandbox size/M 50-249 lines changed labels Apr 25, 2026
reqwest renders timeouts as 'timed out' (no 'timeout' substring), so the
fallback's contains("timeout") check never matched and always appended
a redundant marker.
houko added 4 commits April 25, 2026 23:24
probe_provider built a fresh reqwest::Client on every call, throwing away
the connection pool and TLS sessions between cycles. With the relaxed
remote timeouts in place, every 60-second probe to an HTTPS reverse-proxy
front (Open WebUI, remote ollama) was paying a full ~1-2 s cold-start
TLS handshake — about 29 minutes of avoidable per-day latency per
provider, plus extra connection log noise upstream.

Hold the client in a process-wide OnceLock so the pool persists across
probe cycles. The client is built with the looser remote timeouts;
loopback callers tighten the total budget via per-request
`RequestBuilder::timeout(\`PROBE_TIMEOUT_SECS\`)`. connect_timeout
collapses to the remote 3 s value (reqwest cannot override it
per-request), but a localhost daemon that cannot accept TCP within 3 s
is already broken.

Limitation: cache is not invalidated on `[proxy]` config reload. The
motivating case — reverse-proxy URLs in `[provider_urls]` — does not
involve forward proxies, so this is acceptable until someone hits it.
@houko
houko merged commit 545ca15 into main Apr 25, 2026
20 checks passed
@houko
houko deleted the fix/probe-remote-url-timeout branch April 25, 2026 15:20
@github-actions github-actions Bot removed the ready-for-review PR is ready for maintainer review label Apr 25, 2026
@houko houko mentioned this pull request Apr 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/runtime Agent loop, LLM drivers, WASM sandbox size/M 50-249 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant