Skip to content

Commit 9210808

Browse files
fedegtoneclaude
andcommitted
fix: enable TCP keepalive on the OpenAI client to prevent stale-connection hangs
Root cause of the intermittent multi-minute hangs during multi-place imports: the OpenAI SDK's default transport does not enable TCP-level SO_KEEPALIVE (unlike the Anthropic SDK, which already does). During a burst of sequential calls, if a NAT gateway or load balancer silently drops an idle pooled connection, the client has no way to detect the dead connection at the OS level and blocks on it until the outer request timeout fires. Known upstream issue: openai/openai-python#3269. Confirmed via two live incidents in the same test session, on two different place names, immediately preceded and followed by normal- speed calls -- ruling out both the specific content and a rate-limit cascade, and matching the documented failure mode exactly. Mirrors the socket options the Anthropic SDK already applies by default (anthropic._base_client._DefaultHttpxClient), so both providers now have equivalent connection resilience. Co-Authored-By: Claude Sonnet 5 <[email protected]>
1 parent d14157e commit 9210808

4 files changed

Lines changed: 59 additions & 2 deletions

File tree

‎app/ai/providers/openai_provider.py‎

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
11
import json
22
import logging
33
import os
4-
from typing import Any, Optional
4+
import socket
5+
import sys
6+
from typing import Any, Optional, Union
57

8+
import httpx2
69
import openai
710
from openai import OpenAI
811

@@ -15,10 +18,40 @@
1518
logger = logging.getLogger("app.ai")
1619

1720

21+
def _keepalive_http_client() -> httpx2.Client:
22+
# Unlike the Anthropic SDK, the OpenAI SDK's default transport doesn't
23+
# enable TCP keepalive. Without it, a NAT gateway or load balancer that
24+
# silently drops an idle pooled connection leaves the client blocked
25+
# with no way to detect it until the outer request timeout fires,
26+
# minutes later (known upstream issue: openai/openai-python#3269;
27+
# observed live here during rapid sequential multi-place-import calls).
28+
# Mirrors the socket options the Anthropic SDK already enables by
29+
# default (see `anthropic._base_client._DefaultHttpxClient`).
30+
socket_options: list[tuple[int, int, Union[int, bool]]] = [
31+
(socket.SOL_SOCKET, socket.SO_KEEPALIVE, True)
32+
]
33+
tcp_keepintvl = getattr(socket, "TCP_KEEPINTVL", None)
34+
if tcp_keepintvl is not None:
35+
socket_options.append((socket.IPPROTO_TCP, tcp_keepintvl, 60))
36+
elif sys.platform == "darwin":
37+
tcp_keepalive = getattr(socket, "TCP_KEEPALIVE", 0x10)
38+
socket_options.append((socket.IPPROTO_TCP, tcp_keepalive, 60))
39+
tcp_keepcnt = getattr(socket, "TCP_KEEPCNT", None)
40+
if tcp_keepcnt is not None:
41+
socket_options.append((socket.IPPROTO_TCP, tcp_keepcnt, 5))
42+
tcp_keepidle = getattr(socket, "TCP_KEEPIDLE", None)
43+
if tcp_keepidle is not None:
44+
socket_options.append((socket.IPPROTO_TCP, tcp_keepidle, 60))
45+
transport = httpx2.HTTPTransport(socket_options=socket_options)
46+
return httpx2.Client(transport=transport)
47+
48+
1849
def get_client() -> OpenAI:
1950
global _client
2051
if _client is None:
21-
_client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], timeout=60.0)
52+
_client = OpenAI(
53+
api_key=os.environ["OPENAI_API_KEY"], timeout=60.0, http_client=_keepalive_http_client()
54+
)
2255
return _client
2356

2457

‎pyproject.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ dependencies = [
1313
"faster-whisper>=1.0.0",
1414
"itsdangerous>=2.2.0",
1515
"openai>=3.22.1",
16+
"httpx2>=2.13.1",
1617
]
1718

1819
[dependency-groups]

‎tests/test_openai_provider.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import json
2+
import socket
23
from types import SimpleNamespace
34

45
import openai
@@ -133,3 +134,23 @@ def test_get_client_configures_a_request_timeout(monkeypatch):
133134
client = openai_provider.get_client()
134135

135136
assert client.timeout == 60.0
137+
138+
139+
def test_get_client_enables_tcp_keepalive(monkeypatch):
140+
# Regression guard: unlike the Anthropic SDK (which enables SO_KEEPALIVE
141+
# by default, app/ai/providers/anthropic_provider.py's underlying
142+
# client), the OpenAI SDK's default transport does not -- so when a NAT
143+
# gateway or load balancer silently drops an idle pooled connection
144+
# during a burst of sequential calls (our multi-place import), the
145+
# client blocks on a dead connection with no way to detect it (observed
146+
# live, twice, during real multi-place imports). Without OS-level
147+
# keepalive probes this can only be caught by the outer request
148+
# timeout, minutes late. TCP keepalive lets the OS detect and drop the
149+
# dead connection itself, well before that.
150+
monkeypatch.setattr(openai_provider, "_client", None)
151+
monkeypatch.setenv("OPENAI_API_KEY", "sk-test-dummy")
152+
153+
client = openai_provider.get_client()
154+
155+
socket_options = client._client._transport._pool._socket_options
156+
assert (socket.SOL_SOCKET, socket.SO_KEEPALIVE, True) in socket_options

‎uv.lock‎

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)