Skip to content

Telegram long-poll hangs indefinitely on dead TCP socket — no socket-level read timeout #56061

Description

@micah-lundquist

Title

Telegram long-poll hangs indefinitely on dead TCP socket — no socket-level read timeout

Summary

The Telegram polling runner's getUpdates call can hang indefinitely when a TCP connection silently dies (half-open socket). The watchdog detects the stall but the recovery path itself can cascade into repeated timeout failures.

Environment

  • OpenClaw 2026.3.13 (61d171a)
  • macOS, Node 25.5
  • Telegram long polling mode (not webhook)
  • Running 24/7 via launchd

Observed behavior

From gateway logs over several days:

~90-900s stalls: The poller hangs with no getUpdates completing. The watchdog fires at 90s (POLL_STALL_THRESHOLD_MS), but some stalls reach 900s+ before recovery.

Cascading restart failures (Mar 19-21): After a stall, runner.stop() itself times out after 15s (POLL_STOP_GRACE_MS), the forceCyclePromise fires, and the cycle restarts — only to stall again immediately. This repeats for hours with backoff ramping to 30s.

Network-layer errors (Mar 24-25): ENETUNREACH and ETIMEDOUT errors on IPv6, forcing fallback to IPv4-only. These compound the polling stalls.

Root cause analysis

In createTelegramRunnerOptions():

runner: {
  fetch: {
    timeout: 30,  // Telegram API hold time, NOT a socket read timeout
    allowed_updates: resolveTelegramAllowedUpdates()
  },
  silent: true,
  maxRetryTime: 3600 * 1e3,
  retryInterval: "exponential"
}

fetch.timeout: 30 tells Telegram's API to hold the long-poll connection for 30s before returning empty. But this is NOT a socket-level read timeout on the HTTP request itself. When TCP goes half-open (remote end dies without FIN/RST), Node's fetch/http client waits indefinitely on the dead socket.

The watchdog (POLL_WATCHDOG_INTERVAL_MS = 30s, POLL_STALL_THRESHOLD_MS = 90s) catches this, but:

  1. The stall window is still 90s of being blind to incoming messages
  2. runner.stop() tries to abort the in-flight request via #activeFetchAbort, but the dead socket doesn't respond to abort signals promptly
  3. The stop itself times out at 15s, leading to forced restart cycles

Suggested fix

Add a socket-level read timeout to the HTTP request that calls getUpdates. This should be slightly longer than the Telegram API timeout (e.g., 45s for a 30s long-poll) so it fires only when the connection is actually dead, not during normal long-poll holds.

Options:

  1. grammY transformer/adapter: Add a signal with AbortSignal.timeout(45_000) to the fetch call via bot.api.config.use() — though note that AbortSignal.timeout() has known issues in Node 25.5 (may need socket-level req.setTimeout() instead)
  2. Custom fetcher: Replace grammY's default fetcher with one that uses http.request with an explicit timeout option, similar to what helios-bridge does for its HTTP calls (we hit the same Node 25.5 abort hang there)
  3. Watchdog tightening: Reduce POLL_STALL_THRESHOLD_MS to 45-60s as a band-aid, but this doesn't fix the underlying dead-socket problem

The cleanest fix is option 2 — a custom fetcher with http.request and socket-level timeout, which guarantees the request dies when the connection is dead regardless of Node's fetch/AbortSignal behavior.

Impact

During each stall cycle, the gateway is blind to all incoming Telegram messages for 90s-15min. The watchdog is self-healing but the blind spots are significant for real-time conversation use cases.

Related

Node 25.5 has a known issue where fetch() + AbortSignal.timeout() hangs indefinitely — the timeout fires but the connection is never actually destroyed. This may be contributing to why #activeFetchAbort.abort() doesn't cleanly kill the stalled getUpdates call during recovery.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions