Skip to content

Discord WebSocket: add circuit breaker / async recovery for zombie reconnect loops #14758

Description

@Stache73

Summary

When the Discord WebSocket gateway connection enters a failed-resume loop (codes 1005/1006), OpenClaw's Discord provider retries indefinitely at ~1s intervals with no backoff ceiling or circuit breaker. The gateway never self-recovers — a full process restart is the only fix.

Observed Behaviour

  1. WebSocket closes with code 1006 (abnormal closure) or 1005 (no status).
  2. Provider immediately attempts to resume/reconnect.
  3. Discord responds with "connection stalled: no HELLO received within 30000ms" or silently drops the connection.
  4. Steps 2–3 repeat in a tight loop (~1s interval), indefinitely.
  5. During this loop, all Discord message delivery is dead — the bot appears offline.
  6. The retry config (channels.discord.retry) only governs outbound REST API calls, not the WebSocket gateway connection.

Impact

  • On 2026-02-11 this caused 3.5+ hours of continuous downtime without self-recovery.
  • A second occurrence the same day lasted ~20 minutes before manual restart.
  • The only reliable fix is a full gateway restart (openclaw gateway restart or SIGUSR1).

Proposed Solution

A circuit breaker for the Discord WebSocket provider:

  1. Exponential backoff on consecutive failed resume attempts (e.g. 1s → 2s → 4s → … → 60s cap).
  2. Max retry threshold — after N consecutive failures (e.g. 10), tear down the provider and reinitialise from scratch (fresh identify, not resume).
  3. Provider-level restart without requiring a full gateway restart — isolate the WebSocket lifecycle so it can be independently recycled.
  4. Optionally emit a system event / webhook when the circuit breaker trips, so users can hook into it for alerting.

Related

Environment

  • OpenClaw: latest (as of 2026-02-12)
  • OS: Windows 10 x64
  • Node: v24.6.0
  • Discord provider: Carbon library

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions