Summary

Local CLI commands requiring a full authenticated operator WebSocket session (openclaw devices list, openclaw devices approve, openclaw nodes status) intermittently fail with:

• gateway closed (1000 normal closure): no close reason
• Gateway log: [ws] handshake timeout conn=... remote=127.0.0.1
• Gateway log: [ws] closed before connect ... code=1008 reason=connect challenge timeout

Retrying the same command immediately (1–2s later) almost always succeeds. Commands that do not require a full authenticated RPC session (openclaw status, openclaw gateway probe, openclaw qr --json, openclaw dashboard --no-open) are not affected.

Environment

• Version: 2026.3.13
• OS: macOS 15.7.3 (x64), Mac mini
• Install method: npm global install
• Gateway mode: local LaunchAgent, loopback bind, Tailscale serve for remote access
• Gateway load: active Discord channel, heartbeats, cron jobs, agent sessions running concurrently

Root cause analysis (from source)

I traced this to a race condition between the handshake timer windows:

Server side (gateway source, getHandshakeTimeoutMs()):

const DEFAULT_HANDSHAKE_TIMEOUT_MS = 3000; // hardcoded, not user-configurable
const handshakeTimer = setTimeout(() => {
  if (!client) {
    setCloseCause("handshake-timeout", ...);
    logWsControl.warn(`handshake timeout conn=...`);
    close();
  }
}, handshakeTimeoutMs);

Client side (gateway client source, queueConnect()):

const connectChallengeTimeoutMs = 2000; // default
this.connectTimer = setTimeout(() => {
  if (this.connectSent || this.ws?.readyState !== WebSocket.OPEN) return;
  this.ws?.close(1008, "connect challenge timeout");
}, connectChallengeTimeoutMs);

The server sends the connect.challenge nonce synchronously in the WS open handler, but only after running setup work (attachGatewayWsMessageHandler(), loadConfig(), proxy/IP header resolution, rate limiter setup, origin checks). When the gateway event loop is busy (agent model runs, Discord polling, heartbeats, cron jobs), this synchronous setup block can delay the challenge send past both the 3s server timeout and 2s client timeout.

Steps to reproduce

1. Have a busy gateway: active Discord channel, running agent sessions, heartbeat active, cron jobs running
2. Run openclaw devices list repeatedly in a loop
3. Observe first attempt frequently exits 1 with gateway closed (1000); second attempt at ~1s interval usually succeeds

Expected behavior

The connect.challenge send should be scheduled with setImmediate() or process.nextTick() to ensure it dispatches before any blocking setup work, or the server-side handshake timeout should be user-configurable to accommodate loaded gateways.

Possible fixes

1. Move the connectNonce generation and send({event:"connect.challenge"...}) call to run via setImmediate() at the start of the WS open handler, before other setup work
2. Add a gateway.handshakeTimeoutMs config option to let users extend the 3-second server-side window
3. OPENCLAW_TEST_HANDSHAKE_TIMEOUT_MS env var already exists in the source but is gated on VITEST — consider exposing it for production use

Additional context

This appears to have worsened in 2026.3.13 vs 2026.3.12. Two changes in 3.13 likely tightened event loop scheduling:

• perf(build): deduplicate plugin-sdk chunks to fix ~2x memory regression (perf(build): deduplicate plugin-sdk chunks to fix ~2x memory regression #45426) — changed internal chunk structure and memory footprint
• fix(gateway): bound unanswered client requests (fix(gateway): bound unanswered client requests #45689) — adds new timer pressure on the event loop in the RPC path

Gateway log evidence showing the race (both server and client timeouts firing):

2026-03-18T05:08:07.278-04:00 [ws] handshake timeout conn=28e60671... remote=127.0.0.1
2026-03-18T05:08:08.666-04:00 [ws] closed before connect conn=28e60671... code=1000 reason=n/a
2026-03-18T05:08:16.173-04:00 [ws] handshake timeout conn=b5dd94dc... remote=127.0.0.1
2026-03-18T05:08:16.345-04:00 [ws] closed before connect conn=b5dd94dc... code=1008 reason=connect challenge timeout