Skip to content

fix(slack): pre-set shuttingDown before app.stop() to prevent orphaned ping intervals#56646

Merged
frankekn merged 2 commits into
openclaw:mainfrom
hsiaoa:fix/slack-socket-orphaned-ping
Apr 4, 2026
Merged

fix(slack): pre-set shuttingDown before app.stop() to prevent orphaned ping intervals#56646
frankekn merged 2 commits into
openclaw:mainfrom
hsiaoa:fix/slack-socket-orphaned-ping

Conversation

@hsiaoa

@hsiaoa hsiaoa commented Mar 28, 2026

Copy link
Copy Markdown
Contributor

Fixes #56508

Problem

When the health monitor restarts a stale Slack socket-mode connection, a race condition in @slack/[email protected] can leave orphaned ping intervals running indefinitely:

  1. Health monitor detects stale socket → calls stopChannel() → aborts → app.stop()
  2. app.stop()SocketModeClient.disconnect() sets shuttingDown = true
  3. Race: the library's internal ping interval fires SlackWebSocket.disconnect()terminate() → emits close BEFORE step 2 sets shuttingDown
  4. The close handler sees shuttingDown === false → schedules delayReconnectAttempt(this.start)
  5. Orphaned connection's ping interval keeps firing, logging "A pong wasn't received..." every ~1.6s and leaking file descriptors

Impact: gateway.err.log grows unbounded (~3 warnings/1.6s per account), FD accumulation over time.

Fix

Introduce gracefulStop() that sets shuttingDown = true on the SocketModeClient BEFORE calling app.stop(), closing the race window. Applied to all three stop paths:

  • stopOnAbort handler (health monitor restart)
  • Reconnect loop (disconnect recovery)
  • finally block (provider teardown)

Why not fix upstream?

@slack/[email protected] has no fix for this (only cosmetic async/await changes). The proper upstream fix would be to clear ping intervals in SocketModeClient.disconnect() before calling websocket.disconnect(), but that requires a library release. This workaround is safe because shuttingDown is a simple boolean sentinel — setting it early just makes the close handler skip the reconnect, which is exactly what we want.

Testing

  • Verified on macOS with 1 Slack account in socket mode
  • Health monitor stale-socket restarts no longer produce orphaned pong warnings
  • gateway.err.log stays clean after restart cycles

AI Disclosure

🤖 AI-assisted (Kiro CLI). Root cause analysis from issue #56508, fix verified against @slack/socket-mode source.

@openclaw-barnacle openclaw-barnacle Bot added channel: slack Channel integration: slack size: XS labels Mar 28, 2026
@greptile-apps

greptile-apps Bot commented Mar 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR introduces gracefulStop() to close a race condition in @slack/[email protected] where the library's internal ping timeout can fire disconnect() before SocketModeClient.shuttingDown is set during teardown — causing the close handler to schedule a reconnect attempt and leak ping intervals indefinitely.

The fix is minimal and targeted:

  • gracefulStop() accesses the receiver's client via a loosely-typed cast and sets shuttingDown = true before delegating to app.stop()
  • The optional-chaining guard (?.client + if (socketClient)) ensures the mutation is silently skipped in HTTP mode (where HTTPReceiver has no client property), so there is no regression for non-socket deployments
  • All three teardown paths (stopOnAbort, the reconnect loop, and the finally block) now use gracefulStop() consistently

The approach correctly relies on SocketModeClient.start() resetting shuttingDown on subsequent reconnect attempts (consistent with pre-existing behavior where app.stop()app.start() cycles already worked), and is appropriately scoped as a workaround pending an upstream library fix.

Confidence Score: 5/5

Safe to merge — the fix is a well-scoped workaround for a confirmed race condition, with no regressions in HTTP mode and consistent teardown behavior across all stop paths.

No P0 or P1 issues found. The private-API access via type cast is intentional and guarded by optional chaining; it degrades silently if the library restructures its internals rather than breaking anything. The .catch(() => undefined) in gracefulStop() preserves the pre-existing error-suppression behavior at all call sites. Testing confirmed reconnect cycles still function correctly, validating that SocketModeClient.start() properly handles a pre-set shuttingDown flag.

No files require special attention.

Important Files Changed

Filename Overview
extensions/slack/src/monitor/provider.ts Adds gracefulStop() that pre-sets SocketModeClient.shuttingDown = true before calling app.stop(), closing a race window that caused orphaned ping intervals. Applied consistently to all three stop paths.

Reviews (1): Last reviewed commit: "fix(slack): pre-set shuttingDown before ..." | Re-trigger Greptile

@CHE10X

CHE10X commented Apr 3, 2026

Copy link
Copy Markdown

We're hitting this exact issue in production with 11 Slack accounts configured in socket mode.

Symptoms observed:

  • 360+ pong timeouts per hour
  • 33+ socket reconnects per hour (should be ~1 at startup)
  • gateway.err.log filling with "A pong wasn't received from the server before the timeout of 5000ms!"
  • 1,400+ unique socket IDs cycling through in a single hour

Environment:

  • OpenClaw 2026.4.2
  • 11 Slack accounts (separate bots for agent context isolation)
  • macOS, socket mode

Current workaround: Increased gateway.channelHealthCheckMinutes from 60 to 180 to reduce restart frequency, but this just delays the orphan accumulation.

Would appreciate this fix landing soon — it's a significant stability issue for multi-account Slack deployments. Happy to test a beta if one becomes available.

@hsiaoa

hsiaoa commented Apr 3, 2026

Copy link
Copy Markdown
Contributor Author

There's no dedicated Slack maintainer yet. We'd appreciate it if @frankekn would help move this along.

@frankekn frankekn self-assigned this Apr 4, 2026
hsiaoa and others added 2 commits April 4, 2026 20:39
…d ping intervals

Fixes openclaw#56508

When the health monitor restarts a stale Slack socket-mode connection,
there's a race condition in @slack/[email protected]:

1. Health monitor calls stopChannel() → abort signal → app.stop()
2. app.stop() → SocketModeClient.disconnect() sets shuttingDown = true
3. But the library's internal ping interval may fire first, calling
   SlackWebSocket.disconnect() → terminate() → emits 'close'
4. The close handler checks shuttingDown (still false) and schedules
   an orphaned reconnect via delayReconnectAttempt(this.start)
5. The orphaned connection's ping interval keeps firing indefinitely,
   logging 'A pong wasn't received...' every 1.6s and leaking FDs

Fix: introduce gracefulStop() that sets shuttingDown = true on the
SocketModeClient BEFORE calling app.stop(), closing the race window.
Applied to all three stop paths: abort handler, reconnect loop, and
finally block.

Single file change. No new dependencies.
@frankekn
frankekn force-pushed the fix/slack-socket-orphaned-ping branch from b6c3b14 to f1c91d5 Compare April 4, 2026 12:41
@frankekn
frankekn merged commit ae16452 into openclaw:main Apr 4, 2026
24 of 32 checks passed
@frankekn

frankekn commented Apr 4, 2026

Copy link
Copy Markdown
Contributor

Merged. Thanks @hsiaoa
Head: f1c91d5

lovewanwan pushed a commit to lovewanwan/openclaw that referenced this pull request Apr 28, 2026
…d ping intervals (openclaw#56646)

Merged via squash with admin override.

Prepared head SHA: f1c91d5
Note: required red lanes are currently inherited from latest origin/main, not introduced by this PR.
Co-authored-by: hsiaoa <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
ogt-redknie pushed a commit to ogt-redknie/OPENX that referenced this pull request May 2, 2026
…d ping intervals (openclaw#56646)

Merged via squash with admin override.

Prepared head SHA: f1c91d5
Note: required red lanes are currently inherited from latest origin/main, not introduced by this PR.
Co-authored-by: hsiaoa <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
…d ping intervals (openclaw#56646)

Merged via squash with admin override.

Prepared head SHA: f1c91d5
Note: required red lanes are currently inherited from latest origin/main, not introduced by this PR.
Co-authored-by: hsiaoa <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
…d ping intervals (openclaw#56646)

Merged via squash with admin override.

Prepared head SHA: f1c91d5
Note: required red lanes are currently inherited from latest origin/main, not introduced by this PR.
Co-authored-by: hsiaoa <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
Nachx639 pushed a commit to Nachx639/clawdbot that referenced this pull request Jun 17, 2026
…d ping intervals (openclaw#56646)

Merged via squash with admin override.

Prepared head SHA: f1c91d5
Note: required red lanes are currently inherited from latest origin/main, not introduced by this PR.
Co-authored-by: hsiaoa <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: slack Channel integration: slack size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Slack socket-mode: orphaned ping intervals leak file descriptors and spam error log

3 participants