fix(gateway): preserve runtime-backed health state#39921
Conversation
openclaw-barnacle
Bot
added
channel: discord
Greptile SummaryThis PR fixes three related issues affecting live gateway health diagnostics: Key changes:
Issue found:
Confidence Score: 2/5
Last reviewed commit: 6aca363 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| setHealthCache(snap); | ||
| void refreshHealthSnapshot({ probe: false }).catch((err) => | ||
| logHealth.error(`background health refresh failed: ${formatError(err)}`), | ||
| ); |
There was a problem hiding this comment.
Background refresh overwrites runtime-backed cache
After computing a fresh runtime-backed snapshot and storing it via setHealthCache(snap), the immediately-triggered background refreshHealthSnapshot will overwrite that cache using getHealthSnapshot({ probe: false }) without a runtimeSnapshot. This means:
setHealthCache(snap)broadcasts the correct runtime-backed snapshot to WebSocket subscribers.- Moments later,
refreshGatewayHealthSnapshotcompletes, callssetHealthCache(stale_snap), and broadcasts a second update that lacks all the runtime fields (running,lastStartAt,connected, etc.) — undoing the fix for cache/broadcast consumers.
The RPC response itself is fine (it uses the locally-computed snap), but any subscribers relying on broadcastHealthUpdate (e.g. WebChat) will receive two rapid updates and end up with the stale, runtime-field-free snapshot.
Consider either dropping the background refresh here (the cache was just refreshed with better data, and the periodic health monitor will keep it up to date) or passing the runtime snapshot through:
setHealthCache(snap);
// Background refresh is intentionally skipped here: cache was just populated
// with runtime-backed data. The periodic health monitor handles background updates.
respond(true, snap, undefined);Prompt To Fix With AI
This is a comment left during a code review.
Path: src/gateway/server-methods/health.ts
Line: 20-23
Comment:
**Background refresh overwrites runtime-backed cache**
After computing a fresh runtime-backed snapshot and storing it via `setHealthCache(snap)`, the immediately-triggered background `refreshHealthSnapshot` will overwrite that cache using `getHealthSnapshot({ probe: false })` **without** a `runtimeSnapshot`. This means:
1. `setHealthCache(snap)` broadcasts the correct runtime-backed snapshot to WebSocket subscribers.
2. Moments later, `refreshGatewayHealthSnapshot` completes, calls `setHealthCache(stale_snap)`, and broadcasts a second update that lacks all the runtime fields (`running`, `lastStartAt`, `connected`, etc.) — undoing the fix for cache/broadcast consumers.
The RPC response itself is fine (it uses the locally-computed `snap`), but any subscribers relying on `broadcastHealthUpdate` (e.g. WebChat) will receive two rapid updates and end up with the stale, runtime-field-free snapshot.
Consider either dropping the background refresh here (the cache was just refreshed with better data, and the periodic health monitor will keep it up to date) or passing the runtime snapshot through:
```ts
setHealthCache(snap);
// Background refresh is intentionally skipped here: cache was just populated
// with runtime-backed data. The periodic health monitor handles background updates.
respond(true, snap, undefined);
```
How can I resolve this? If you propose a fix, please make it concise.
|
Follow-up pushed in Addressed the two failing checks and the review note:
Local verification:
|
|
Follow-up pushed in 8746e7d. Addressed the two failing checks and the review note:
Local verification:
Checking formatting... All matched files use the correct format.
Found 0 warnings and 0 errors.
OK: bundled plugin source files use scoped plugin-sdk subpaths (763 checked).
OK apps/macos/Sources/OpenClaw/HostEnvSecurityPolicy.generated.swiftRUN v4.0.18 /home/wolfstain/tmp/openclaw-upstream-fix ✓ src/discord/monitor/provider.test.ts (15 tests) 210ms Test Files 2 passed (2) |
8746e7d to
bb4ddf5
Compare
|
This pull request has been automatically marked as stale due to inactivity. |
|
Codex automated review: keeping this open. Keep open. Current main at e672b61 still does not implement PR #39921's central health-runtime behavior, and the Discord startup connected-state race remains at least untested and not resolved in the same way. Related open PRs #42586 and #52770 overlap the health work but do not clearly supersede the whole PR. Best possible solution: Keep #39921 open or consolidate it into a maintainer-chosen canonical gateway health/runtime fix. The product path should thread ChannelRuntimeSnapshot into health snapshot construction and cache/broadcast refreshes, make probe intent explicit, preserve runtime-backed channel/account fields, settle the Discord successful-login connected-state race with regression coverage, and coordinate with #42586/#52770 if those become the canonical health branch. What I checked:
Remaining risk / open question:
Codex Review notes: model gpt-5.5, reasoning high; reviewed against e672b61417af. |
2 participants
Summary
getHealthSnapshotgateway.healthfrom a fresh snapshot that overlayscontext.getRuntimeSnapshot()before updating the cache/broadcastProblem
On a live
2026.3.7npm install with a local gateway and 6 Discord bots, the runtime was healthy but the diagnostics were not:openclaw doctorcould reportGateway not runningopenclaw gateway health --jsoncould drop live runtime state and fall back to stale/incomplete channel dataopenclaw channels status --jsoncould leave Discord accounts asconnected:falseeven after successful loginThe root causes were:
getHealthSnapshot()implicitly probing by default instead of treating probe as opt-ingateway.isConnected === true, which races with provider loginWhat changed
getHealthSnapshot()now treatsprobeas explicit opt-in and accepts an optionalruntimeSnapshotbuildChannelAccountSnapshot(...)so runtime-backed fields survive refreshesgateway.healthnow computes a fresh snapshot usingcontext.getRuntimeSnapshot(), updates the shared cache, and still kicks off the background refreshVerification
pnpm exec vitest run src/commands/health.snapshot.test.ts src/discord/monitor/provider.test.tsopenclaw doctorstopped reportingGateway not runningopenclaw gateway health --jsonreflected running Discord/Telegram channelsopenclaw channels status --jsonshowed all 6 Discord bots asrunning:trueandconnected:trueRelated