Skip to content

fix(gateway): log websocket handshake phase#93402

Merged
vincentkoc merged 2 commits into
openclaw:mainfrom
849261680:fix/79603-gateway-ws-phase-logging
Jul 8, 2026
Merged

fix(gateway): log websocket handshake phase#93402
vincentkoc merged 2 commits into
openclaw:mainfrom
849261680:fix/79603-gateway-ws-phase-logging

Conversation

@849261680

@849261680 849261680 commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds a Gateway WebSocket handshake phase tracker so failure logs report the last completed phase instead of only the terminal outcome.
  • Emits phase=<value> on preauth handshake timeout and pre-connect close logs, with matching structured metadata.
  • Keeps the change scoped to Gateway diagnostics: no protocol, auth, config, migration, provider, or network behavior changes.
  • Uses credential-neutral auth_credentials_received and only advances that phase when a shared/device/bootstrap credential is actually present.
  • Review focus: whether the phase transitions match the existing WebSocket handshake owner boundary and whether failure-only log shape is acceptable.

Linked context

Closes #79603

Related #79654, the older closed attempt that mixed in extra surfaces and initially lacked contributor real behavior proof.

This follows the issue's latest requested narrow replacement shape: Gateway WebSocket handshake diagnostics only.

Real behavior proof (required for external PRs)

  • Behavior addressed: Gateway WebSocket handshake timeout and closed before connect logs now include the last completed phase, so operators can distinguish a connection that only reached the WebSocket upgrade/challenge stage from later auth/session phases.
  • Real environment tested: Local macOS source checkout, Node.js via node --import tsx, real node:http server, real ws WebSocketServer, real ws client, temporary empty OpenClaw config and state dir.
  • Exact steps or command run after this patch: Started a real in-process Gateway WebSocket connection handler from src/gateway/server/ws-connection.ts, configured preauthHandshakeTimeoutMs: 250, opened ws://127.0.0.1:<port>, did not send a connect request, and waited for the production preauth timer to close the socket.
  • Evidence after fix:
[repro] gateway listening on ws://127.0.0.1:52169
[repro] preauth handshake timeout = 250ms
[warn] gateway/ws: handshake timeout conn=411b0bce-066a-4be3-86c6-dd68d11ae164 peer=127.0.0.1:52170->127.0.0.1:52169 remote=127.0.0.1 phase=ws_upgrade_started
[repro] received: {"type":"event","event":"connect.challenge","payload":{"nonce":"a7f8c1a1-e76c-478a-9bc5-fab064d8af54","ts":1781549788029}}
[repro] ws opened, idling so server preauth timer fires...
[warn] gateway/ws: closed before connect conn=411b0bce-066a-4be3-86c6-dd68d11ae164 peer=127.0.0.1:52170->127.0.0.1:52169 remote=127.0.0.1 fwd=n/a origin=n/a host=127.0.0.1:52169 ua=n/a code=1000 reason=n/a phase=ws_upgrade_started {"cause":"handshake-timeout","handshake":"failed","phase":"ws_upgrade_started","durationMs":1333,"host":"127.0.0.1:52169","remoteAddr":"127.0.0.1","remotePort":52170,"localAddr":"127.0.0.1","localPort":52169,"endpoint":"127.0.0.1:52170->127.0.0.1:52169","handshakeMs":1325}
[repro] OK: phase log emitted (2 matching warning(s))
  • Observed result after fix: Both runtime warn paths contain phase=ws_upgrade_started, and the structured close context includes "phase":"ws_upgrade_started".
  • What was not tested: Windows 11 runtime from the original report; the changed logic is platform-independent TypeScript in the Gateway WebSocket handler.
  • Proof limitations or environment constraints: The focused runtime proof exercises the reported preauth timeout path with loopback networking, not a packaged npm install or deployed Gateway service.
  • Before evidence (optional but encouraged): Current main did not carry phase in the timeout or pre-connect close contexts; [Bug]: [Feature]: Add handshake phase logging to gateway/ws subsystem #79603 and the ClawSweeper issue review identify that source-level gap.

Tests and validation

  • git diff --check — passed after rebase conflict resolution.
  • node scripts/run-vitest.mjs src/gateway/server/ws-connection.test.ts — passed, Gateway shard selected 4 files / 44 tests.
  • node scripts/run-vitest.mjs src/gateway/server/ws-connection/message-handler.post-connect-health.test.ts — passed, Gateway shard selected 4 files / 92 tests.
  • node_modules/.bin/oxfmt --check src/gateway/server/ws-connection.test.ts src/gateway/server/ws-connection.ts src/gateway/server/ws-connection/message-handler.post-connect-health.test.ts src/gateway/server/ws-connection/message-handler.ts src/gateway/server/ws-types.ts — passed.
  • PNPM_CONFIG_VERIFY_DEPS_BEFORE_RUN=false corepack pnpm tsgo:core — passed.
  • .agents/skills/autoreview/scripts/autoreview --mode branch --base upstream/main — clean, no accepted/actionable findings.
  • Runtime repro command described above — previously passed and emitted phase=ws_upgrade_started in both warn lines.

Note: a combined two-file local Vitest invocation hit the harness timeout before useful output, so the same two directly relevant files were rerun separately and both passed.

Regression coverage added in src/gateway/server/ws-connection.test.ts for monotonic phase advancement, pre-connect close log metadata, timeout log metadata, and omitting phase metadata after a connection reaches ready.

Known validation limitation: the full CI matrix is left to GitHub after this push; local proof stayed focused on the touched Gateway surface.

Risk checklist

Did user-visible behavior change? Yes — failure logs gain a low-cardinality phase token and structured field.

Did config, environment, or migration behavior change? No

Did security, auth, secrets, network, or tool execution behavior change? No

What is the highest-risk area? Misleading diagnostics if phase advancement does not match the actual handshake path.

How is that risk mitigated? Phase values are a closed enum, advancement is monotonic, credential receipt only advances when credentials are actually present, and tests lock the timeout/pre-connect/ready cases.

Current review state

Next action: waiting for CI, ClawSweeper review, and maintainer review.

Author-side work complete for the narrow fix. The earlier autoreview finding about auth_token_received being misleading was addressed by switching to auth_credentials_received and gating it on actual credential presence.

@openclaw-barnacle openclaw-barnacle Bot added gateway Gateway runtime size: S proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 15, 2026
@clawsweeper

clawsweeper Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 7, 2026, 11:25 PM ET / 03:25 UTC.

Summary
The branch adds closed Gateway WebSocket handshake phase tracking and emits the last completed phase in timeout and pre-connect-close diagnostics with focused regression tests.

PR surface: Source +39, Tests +116. Total +155 across 5 files.

Reproducibility: yes. by source inspection: current main's timeout and pre-connect close paths omit phase metadata, while the PR body includes after-fix live output from a real local Node http/ws server and client. I did not rerun the runtime repro in this read-only review.

Review metrics: none identified.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #79603
Summary: This PR is the active focused fix candidate for the canonical Gateway WebSocket handshake phase logging issue; older same-goal PRs are closed unmerged, and adjacent Gateway diagnostics asks remain distinct.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

  • [P1] Exact-head CI still had pending jobs during this read-only review, so merge should wait for required checks to finish green.

Maintainer options:

  1. Decide the mitigation before merge
    Land this focused Gateway diagnostics change after maintainer review and exact-head checks pass, then let the linked canonical issue close through the merge.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • No ClawSweeper repair job is indicated because no discrete patch defect was found; maintainers should review the focused Gateway change and wait for exact-head CI to finish.

Security
Cleared: The diff adds credential-neutral diagnostic phase strings and tests only; it does not add dependencies, permissions, secrets handling, network behavior, or auth decision changes.

Review details

Best possible solution:

Land this focused Gateway diagnostics change after maintainer review and exact-head checks pass, then let the linked canonical issue close through the merge.

Do we have a high-confidence way to reproduce the issue?

Yes, by source inspection: current main's timeout and pre-connect close paths omit phase metadata, while the PR body includes after-fix live output from a real local Node http/ws server and client. I did not rerun the runtime repro in this read-only review.

Is this the best way to solve the issue?

Yes. Tracking a closed monotonic phase in the existing Gateway WebSocket connection and message-handler lifecycle is narrower than adding protocol, config, client, or provider surface, and prior review feedback corrected the credential-phase placement and phase naming.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 082bd4513088.

Label changes

Label justifications:

  • P3: This is a low-blast-radius Gateway observability improvement with no config, protocol, data migration, availability, or security-boundary change found.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied after-fix live output from a real local Node http/ws Gateway server and client path showing phase=ws_upgrade_started in both warning paths and structured close metadata.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied after-fix live output from a real local Node http/ws Gateway server and client path showing phase=ws_upgrade_started in both warning paths and structured close metadata.
Evidence reviewed

PR surface:

Source +39, Tests +116. Total +155 across 5 files.

View PR surface stats
Area Files Added Removed Net
Source 3 43 4 +39
Tests 2 117 1 +116
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 5 160 5 +155

What I checked:

Likely related people:

  • steipete: GitHub path history shows repeated recent work on Gateway WebSocket connection handling, startup retry noise, terminal/WebSocket lifecycle, and Gateway connection helper docs near this surface. (role: frequent Gateway WebSocket diagnostics contributor; confidence: high; commits: 5938c7809ad2, f083f35ddb46, 4252f07ff0f1; files: src/gateway/server/ws-connection.ts, src/gateway/server/ws-connection/message-handler.ts)
  • vincentkoc: The PR is currently assigned to vincentkoc, and path history shows recent Gateway auth/security-event and WebSocket slow-consumer work adjacent to the changed lifecycle. (role: recent Gateway auth contributor and current assignee; confidence: high; commits: 157da3621a7e, 299d31c56ef1, 16865d58ca1f; files: src/gateway/server/ws-connection.ts, src/gateway/server/ws-connection/message-handler.ts)
  • fuller-stack-dev: Recent path history shows node reapproval diagnostics and connect-error narrowing work in the same Gateway WebSocket connection/message-handler area. (role: adjacent Gateway diagnostics contributor; confidence: medium; commits: 65b460f2345a; files: src/gateway/server/ws-connection.ts, src/gateway/server/ws-connection/message-handler.ts)
  • youngting520: The PR review discussion shows focused feedback on credential-phase placement and phase naming, followed by confirmation that the revised phase semantics looked good. (role: reviewer of phase semantics; confidence: medium; files: src/gateway/server/ws-connection/message-handler.ts, src/gateway/server/ws-types.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (1 earlier review cycle)
  • reviewed 2026-07-02T03:56:03.953Z sha d3f16bf :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jun 15, 2026
deviceTokenCandidateSource,
} = connectAuthState;
if (hasSharedAuth || bootstrapTokenCandidate || deviceTokenCandidate || device) {
advanceHandshakePhase("auth_credentials_received");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small placement question: this runs after resolveConnectAuthState(...), but that helper already performs async auth work. If the connection stalls there, the last phase would still be ws_upgrade_started even though credential material was present in the connect params. Would it be better to advance this phase immediately after observing raw auth/device material?

Comment thread src/gateway/server/ws-types.ts Outdated
"auth_credentials_received",
"auth_validated",
"session_attached",
"subscriptions_registered",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this phase name may be misleading. The current connect flow does not actually register session/message subscriptions here; those happen later through RPC methods such as sessions.subscribe and sessions.messages.subscribe.

This point seems closer to initial hello/snapshot/features preparation, so maybe initial_state_prepared or hello_payload_prepared would describe the existing flow more accurately.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Addressed in c0c91c6: moved auth_credentials_received before async auth resolution and renamed subscriptions_registered to hello_payload_prepared.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great!

@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 16, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 16, 2026
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 26, 2026
@849261680
849261680 force-pushed the fix/79603-gateway-ws-phase-logging branch from c0c91c6 to d3f16bf Compare July 2, 2026 03:42
@849261680

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. labels Jul 2, 2026
@vincentkoc vincentkoc self-assigned this Jul 8, 2026
@vincentkoc
vincentkoc force-pushed the fix/79603-gateway-ws-phase-logging branch from d3f16bf to 022581d Compare July 8, 2026 03:20
@vincentkoc

Copy link
Copy Markdown
Member

Land-ready maintainer proof for exact head 022581d9bc68f67809d3dfd98a835fc5e1b3a85c:

  • Rebased patch-identically onto current main through scripts/pr prepare-sync-head with the wrapper's exact lease; the PR is mergeable.
  • Sanitized untrusted-source proof on AWS Crabbox: provider aws, lease cbx_814c20b41e15, run run_b01c493e8f44 (portal). The focused command passed 10 test files / 148 tests across ws-connection.test.ts, message-handler.post-connect-health.test.ts, and server.preauth-hardening.test.ts on Node 24.15.0.
  • Fresh Codex gpt-5.5 high-thinking autoreview on the final branch diff: no accepted/actionable findings.
  • Exact-head hosted CI run 28915022156 completed successfully.
  • Native scripts/pr review-validate-artifacts 93402 and OPENCLAW_TESTBOX=1 scripts/pr prepare-run 93402 passed.

The original Windows/npm environment was not rerun. The changed phase tracker is platform-independent; Linux Node 24 proof and hosted cross-platform CI cover the touched behavior.

@vincentkoc
vincentkoc merged commit bf31035 into openclaw:main Jul 8, 2026
101 checks passed
@vincentkoc

Copy link
Copy Markdown
Member

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 8, 2026
* fix(gateway): log websocket handshake phase

* fix(gateway): clarify websocket handshake phases
giodl73-repo pushed a commit to giodl73-repo/openclaw that referenced this pull request Jul 8, 2026
* fix(gateway): log websocket handshake phase

* fix(gateway): clarify websocket handshake phases
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gateway Gateway runtime P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: [Feature]: Add handshake phase logging to gateway/ws subsystem

3 participants