Skip to content

test: stabilize gateway server shard#77131

Merged
amknight merged 1 commit into
mainfrom
ak/fix-gateway-config-flake
May 4, 2026
Merged

test: stabilize gateway server shard#77131
amknight merged 1 commit into
mainfrom
ak/fix-gateway-config-flake

Conversation

@amknight

@amknight amknight commented May 4, 2026

Copy link
Copy Markdown
Member

Summary

  • isolate the gateway-server Vitest config so suite-scoped Gateway server/env/logger state cannot leak across parallel test files
  • make request-trace coverage look for the traced request log record instead of assuming it is the first line in the log file
  • add bounded websocket open/close waits in shared-auth helpers and close test-created Undici dispatchers in the network-runtime e2e

Validation

  • pnpm exec oxfmt --check --threads=1 test/vitest/vitest.gateway-server.config.ts src/gateway/server-http.request-trace.test.ts src/gateway/shared-auth.test-helpers.ts src/gateway/server-network-runtime.e2e.test.ts CHANGELOG.md
  • OPENCLAW_VITEST_MAX_WORKERS=2 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=60000 pnpm exec node scripts/run-vitest.mjs run --config test/vitest/vitest.gateway-server.config.ts --reporter=dot
  • pnpm check:changed
  • pnpm test:changed

Notes

Reproduced two failure classes locally before the fix: request-trace assertions reading unrelated startup logs, and a gateway-server no-output timeout under non-isolated parallel file execution. The exact CI-side server.config-patch.test.ts assertion failure did not reproduce on latest main, but it matched the same shared process-state pattern.

@openclaw-barnacle openclaw-barnacle Bot added gateway Gateway runtime size: S maintainer Maintainer-authored PR labels May 4, 2026
@clawsweeper

clawsweeper Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge.

Summary
The PR isolates the gateway-server Vitest shard, makes request-trace coverage find the target log record, adds bounded shared-auth websocket waits, closes gateway network-runtime dispatchers, and adds a changelog entry.

Reproducibility: Do we have a high-confidence way to reproduce the issue? Source-reproducible, but not live-reproduced in this read-only pass: current main shows the non-isolated shard and first-line log assumption, and upstream ws source confirms the PR's timeout path can emit an unhandled error.

Next step before merge
A narrow automated repair can keep or drain an error listener around the ws.close() timeout path; the maintainer label still leaves merge approval human-gated.

Security
Cleared: The diff is limited to tests and CHANGELOG.md, with no dependency, workflow, permission, secret-handling, or production execution-path change.

Review findings

  • [P2] Drain abort errors when timing out websocket opens — src/gateway/shared-auth.test-helpers.ts:32-33
Review details

Best possible solution:

Keep the useful shard isolation and log-record lookup changes, but drain the expected ws abort error during the open-timeout close path before merge.

Do we have a high-confidence way to reproduce the issue?

Do we have a high-confidence way to reproduce the issue? Source-reproducible, but not live-reproduced in this read-only pass: current main shows the non-isolated shard and first-line log assumption, and upstream ws source confirms the PR's timeout path can emit an unhandled error.

Is this the best way to solve the issue?

Is this the best way to solve the issue? No, not as submitted. The selected stabilization surfaces are narrow, but the websocket open timeout must keep or install an error listener while closing a CONNECTING socket.

Full review comments:

  • [P2] Drain abort errors when timing out websocket opens — src/gateway/shared-auth.test-helpers.ts:32-33
    When the open timeout fires, cleanup() removes the error listener before ws.close(). In ws 8.20.0, closing a CONNECTING socket aborts the handshake and emits an error on the next tick, so this can crash the Vitest worker instead of producing the bounded timeout failure this helper is meant to add.
    Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.86

Acceptance criteria:

  • pnpm exec oxfmt --check --threads=1 src/gateway/shared-auth.test-helpers.ts
  • OPENCLAW_VITEST_MAX_WORKERS=2 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=60000 pnpm exec node scripts/run-vitest.mjs run --config test/vitest/vitest.gateway-server.config.ts --reporter=dot
  • pnpm check:changed

What I checked:

  • Protected PR state: The PR is open, ready for review, and labeled maintainer, so cleanup policy keeps it open even aside from review findings. (49c1506e8be1)
  • Gateway shard isolation source: Current main's gateway-server config does not pass isolate, while createScopedVitestConfig resolves isolation from env and uses the non-isolated runner when isolation is false. (test/vitest/vitest.gateway-server.config.ts:11, e8d0cf75ea0e)
  • Request-trace flake source: Current main parses only the first log line before asserting trace fields, so an unrelated startup record can make the test inspect the wrong JSON record. (src/gateway/server-http.request-trace.test.ts:91, e8d0cf75ea0e)
  • Introduced websocket timeout bug: The PR timeout path calls cleanup() before ws.close(), removing the error listener before aborting a CONNECTING websocket. (src/gateway/shared-auth.test-helpers.ts:32, 49c1506e8be1)
  • ws dependency behavior: ws 8.20.0 close() on a CONNECTING socket calls abortHandshake; that path schedules emitErrorAndClose, which emits error, so the PR must keep or install an error listener while closing. (49c1506e8be1)
  • Routing history: GitHub commit history for the touched paths points to Vincent Koc for recent gateway-server shard/request-trace work and Peter Steinberger for the shared-auth helper and network-runtime e2e.

Likely related people:

  • vincentkoc: Recent history shows Vincent Koc authored gateway-server shard stabilization and the request trace propagation test area affected by this PR. (role: recent gateway-server shard and request-trace maintainer; confidence: high; commits: 833a42c25338, 3ad0ad994af2, 3ae6f01d619e; files: test/vitest/vitest.gateway-server.config.ts, test/vitest/vitest.scoped-config.ts, src/gateway/server-http.request-trace.test.ts)
  • steipete: Peter Steinberger introduced the shared-auth websocket helper, has recent shared-auth flake fixes, and authored the gateway env proxy dispatcher e2e surface. (role: shared-auth and gateway network-runtime maintainer; confidence: high; commits: ee54a8d298ce, 15f2cebc3ee2, d513dc71469f; files: src/gateway/shared-auth.test-helpers.ts, src/gateway/server.shared-auth-rotation.test.ts, src/gateway/server-network-runtime.e2e.test.ts)
  • ngutman: Nimrod Gutman has prior merged work reducing shared auth rotation teardown flakes in the same helper/test family. (role: adjacent shared-auth flake contributor; confidence: medium; commits: 6474795890be; files: src/gateway/server.shared-auth-rotation.test.ts)

Remaining risk / open question:

  • No live Vitest reproduction was run in this read-only review; the positive flake assessment rests on source inspection and the PR author's reported validation.
  • Current main has advanced past the PR base, so changed-gate proof should be refreshed after any rebase or repair.

Codex review notes: model gpt-5.5, reasoning high; reviewed against e8d0cf75ea0e.

@amknight
amknight marked this pull request as ready for review May 4, 2026 07:42
@amknight
amknight requested a review from a team as a code owner May 4, 2026 07:42
@amknight
amknight merged commit be41b8c into main May 4, 2026
151 of 158 checks passed
@amknight
amknight deleted the ak/fix-gateway-config-flake branch May 4, 2026 08:42
lxe pushed a commit to lxe/openclaw that referenced this pull request May 6, 2026
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
jameslcowan pushed a commit to jameslcowan/openclaw that referenced this pull request Jun 2, 2026
sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gateway Gateway runtime maintainer Maintainer-authored PR size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant