test(qa): exercise Gateway and MCP scenarios over real transports#99735
Conversation
|
Codex review: needs maintainer review before merge. Reviewed July 3, 2026, 9:40 PM ET / 01:40 UTC. Summary PR surface: Source +85, Tests +723, Other +17. Total +825 across 8 files. Reproducibility: not applicable. This PR is a QA coverage/evidence change, not a reported product bug. The PR body and current-head proof check provide real scenario producer evidence instead of a bug reproduction path. Review metrics: 1 noteworthy metric.
Stored data model Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Land the real-transport QA evidence path once maintainers accept the automation cost and current-head required checks remain green. Do we have a high-confidence way to reproduce the issue? Not applicable: this PR is a QA coverage/evidence change, not a reported product bug. The PR body and current-head proof check provide real scenario producer evidence instead of a bug reproduction path. Is this the best way to solve the issue? Yes: using the existing QA Gateway child fixture, script evidence writer, and MCP/Gateway transport surfaces is the right boundary for proving these scenarios without changing product runtime behavior. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against fd0355d5b1a1. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Source +85, Tests +723, Other +17. Total +825 across 8 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
861c355 to
3e14214
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
3e14214 to
e4a9694
Compare
e4a9694 to
02d017f
Compare
What Problem This Solves
The
gateway-smoke,mcp-gateway-connect-startup-retry, andmcp-plugin-tools-callQA scenarios claimed Gateway/MCP behavior from helper-level or in-memory tests instead of exercising the real process and transport boundaries.Why This Change Was Made
onListeninglifecycle hook so a client can start after the real socket binds but before Gateway readiness.gateway-smokeagainst a real Gateway child and its real WebSocket/health surfaces.memory_searchwith the MCP SDK client.User Impact
No product behavior changes. QA evidence now fails when the real Gateway or MCP process, readiness, retry, negotiation, plugin discovery, stdio transport, or tool invocation boundary regresses.
Evidence
gateway-smoke: passmcp-plugin-tools-call: passmcp-gateway-connect-startup-retry: three consecutive frozen-base runs passed; each observed 2 retryable startup-unavailable responses, 3 connect frames, protocol 4, and MCP start 7.0–9.7 seconds before Gateway readiness.node scripts/run-vitest.mjs extensions/qa-lab/src/gateway-child.test.ts extensions/qa-lab/src/scenario-catalog.test.ts test/e2e/qa-lab/runtime/gateway-smoke.e2e.test.ts src/mcp/plugin-tools-mcp-client.test.ts— 100 tests passed.node scripts/run-oxlint.mjs --tsconfig tsconfig.json extensions/qa-lab/api.ts extensions/qa-lab/src/bundled-plugin-staging.ts extensions/qa-lab/src/gateway-child.ts extensions/qa-lab/src/gateway-child.test.ts test/e2e/qa-lab/runtime/gateway-mcp-real-transports.ts— passed.tbx_01kwnaep24a90x84dwp3dmghaw,check:changedexit 0, Actions run28690164618.