[Bug]: FailOpen MCP group fanout sends two JSON-RPC responses (error + result) for the same request id when an upstream returns a malformed response #2421

Component: MCP proxy / multiplexed (group) backend fanout
Version: v1.3.0-beta.1 (serverInfo.name=agentgateway, version=1.3.0-beta.1)
Related but distinct: #2373 (that one is about POST responses duplicated onto the GET SSE stream; this one is two responses for the same request id on the same POST response stream)

Summary

When an AgentgatewayBackend with multiple MCP targets (spec.mcp.targets, failureMode: FailOpen) handles initialize, and one of the upstream targets fails mid-handshake (in our case the upstream process crashed and returned a malformed/empty response), agentgateway emits two JSON-RPC responses for the same request id in the same SSE response stream: first the failed upstream's error frame, then the aggregated success result.

data: {"jsonrpc":"2.0","id":1,"error":{"code":-32001,"message":"JSON RPC unknown error: failed to unmarshal response: no result in response"}}

data: {"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-03-26","capabilities":{...},"serverInfo":{"name":"agentgateway","version":"1.3.0-beta.1"},...}}

JSON-RPC 2.0 requires exactly one response per request id. Spec-compliant MCP clients (e.g. @modelcontextprotocol/sdk StreamableHTTPClientTransport, as used by Claude Code and VS Code) latch onto the first frame, treat the initialize as failed, and mark the whole server as disconnected — even though the aggregated result that follows is valid. Net effect: one flaky target in a group takes down the entire group for compliant clients, which defeats the purpose of FailOpen.

Expected behavior under FailOpen: the failed target is skipped (as the log line upstream '<name>' failed during fanout, skipping suggests is intended) and only the single merged result is sent. The upstream's error frame should never be forwarded to the client once the gateway has decided to fail open, and certainly not alongside a second response with the same id.

Reproduction

1. Create a multiplexed MCP backend with ≥2 static StreamableHTTP targets, failureMode: FailOpen.
2. Make one target unhealthy in a way that produces a malformed response rather than a connection refusal — e.g. a stdio-wrapped server whose child process exits after the HTTP layer accepts the request (we hit this with a nanobot-wrapped npx server whose install cache was corrupted; the wrapper returned a response with no result).
3. POST an initialize to the group route:

curl -N -X POST http://<gateway>/<group>/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"debug","version":"1.0"}}}'

4. Observe two data: frames both answering id:1 — an error, then a result.

Gateway logs at the same moment show the fanout skip path being taken for other failure shapes (HTTP 4xx/5xx are skipped cleanly and only logged):

warn mcp::handler  upstream 'metabase' failed during fanout, skipping: http upstream error: http request failed with code: 405 Method Not Allowed

…so it appears the "response received but unmarshal failed" path forwards the error frame to the client before the merge/skip logic runs, while transport-level failures are correctly swallowed.

Environment

• agentgateway v1.3.0-beta.1 (Helm, kube backend), Kubernetes 1.32
• Backend: AgentgatewayBackend with 5 static StreamableHTTP targets, failureMode: FailOpen
• Clients affected: Claude Code, VS Code MCP, LibreChat (all report the server as failed/not authenticated on the error frame)