Skip to content

fix(jsonrpc): suppress busy responses for websocket notifications - #4199

Merged
rodrodros merged 9 commits into
NethermindEth:mainfrom
arunimshukla:arunimshukla/fix-ws-busy-notifications
Nov 3, 2026
Merged

rodrodros merged 9 commits into
NethermindEth:mainfrom
arunimshukla:arunimshukla/fix-ws-busy-notifications

Conversation

@arunimshukla

@arunimshukla arunimshukla commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

When the WebSocket admission gate is full, a valid single-request notification receives a -32004 error with id: null. Rejection happens before normal dispatch, so notification response suppression is never reached.

Decode one complete JSON value with the current JSON v2 decoder and validate the request shape before treating an absent ID as a notification. Return nil for those notifications and skip the WebSocket write. Requests with IDs and malformed input continue receiving busy errors. Batch admission behaviour is unchanged.

The regression test fills the gate through its public API and uses a real WebSocket connection. An identified probe sent after each notification checks that no unsolicited reply was emitted, without sleeps. Twelve cases cover notifications, unknown methods, invalid method arguments, trailing whitespace, string and large numeric IDs, explicit null IDs, invalid IDs, malformed request shapes, trailing garbage, and a second JSON value.

Review changes

  • Merge current main (765b182f3e017277403a87a0851517529254d582) and resolve the conflict using json.Unmarshal(body, &req, decodeOptions); the old decoder helper was removed upstream.
  • Document that a nil busyResponse means no reply must be sent.
  • Check the error returned by conn.CloseNow() in test cleanup, fixing the reproduced errcheck failure.
  • Remove the handler counter and assertion as requested.

Validation

Go 1.27.0; golangci-lint 2.15.0:

  • Run the exact expanded regression test against unmodified current-main production code: all four notification cases fail; all eight response/control cases pass.
  • go test -short ./jsonrpc -run '^TestWebsocketGateNotificationSemantics$' -count=1 -v — all 12 cases pass with the fix.
  • go test -race -short ./jsonrpc -count=1 — passes.
  • go vet ./jsonrpc — passes.
  • golangci-lint run ./jsonrpc/... — zero issues.
  • golangci-lint run --config=.golangci_diff.yaml --new-from-rev=origin/main ./jsonrpc/... — zero issues.
  • golangci-lint fmt --diff ./jsonrpc/... — no formatting changes.
  • git diff --check origin/main — passes.

AI assistance: prepared and tested with Codex on behalf of @arunimshukla.

Validate a single complete request before identifying notifications on the
WebSocket admission rejection path, and skip writes for valid notifications.
Add transport regression coverage for notification errors, request IDs, and
malformed input while the shared request gate is full.

Signed-off-by: Arunim Shukla <[email protected]>
@arunimshukla
arunimshukla force-pushed the arunimshukla/fix-ws-busy-notifications branch from 97169f0 to 49efc3f Compare October 19, 2026 02:13

@brbrr brbrr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this fix.

Before we can merge it, though:

  1. There is a merge conflict that needs to be addressed.
  2. Lint error in tests

Also, see the small comments below.

Comment thread jsonrpc/websocket.go Resolved
Comment thread jsonrpc/websocket_test.go Resolved
@arunimshukla
arunimshukla force-pushed the arunimshukla/fix-ws-busy-notifications branch from 731ba2e to 49efc3f Compare October 23, 2026 13:35
Check WebSocket cleanup errors, remove the redundant handler counter, and expand notification and malformed-input regression coverage to 12 cases.

Validated with Go 1.27.0: regression tests, the JSON-RPC race suite, go vet, both lint configurations, and formatting checks pass.

Signed-off-by: Arunim Shukla <[email protected]>
Comment thread jsonrpc/websocket.go Resolved

@brbrr brbrr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one last thing is left - properly handle the case of batched notifications.

Recognise non-empty batches containing only sane requests without IDs before returning a WebSocket admission error. Preserve existing busy responses for mixed, empty and malformed batches.

Expand the public WebSocket regression from 12 to 30 cases. The five new notification-batch cases fail on the previous PR head; all 30 pass with this fix. The JSON-RPC race suite, go vet, both lint configurations and formatting checks pass with Go 1.27.0.

Signed-off-by: Arunim Shukla <[email protected]>

@brbrr brbrr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, the notification-only batch case works now. Two issues remain:

  1. The busy path does not apply the batch limits. The normal path rejects a batch when batches are disabled or when the batch is larger than maxBatchElements. This branch skips both checks. Thus, when the gate is full, a notification-only batch that is too large gets no reply. When the gate is free, the same batch gets InvalidRequest. Please return serverBusyResponse in these two cases.

  2. The busy path now decodes the full batch. json.Unmarshal into []Request decodes and copies every element before the check. A 10,000-element batch now takes 3.2 ms and 4.5 MB to reject (before: 0.3 µs). This occurs when the node is already saturated. Please decode one element at a time with a jsontext.Decoder, and stop at the first element that has an ID or is not valid.

Respect DisableBatchRequests and WithMaxBatchElements before suppressing notification-only batches. Read one element at a time with jsontext.Decoder, stop at the first identified or invalid request, and validate the closing delimiter and EOF before returning no response.

Use an aliased bytes.Buffer and the pooled request decoder rather than materialising []Request or allocating a string cache for each busy batch.

Add batch-policy boundary cases and compare them with normal dispatch through public APIs. Three of 37 WebSocket cases fail on the previous PR head; all 37 pass with this correction. The full JSON-RPC race suite, vet, both lint configurations and formatting checks pass with Go 1.27.0 and golangci-lint 2.15.0.

Public WebSocket benchmarks (100 iterations, three repeats): the 10,000-element first-ID case has median 4.06 ms / 4.30 MB before and 0.53 ms / 0.72 MB after, including transport costs. Notification-only and small batches are also benchmarked. Small batches retain about 0.3-0.4 KB of additional streaming-decoder overhead; no constant-time claim is made for batches containing only notifications.

Signed-off-by: Arunim Shukla <[email protected]>
@codecov

codecov Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.59459% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.17%. Comparing base (0f3f21d) to head (f3467de).

Files with missing lines Patch % Lines
jsonrpc/websocket.go 94.59% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4199      +/-   ##
==========================================
- Coverage   81.27%   81.17%   -0.10%     
==========================================
  Files         485      485              
  Lines       35667    35701      +34     
==========================================
- Hits        28988    28980       -8     
- Misses       6673     6715      +42     
  Partials        6        6              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@rodrodros
rodrodros merged commit 4a498bf into NethermindEth:main Nov 3, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants