perf(ext/node): right-size small socket reads instead of pinning the 64KB slab#35779
Merged
nathanwhit merged 7 commits intoJul 15, 2026
Merged
Conversation
…64KB slab Every read on the node-compat stream path (node:net, pipes, TLS ciphertext ingest) transferred ownership of the full 65536-byte read slab into the ArrayBuffer handed to JS, regardless of nread. Buffer.from(ab, 0, nread) then kept the whole slab alive until GC, so a client that retains small chunks (protocol reassembly: tedious TDS messages, length-prefixed framing) resident-pinned ~16KB per chunk (page-rounded; 64KB virtual) — measured at 14x RSS amplification vs ~2x on Node, which right-sizes the store via BackingStore::Reallocate in EmitToJSStreamListener::OnStreamRead. Handing off every slab also kept more than POOLED_BUF_MAX slabs in flight in steady state (freed only at GC), starving the read-buffer pool so reads fell back to the system allocator — the exact mach_vm-reclaim cost the pool was added to avoid. Reads at or below READ_COPY_THRESHOLD (half a slab, 32KB) are now copied into an exact-size backing store and the slab is returned to the pool immediately; larger reads (bulk transfers filling most of the slab) keep the zero-copy slab handoff. The copy costs at most ~1us against a read dispatch that costs ~10us+. Co-authored-by: Cursor <[email protected]>
tomas-zijdemans
marked this pull request as ready for review
July 4, 2026 21:17
Member
|
Thank you for the PR, this is a really nice performance improvement and the analysis and benchmarking are very thorough! Two small asks before merging:
|
Replace the non-ASCII characters introduced by this PR with ASCII equivalents and shrink the READ_COPY_THRESHOLD doc comment to just the invariant, moving the measurements to the PR description where they belong. Co-authored-by: Cursor <[email protected]>
Contributor
Author
|
Sorry. Should be better now. |
nathanwhit
reviewed
Jul 7, 2026
nathanwhit
left a comment
Member
There was a problem hiding this comment.
I found one performance/compatibility concern around the threshold behavior.
Per review feedback, replace the READ_COPY_THRESHOLD heuristic with Node's exact condition: copy whenever nread < the slab size, so no partial read (including the previously uncovered 33-63KB band) pins the full 64KB slab, and the slab always returns to the pool immediately. Only exactly-full slab reads keep the zero-copy handoff, matching EmitToJSStreamListener::OnStreamRead. Co-authored-by: Cursor <[email protected]>
… into perf/stream-wrap-read-rightsize
nathanwhit
enabled auto-merge (squash)
July 7, 2026 19:32
nathanwhit
disabled auto-merge
July 15, 2026 09:56
nathanwhit
enabled auto-merge (squash)
July 15, 2026 09:56
bartlomieju
pushed a commit
that referenced
this pull request
Jul 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A
node:netclient that retains incoming chunks (protocol reassembly: tedious TDS messages, length-prefixed framing, etc.) pins up to 14x more memory than the data it holds, because every read hands JS an ArrayBuffer that owns the full 64 KB read slab regardless ofnread. This PR right-sizes small reads, dropping retention to Node parity:20,000 retained chunks per run, ack-paced so each write lands as its own read (as on a real network, where reads don't coalesce to full slabs); medians of 5, macOS aarch64, both binaries built from the same tree with the same profile — the baseline is this branch's base commit, so the only difference is this diff. Every run content-verified (all retained bytes checked, totals exact). The baseline's flat ~380 MiB independent of chunk size is the signature of the bug: 20,000 × ~19 KB resident (the 64 KB slab, page-rounded by the OS) no matter how few bytes each read carried. With this PR, growth tracks the actual retained data at every size, within a few MiB of Node.
Wall time on the same runs improves as a side effect of the reduced allocator/GC pressure: 912→743 ms (512 B), 919→769 ms (1400 B), 955→841 ms (4 KiB); 16 KiB unchanged within noise.
Cause
on_uv_read(ext/node/ops/stream_wrap.rs) always transferred ownership of the 65536-byte slab fromon_uv_allocinto the ArrayBuffer passed toonread, with a deleter that frees it at GC.Buffer.from(ab, 0, nread)inonStreamReadthen views onlynreadbytes, but the view keeps the whole slab alive for as long as the consumer holds the chunk. Node instead shrinks the store tonreadbefore handing it to JS (BackingStore::ReallocateinEmitToJSStreamListener::OnStreamRead,src/stream_base.cc).Beyond RSS, the handoff also defeats the slab pool this file maintains (
READ_BUF_POOL, 128 slots): slabs come back only at GC, so in steady state the pool runs dry and reads fall back to the system allocator — the exactmach_vm_reclaimoverhead (~6% CPU in the profile that motivated the pool) it was added to avoid.Fix
Reads with
nread <= READ_COPY_THRESHOLD(half a slab, 32 KB) are copied into an exact-size backing store (new_backing_store_from_vec) and the slab is returned to the pool immediately. Larger reads — bulk transfers actually filling most of the slab — keep the existing zero-copy slab handoff, so the well-behaved path is untouched. The copy is bounded at 32 KB (~1 µs) against a read dispatch that costs ~10 µs+.No regressions on the paths that were already fine
Verification
unit_node::net_test,unit_node::http_test,unit_node::tls_test— pass on this branch (TLS matters here: ciphertext ingest rides this same read path via the TLSWrap onread forwarder).specs::node(257 tests): failure set byte-identical to an upstream/main baseline build run on the same machine — the same 2 pre-existingchild_process_ipc_handle::destroy_before_ack_*flakes on both (they reproduce at ~3/30 on unpatched debug builds).tools/format.jsandtools/lint.jspass.Benchmark source (amplification)
The regression guards (bulk 1 GiB fire-and-forget writes and 512 B ping-pong) are separate scripts; happy to paste them on request.
I used Cursor (Claude) to help investigate and write this change.