Skip to content

perf(ext/node): right-size small socket reads instead of pinning the 64KB slab#35779

Merged
nathanwhit merged 7 commits into
denoland:mainfrom
tomas-zijdemans:perf/stream-wrap-read-rightsize
Jul 15, 2026
Merged

perf(ext/node): right-size small socket reads instead of pinning the 64KB slab#35779
nathanwhit merged 7 commits into
denoland:mainfrom
tomas-zijdemans:perf/stream-wrap-read-rightsize

Conversation

@tomas-zijdemans

Copy link
Copy Markdown
Contributor

A node:net client that retains incoming chunks (protocol reassembly: tedious TDS messages, length-prefixed framing, etc.) pins up to 14x more memory than the data it holds, because every read hands JS an ArrayBuffer that owns the full 64 KB read slab regardless of nread. This PR right-sizes small reads, dropping retention to Node parity:

retained chunk size retained data baseline RSS growth this PR Node 26.3
512 B 9.8 MiB 391 MiB 32 MiB (12.2x less) 29 MiB
1400 B 26.7 MiB 377 MiB 53 MiB (7.1x less) 51 MiB
4 KiB 78.1 MiB 382 MiB 100 MiB (3.8x less) 98 MiB
16 KiB 312.5 MiB 375 MiB 333 MiB (1.1x less) 330 MiB

20,000 retained chunks per run, ack-paced so each write lands as its own read (as on a real network, where reads don't coalesce to full slabs); medians of 5, macOS aarch64, both binaries built from the same tree with the same profile — the baseline is this branch's base commit, so the only difference is this diff. Every run content-verified (all retained bytes checked, totals exact). The baseline's flat ~380 MiB independent of chunk size is the signature of the bug: 20,000 × ~19 KB resident (the 64 KB slab, page-rounded by the OS) no matter how few bytes each read carried. With this PR, growth tracks the actual retained data at every size, within a few MiB of Node.

Wall time on the same runs improves as a side effect of the reduced allocator/GC pressure: 912→743 ms (512 B), 919→769 ms (1400 B), 955→841 ms (4 KiB); 16 KiB unchanged within noise.

Cause

on_uv_read (ext/node/ops/stream_wrap.rs) always transferred ownership of the 65536-byte slab from on_uv_alloc into the ArrayBuffer passed to onread, with a deleter that frees it at GC. Buffer.from(ab, 0, nread) in onStreamRead then views only nread bytes, but the view keeps the whole slab alive for as long as the consumer holds the chunk. Node instead shrinks the store to nread before handing it to JS (BackingStore::Reallocate in EmitToJSStreamListener::OnStreamRead, src/stream_base.cc).

Beyond RSS, the handoff also defeats the slab pool this file maintains (READ_BUF_POOL, 128 slots): slabs come back only at GC, so in steady state the pool runs dry and reads fall back to the system allocator — the exact mach_vm_reclaim overhead (~6% CPU in the profile that motivated the pool) it was added to avoid.

Fix

Reads with nread <= READ_COPY_THRESHOLD (half a slab, 32 KB) are copied into an exact-size backing store (new_backing_store_from_vec) and the slab is returned to the pool immediately. Larger reads — bulk transfers actually filling most of the slab — keep the existing zero-copy slab handoff, so the well-behaved path is untouched. The copy is bounded at 32 KB (~1 µs) against a read dispatch that costs ~10 µs+.

No regressions on the paths that were already fine

  • Bulk 1 GiB transfer (reads coalesce to ~64 KB → slab-handoff path): 2470 → 2463 MiB/s, medians of 7. Unchanged, as intended.
  • 512 B ping-pong, 32 connections × 5000 round trips (per-read fixed cost): 23.2 → 22.9 µs/rt, medians of 5.

Verification

  • unit_node::net_test, unit_node::http_test, unit_node::tls_test — pass on this branch (TLS matters here: ciphertext ingest rides this same read path via the TLSWrap onread forwarder).
  • specs::node (257 tests): failure set byte-identical to an upstream/main baseline build run on the same machine — the same 2 pre-existing child_process_ipc_handle::destroy_before_ack_* flakes on both (they reproduce at ~3/30 on unpatched debug builds).
  • Integrity checked on every benchmark run: every retained byte verified against the sent pattern, byte counts exact.
  • tools/format.js and tools/lint.js pass.
Benchmark source (amplification)
// Ack-paced small reads; client retains all chunks (protocol-assembly
// pattern). Usage: deno run -A bench.ts [chunks] [chunkBytes]
import net from "node:net";
import process from "node:process";

const args = typeof Deno !== "undefined" ? Deno.args : process.argv.slice(2);
const rssNow = () =>
  typeof Deno !== "undefined" ? Deno.memoryUsage().rss : process.memoryUsage().rss;

const CHUNKS = Number(args[0]) || 20000;
const CHUNK = Number(args[1]) || 1400;

const server = net.createServer((sock) => {
  sock.setNoDelay(true);
  const chunk = Buffer.alloc(CHUNK, 0x61);
  let sent = 0;
  // One chunk per 1-byte ack from the client, so reads never coalesce.
  sock.on("data", () => {
    if (sent >= CHUNKS) return sock.end();
    sent++;
    sock.write(chunk);
  });
  sock.on("error", () => {});
});
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
const port = (server.address() as net.AddressInfo).port;

const rssStart = rssNow();
const t0 = performance.now();
const retained: Buffer[] = [];
let received = 0, reads = 0;

await new Promise<void>((resolve, reject) => {
  const ack = Buffer.from([1]);
  const sock = net.connect({ port, host: "127.0.0.1" }, () => {
    sock.setNoDelay(true);
    sock.write(ack);
  });
  sock.on("data", (buf: Buffer) => {
    retained.push(buf);
    received += buf.length;
    reads++;
    sock.write(ack);
  });
  sock.on("end", resolve);
  sock.on("error", reject);
});

const wall = performance.now() - t0;
server.close();
const rssEnd = rssNow();

let badBytes = 0;
for (const b of retained) for (let i = 0; i < b.length; i++) if (b[i] !== 0x61) badBytes++;
console.log(`chunks=${CHUNKS} x ${CHUNK}B  wall ${wall.toFixed(0)} ms  reads=${reads}`);
console.log(
  `retainedData ${(received / 1048576).toFixed(1)} MiB  growth ${
    ((rssEnd - rssStart) / 1048576).toFixed(0)
  } MiB  amplification ${((rssEnd - rssStart) / received).toFixed(1)}x`,
);
console.log(`integrity: ${badBytes === 0 && received === CHUNKS * CHUNK ? "OK" : "FAIL"}`);
if (retained.length === -1) console.log(retained.length); // keep alive

The regression guards (bulk 1 GiB fire-and-forget writes and 512 B ping-pong) are separate scripts; happy to paste them on request.

I used Cursor (Claude) to help investigate and write this change.

…64KB slab

Every read on the node-compat stream path (node:net, pipes, TLS
ciphertext ingest) transferred ownership of the full 65536-byte read
slab into the ArrayBuffer handed to JS, regardless of nread.
Buffer.from(ab, 0, nread) then kept the whole slab alive until GC, so a
client that retains small chunks (protocol reassembly: tedious TDS
messages, length-prefixed framing) resident-pinned ~16KB per chunk
(page-rounded; 64KB virtual) — measured at 14x RSS amplification vs
~2x on Node, which right-sizes the store via BackingStore::Reallocate
in EmitToJSStreamListener::OnStreamRead. Handing off every slab also
kept more than POOLED_BUF_MAX slabs in flight in steady state (freed
only at GC), starving the read-buffer pool so reads fell back to the
system allocator — the exact mach_vm-reclaim cost the pool was added
to avoid.

Reads at or below READ_COPY_THRESHOLD (half a slab, 32KB) are now
copied into an exact-size backing store and the slab is returned to
the pool immediately; larger reads (bulk transfers filling most of the
slab) keep the zero-copy slab handoff. The copy costs at most ~1us
against a read dispatch that costs ~10us+.

Co-authored-by: Cursor <[email protected]>
@tomas-zijdemans
tomas-zijdemans marked this pull request as ready for review July 4, 2026 21:17
@bartlomieju

Copy link
Copy Markdown
Member

Thank you for the PR, this is a really nice performance improvement and the analysis and benchmarking are very thorough!

Two small asks before merging:

  1. Could you remove the non-ASCII characters from the comments (à la, µs, and the em-dashes)? We keep source code ASCII-only.
  2. Could you trim down the doc comment on READ_COPY_THRESHOLD? It currently reads more like the PR description (benchmark numbers, profiling percentages). A few lines stating the invariant would be enough: small reads are copied so the slab returns to the pool immediately and the JS Buffer doesn't pin the full 64KB; large reads keep the zero-copy handoff; mirrors Node's BackingStore::Reallocate. The measurements are great context, but they belong in the PR/commit message rather than the code.

Replace the non-ASCII characters introduced by this PR with ASCII
equivalents and shrink the READ_COPY_THRESHOLD doc comment to just the
invariant, moving the measurements to the PR description where they
belong.

Co-authored-by: Cursor <[email protected]>
@tomas-zijdemans

Copy link
Copy Markdown
Contributor Author

Sorry. Should be better now.

@nathanwhit nathanwhit left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found one performance/compatibility concern around the threshold behavior.

Comment thread ext/node/ops/stream_wrap.rs Outdated
tomas-zijdemans and others added 2 commits July 7, 2026 20:21
Per review feedback, replace the READ_COPY_THRESHOLD heuristic with
Node's exact condition: copy whenever nread < the slab size, so no
partial read (including the previously uncovered 33-63KB band) pins
the full 64KB slab, and the slab always returns to the pool
immediately. Only exactly-full slab reads keep the zero-copy handoff,
matching EmitToJSStreamListener::OnStreamRead.

Co-authored-by: Cursor <[email protected]>

@nathanwhit nathanwhit left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@nathanwhit
nathanwhit enabled auto-merge (squash) July 7, 2026 19:32
@nathanwhit
nathanwhit disabled auto-merge July 15, 2026 09:56
@nathanwhit
nathanwhit enabled auto-merge (squash) July 15, 2026 09:56
@nathanwhit
nathanwhit merged commit 208e85f into denoland:main Jul 15, 2026
268 of 270 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants