Skip to content

perf(ext/node): zero-copy async Buffer writes on the stream_wrap path#35780

Merged
nathanwhit merged 1 commit into
denoland:mainfrom
tomas-zijdemans:perf/stream-wrap-write-zero-copy
Jul 6, 2026
Merged

perf(ext/node): zero-copy async Buffer writes on the stream_wrap path#35780
nathanwhit merged 1 commit into
denoland:mainfrom
tomas-zijdemans:perf/stream-wrap-write-zero-copy

Conversation

@tomas-zijdemans

Copy link
Copy Markdown
Contributor

A large socket.write(buffer) on node:net memcpy'd (nearly) the entire payload into an owned Vec and held that duplicate until the write completed — doubling peak memory and burning a redundant copy. This PR queues the write as an iovec pointing directly at the ArrayBuffer's backing store:

write size baseline write cb this PR baseline peak RSS this PR RSS saved
64 MiB 43 ms 39 ms 247 MiB 188 MiB 59 MiB
128 MiB 73 ms 65 ms 442 MiB 316 MiB 126 MiB
256 MiB 128 ms 108 ms 731 MiB 485 MiB 246 MiB
512 MiB 253 ms 219 ms 1352 MiB 838 MiB 514 MiB

Loopback node:net, client.write(payload) timed from write() to its completion callback with a full-speed reader; medians of 7 per cell, macOS aarch64, both binaries built from the same tree with the same profile — the baseline is this branch's base commit, so the only difference is this diff. The tell is in the RSS column: the saving equals the payload size at every point (the eliminated duplicate), and the ~1.1–1.2x time win is the skipped memcpy. Node 26.3 on the same machine and bench: 201 ms / 658 MiB at 512 MiB, so this closes most of the gap.

Integrity: a patterned payload (offset stamped every 4 KiB, so reordering or corruption changes the digest) hashed with streaming SHA-256 at the receiver matches the sender's hash — byte-exact, and the same digest on baseline and patched binaries.

Cause

write_buffer (ext/node/ops/stream_wrap.rs) starts with a sync uv_try_write, which usually drains small writes. Whenever it can't — backpressure, or any write larger than the socket buffer — the async fallback went through uv_write, whose collect_bufs copies the whole undrained tail into an owned Vec. The copy was redundant: JS already retains the Buffer on the write request until oncomplete (req.buffer = data in handleWriteReq, stream_base_commons.ts) — the very same retention contract the writev op already relies on for its zero-copy iovecs.

Fix

Queue a one-entry iovec at the backing store via uv_writev_owned (the existing zero-copy machinery), instead of uv_write's copy.

One case keeps an owned copy: V8 stores small typed arrays (≤ 64 bytes, TYPED_ARRAY_MAX_SIZE_IN_HEAP) on the JS heap, and get_contents copies those into a stack buffer that dies when the op returns. That case is detected by pointer identity against the stack buffer and routed through uv_write_owned — a ≤ 64-byte copy.

No regressions on adjacent paths

  • 512 B ping-pong, 32 connections × 5000 round trips (small writes take the sync uv_try_write path): 22.8 → 22.6 µs/rt, medians of 5.
  • 256 MiB enqueued in 64 KiB chunks behind a paused reader (deep write queue): 146 → 149 ms total, medians of 5 — noise; the stream layer keeps one uv write in flight and drains the rest through the already-zero-copy writev.
  • node:tls write patterns (2000 × 4 KB awaited / fire-and-forget / corked writev / 1 × 32 MB): all within noise — TLS encrypted output goes through its own uv_write with Rust-owned ciphertext and is untouched by this change.

Verification

  • unit_node::net_test, unit_node::http_test, unit_node::tls_test — pass on this branch. Full transparency: in 22 runs of tls_test on this branch, one run failed tls js-backed duplex client end() during handshake still sends close_notify (TLSv1.3) (a test that races end() against handshake completion); it passed the other 21 runs here and 12/12 runs on the baseline, and I could not reproduce it again.
  • specs::node (257 tests): failure set byte-identical to an upstream/main baseline build run on the same machine — the same 2 pre-existing child_process_ipc_handle::destroy_before_ack_* flakes on both (they reproduce at ~3/30 on unpatched debug builds).
  • SHA-256 integrity check byte-exact on every verify run (see above).
  • tools/format.js and tools/lint.js pass.
Benchmark source (large write)
// Single large Buffer write; reader consumes at full speed.
// Usage: deno run -A bench.ts [sizeMiB] [verify]
import { createHash } from "node:crypto";
import net from "node:net";
import process from "node:process";

const args = typeof Deno !== "undefined" ? Deno.args : process.argv.slice(2);
const rssNow = () =>
  typeof Deno !== "undefined" ? Deno.memoryUsage().rss : process.memoryUsage().rss;

const SIZE = (Number(args[0]) || 512) * 1024 * 1024;
const VERIFY = args[1] === "verify";

let peakRss = 0;
const rssTimer = setInterval(() => {
  const rss = rssNow();
  if (rss > peakRss) peakRss = rss;
}, 5);

let received = 0;
const rxHash = VERIFY ? createHash("sha256") : null;
const server = net.createServer((sock) => {
  sock.on("data", (b: Buffer) => {
    received += b.length;
    if (rxHash) rxHash.update(b);
  });
  sock.on("error", () => {});
});
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
const port = (server.address() as net.AddressInfo).port;

const sock: net.Socket = await new Promise((resolve, reject) => {
  const c = net.connect({ port, host: "127.0.0.1" }, () => resolve(c));
  c.on("error", reject);
});

const big = Buffer.alloc(SIZE, 0x61);
if (VERIFY) for (let i = 0; i < SIZE; i += 4096) big.writeUInt32LE(i >>> 0, i);

const t0 = performance.now();
await new Promise<void>((res, rej) => sock.write(big, (e) => e ? rej(e) : res()));
const writeDone = performance.now() - t0;
await new Promise<void>((resolve) => {
  const iv = setInterval(() => {
    if (received >= SIZE) { clearInterval(iv); resolve(); }
  }, 2);
});
sock.end();
server.close();
clearInterval(rssTimer);

console.log(
  `size=${SIZE / 1048576}MiB  writeCb ${writeDone.toFixed(0)} ms  peakRSS ${
    (peakRss / 1048576).toFixed(0)
  } MiB`,
);
if (rxHash) {
  const sent = createHash("sha256").update(big).digest("hex");
  console.log(`integrity: ${sent === rxHash.digest("hex") ? "OK " + sent.slice(0, 16) : "FAIL"}`);
}

I used Cursor (Claude) to help investigate and write this change.

write_buffer's async fallback (taken whenever a Buffer write does not
fully drain synchronously — backpressure, or any large write) went
through uv_write, whose collect_bufs copies the entire undrained tail
into an owned Vec. A single large socket.write(buf) therefore memcpy'd
nearly the whole payload and held a duplicate of it until the write
completed, doubling peak memory. The copy was redundant: JS already
retains the Buffer on the write request until oncomplete
(req.buffer = data in stream_base_commons' handleWriteReq), the same
retention contract the writev op relies on for its zero-copy iovecs.

Queue a one-entry iovec pointing at the ArrayBuffer backing store via
uv_writev_owned instead. One exception keeps an owned copy: V8 copies
on-heap typed arrays (<= 64 bytes, TYPED_ARRAY_MAX_SIZE_IN_HEAP) into
get_contents' stack buffer, which dies with the op — detect that by
pointer identity and route it through uv_write_owned.

Co-authored-by: Cursor <[email protected]>
@tomas-zijdemans
tomas-zijdemans marked this pull request as ready for review July 4, 2026 21:16

@nathanwhit nathanwhit left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!

@nathanwhit
nathanwhit merged commit aa1380b into denoland:main Jul 6, 2026
172 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants