Skip to content

perf(ext/web): reduce per-chunk allocations on stream read and tee paths#35786

Merged
bartlomieju merged 4 commits into
mainfrom
perf/web-streams-tee
Jul 6, 2026
Merged

perf(ext/web): reduce per-chunk allocations on stream read and tee paths#35786
bartlomieju merged 4 commits into
mainfrom
perf/web-streams-tee

Conversation

@bartlomieju

@bartlomieju bartlomieju commented Jul 5, 2026

Copy link
Copy Markdown
Member

Profiling the Web Streams tee() hot loop (V8 sampling profiler plus GC
scavenge counting) showed the per-chunk cost concentrated in allocation
churn: read request object literals carrying three closure methods each,
tee read requests and their chunk-steps microtask callbacks recreated for
every chunk, pull-promise reaction closures recreated on every pull, and
a fresh resolved promise allocated per internal pull just so
callPullIfNeeded had something to subscribe to.

Three commits, each independently measured on release builds: read
requests in reader.read()/read(view) become classes holding the Deferred
(one allocation instead of four); both tee variants hoist their read
request and chunk-steps microtask callback to per-tee scope, reusing them
across chunks (safe: the reading flag guarantees one read in flight); and
pull reactions get per-controller handler pairs plus a
synchronous-completion sentinel so internal pull algorithms skip the
promise pair entirely. Observable ordering is unchanged; a
queueMicrotask-based dispatch was tried and rejected as a measured
regression, details in the commit message.

Benchmark (M1 Max, release build, 1000 chunks/stream):

scenario before after change
tee + drain both branches 472 ns/chunk 391 ns/chunk -17%
drain, no tee 168 ns/chunk 155 ns/chunk -8%
byte tee + drain both 1468 ns/chunk 1390 ns/chunk -5%
GC share of profile ticks 7.4% ~3%

WPT streams is green (153 passed, 0 failed, 4 expected failures,
identical to baseline), as are fetch/api/response (54), streams/piping
(26), and streams/readable-byte-streams (20, including the BYOB tee
alternating-branch tests).

Towards #35768, #35771

…r reader.read()

Replaces per-read readRequest/readIntoRequest object literals (one object
plus three closure methods per read) with classes holding the Deferred in
ReadableStreamDefaultReader.read() and ReadableStreamBYOBReader.read(view).

Hoists the tee read requests and their chunkSteps microtask callbacks out
of the pull algorithms in readableStreamDefaultTee and readableByteStreamTee
so they are allocated once per tee() instead of once per chunk. Safe because
the reading flag guarantees at most one read is in flight; the pending chunk
travels through a scope slot consumed at the start of the microtask.

Benchmark (M1 Max, release build, 1000 chunks/stream, ns/chunk):

  tee + drain both branches   472 -> 429  (-9%)
  drain, no tee               168 -> 160  (-5%)
  byte tee + drain both      1468 -> 1423 (-3%)

GC share of profiled ticks drops from 7.4% to 3.0% on the tee hot loop.

Towards #35768, #35771
…ller

readableStreamDefaultControllerCallPullIfNeeded and
readableByteStreamControllerCallPullIfNeeded allocated four objects per
pull for the pull-promise reactions (two handlers plus the two
assertion-rethrow wrappers uponPromise creates). Create the pre-wrapped
handler pair lazily on first pull and reuse it for every subsequent pull
on that controller. Same pattern as the sink write reaction handlers on
the writable side.

Benchmark (M1 Max, release build, 1000 chunks/stream, ns/chunk,
on top of the read-request reuse commit):

  tee + drain both branches   429 -> 396
  drain, no tee               160 -> 157
  byte tee + drain both      1423 -> 1367

Cumulative from baseline: tee -16%, drain -7%, byte tee -7%.

Towards #35768
…thms

Internal pull algorithms (both tee variants, the default no-op pull, and
the static fetch body stream) returned PromiseResolve(undefined) on every
pull, which callPullIfNeeded immediately subscribed to, allocating a
fresh resolved promise per pull. They now return undefined as a
synchronous completion sentinel, and both callPullIfNeeded variants
subscribe the per-controller fulfilled handler to a shared lazily
created resolved promise instead.

A reaction subscribed to an already-resolved promise runs at the same
position in the microtask queue regardless of which resolved promise it
is attached to, so observable ordering (number and timing of source
pull() calls, read resolution order) is unchanged. WPT streams is green
(153 passed, 0 failed, 4 expected failures, identical to baseline), as
is fetch/api/response (54 passed).

Dispatching through queueMicrotask instead was measured and rejected: its
dispatch is ~15ns/tick slower than a resolved-promise reaction (plus the
core wrapper closure), a net regression on the tee bench.

User-provided pull callbacks are unaffected: they go through
webidl.invokeCallbackFunction, which always returns a promise.

Benchmark (M1 Max, release build, 1000 chunks/stream, ns/chunk, on top
of the per-controller pull handlers commit):

  tee + drain both branches   396 -> 391
  drain, no tee               157 -> 155

Cumulative from the pre-series baseline: tee 472 -> 391 (-17%),
drain 168 -> 155 (-8%), byte tee 1468 -> 1390 (-5%).

Towards #35768, #35771
@bartlomieju

Copy link
Copy Markdown
Member Author

Self-review notes:

  • Stale JSDoc on the sentinel contract: createReadableStream (line 541) was updated to () => Promise<void> | undefined, but three other entry points that now receive sentinel-returning pull algorithms still say () => Promise<void>:

    • createReadableByteStream (line 682) — gets pull1Algorithm/pull2Algorithm from the byte tee
    • setUpReadableByteStreamController (line 3942)
    • setUpReadableStreamDefaultController (line 4059)

    Since "pull may return undefined instead of a promise" is a new implicit protocol, keeping every entry point's type annotation accurate is the main defense against someone later calling .then() on a pull result. Will fix.

  • Symbol naming: Symbol("[[onPullFulfilled]]") / Symbol("[[onPullRejected]]") use the double-bracket spec-slot convention, but these are implementation slots, not spec slots. The file's precedent for non-spec slots is single brackets (Symbol("[pendingAbortRequest]")).

  • Additional test runs before merge (beyond WPT streams + fetch/api/response): tests/unit/streams_test.ts, the WPT piping/ suite (heaviest callPullIfNeeded consumer), and a node-compat streams pass (node:stream/web re-exports these implementations). The case most dependent on the new shared-slot invariant (byobForBranch2 consumed by a reused read-into request) is BYOB tee with alternating branch reads — WPT readable-byte-streams/tee covers it, worth confirming it's in the run.

…on-slot symbol naming

Updates the pull algorithm type to '() => Promise<void> | undefined' on
createReadableByteStream, setUpReadableByteStreamController, and
setUpReadableStreamDefaultController, which all receive
sentinel-returning pull algorithms; the annotation is the main defense
against a future caller invoking .then() on a pull result.

Renames the pull reaction handler symbols to the single-bracket
convention the file uses for implementation slots that are not spec
slots.
@bartlomieju

Copy link
Copy Markdown
Member Author

Addressed in 809ffc2:

  • Sentinel JSDoc: pullAlgorithm is now typed
    () => Promise<void> | undefined on all four entry points that
    receive sentinel-returning pull algorithms (createReadableStream,
    createReadableByteStream, setUpReadableByteStreamController,
    setUpReadableStreamDefaultController), with the contract note on
    both create* helpers.
  • Symbol naming: _onPullFulfilled/_onPullRejected now use the
    single-bracket implementation-slot convention
    (Symbol("[onPullFulfilled]")), matching [pendingAbortRequest].

Test runs from the notes:

  • WPT streams/piping/: 26 passed, 0 failed.
  • WPT streams/readable-byte-streams/: 20 passed, 0 failed, and
    readable-byte-streams/tee.any.{html,worker.html} (the BYOB tee
    alternating-branch reads that exercise the reused read-into request's
    byobForBranch2 slot) confirmed present and passing in the run.
  • tests/unit/streams_test.ts and the node-compat streams pass run in
    this PR's CI matrix on the new push.

@bartlomieju
bartlomieju enabled auto-merge (squash) July 6, 2026 09:36
@bartlomieju
bartlomieju merged commit da93007 into main Jul 6, 2026
138 checks passed
@bartlomieju
bartlomieju deleted the perf/web-streams-tee branch July 6, 2026 10:19
bartlomieju added a commit that referenced this pull request Jul 6, 2026
Conflicts resolved keeping main's single-bracket symbol naming and this
branch's synchronous pull completion (which also removes the now-unused
resolvedPromise() helper).
bartlomieju added a commit that referenced this pull request Jul 6, 2026
#35788)

Follow-up to #35786. Even with the synchronous-completion sentinel from
that PR, every synchronously-completing internal pull still paid one
promise reaction per pull just to flip `pulling` back to false a
microtask later. This runs those upon-fulfillment steps synchronously
instead: clear the flag, and if pullAgain was set during the pull,
recurse into callPullIfNeeded, which re-checks shouldCallPull exactly
like the async reaction would. Termination is guaranteed by the internal
algorithms' own re-entry guards (the tee reading flag, closeRequested on
one-shot sources). The now-unused shared resolved promise helper is
removed.

The observable difference is that a subsequent pull can begin in the
same synchronous segment instead of one microtask later. WPT streams
(153 passed, 0 failed, 4 expected failures, identical to baseline) and
fetch/api/response (54 passed) show this is not distinguishable by the
spec's ordering tests.

Benchmark (M1 Max, release build, 1000 chunks/stream):

| scenario | before | after | change |
|---|---|---|---|
| tee + drain both branches | 391 ns/chunk | 348 ns/chunk | -11% |
| drain, no tee | 155 ns/chunk | 152 ns/chunk | -2% |
| byte tee + drain both | 1390 ns/chunk | 1341 ns/chunk | -4% |

Towards #35768, #35771
bartlomieju added a commit that referenced this pull request Jul 6, 2026
…tent read request (#35790)

Follow-up to #35786 and #35788. The slow path of
readableStreamCollectIntoUint8Array, which backs every Body consumer for
JS-source streams (.bytes(), .text(), .json(), .arrayBuffer(), Blob),
looped over reader.read() and paid a Deferred, a read request instance,
a {value, done} result object, and an await microtask hop per chunk.

It now pumps with one persistent read request: synchronously available
chunks are delivered re-entrantly and re-armed in a do/while at constant
stack depth with zero microtask hops, and asynchronous chunks re-enter
the pump paying only the source's own pull reaction hop. Error and close
semantics are unchanged, including the TypeError on non-Uint8Array
chunks.

Benchmark (M1 Max, release build):

| scenario | before | after |
|---|---|---|
| .bytes(), 1KB chunks | 791 ns/chunk | 657 ns/chunk |
| same, equivalent userland read loop | 774 ns/chunk | 700 ns/chunk |

The consumer previously trailed a hand-written reader.read() loop over
the same stream; it now beats it. WPT streams (153 passed, 0 failed, 4
expected failures, identical to baseline), fetch/api/response (54), and
fetch/api/body (4) are green.

Towards #35768
bartlomieju added a commit that referenced this pull request Jul 7, 2026
Addresses the remaining generic-`pipeTo` work from #35768 (the
read-request reuse, pull-reaction, and identity-bypass items already landed in
#35786, #35788, #35790, #35799).

Towards #35768
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant