perf(ext/http): coalesce chunked response writes into a single write#35523
Merged
nathanwhit merged 4 commits intoJun 26, 2026
Merged
Conversation
A Deno.serve response with a ReadableStream body (Transfer-Encoding: chunked) emitted the head, body chunk, and terminator as three separate write syscalls; the fixed/Content-Length path coalesces head+body into one. This buffers the chunked path into the connection's existing per-request scratch buffer and flushes once. poll_start_chunked_response_with defers the head for Chunked responses (NoBody/CloseDelimited still flush immediately); poll_write_response_chunk_with appends each chunk's framing, flushing only past a 64 KiB threshold; poll_finish_response_with appends the terminator, flushes once, and shrinks the buffer back to its default capacity. The streaming drivers flush when the body source returns Pending so incremental/SSE responses stay prompt. chunked: 3 -> 1 write/req; stream throughput +35% (75.6k -> 102k rps, 100-byte body); fixed path and idle RSS unchanged. No new per-connection allocation.
Deferring the chunked response head (to coalesce it with the body) meant a body stream that errored before any flush left the head unsent, so the client saw a connection close before headers instead of the committed 200 + truncated body. Flush buffered bytes best-effort on a mid-stream body error, matching the prior eager-write behavior. Fixes specs::serve::stream_body_error.
…rites The duplex/FragmentIo-backed h1 tests always accept a full buffer in one poll_write, so they never exercised the buffered chunked path's partial-flush resume (write_flushed) or the buffered re-poll guard. Add a ShortWriteIo sink that caps bytes per write and periodically returns Pending, plus two tests: a byte-at-a-time coalesced response, and a chunk past CHUNKED_FLUSH_THRESHOLD that flushes mid-stream under short writes.
Satisfies clippy::manual_is_multiple_of (-D clippy::all).
nathanwhit
added a commit
that referenced
this pull request
Jun 30, 2026
…onse_with` (#35649) ## Issue details See #35648 ## Fix Add missing `scratch.write_flushed = scratch.write_buf.len();` in `poll_start_fixed_response_with`. _Without it_, `poll_flush_write_buf` re-sends the response head as the body when the `ReadableStream` source yields `Poll::Pending`. It mirrors what `poll_start_chunked_response_with` _already_ does (introduced in #35523) ## Verify - Create `example.ts` (change path to your PNG file as needed) ```ts Deno.serve({ port: 9999 }, async (request) => { await request.body?.cancel(); const path = "your-image.png"; const { size } = await Deno.stat(path); const file = await Deno.open(path, { read: true }); return new Response(file.readable, { headers: { "content-type": "image/png", "content-length": String(size), }, }); }); ``` - Run the example ```sh deno run -A ./example.ts Listening on http://0.0.0.0:9999/ (http://localhost:9999/) ``` - Check the output **Before:** ```sh curl -s http://localhost:9999 | head -c 8 HTTP/1.1 ``` **Now:** ```sh curl -s http://localhost:9999 | head -c 8 PNG ``` Fixes #35648 --------- Co-authored-by: Nathan Whitaker <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Deno.serveresponses whose body is aReadableStream(sentTransfer-Encoding: chunked) regressed in throughput when the HTTP/1 layer was rewritten: the chunked path emits the head, each body chunk, and the terminating0\r\n\r\nas three separatewritesyscalls, where it previously coalesced them into one. The fixed/Content-Lengthpath was unaffected (it already coalesces head+body into a single write), so the regression is specific to streamed/chunked responses — which is what shows up as "streaming got slower" in the runtime benchmarks (a header-mutating Hono middleware re-emits the response as aReadableStream, so it rides this path).Measured with a
write/sendtointerposer, 30k requests, 100-byte body:ReadableStream)Content-Length)Fix
Buffer a chunked response into the connection's existing per-request scratch buffer (
SharedScratch::write_buf) and flush it once, instead of writing each piece straight to the socket:poll_start_chunked_response_withleaves the head inwrite_buf(deferred) forChunkedresponses.NoBody/CloseDelimited(HTTP/1.0) still flush the head immediately, since they have no chunk framing to coalesce behind.poll_write_response_chunk_withappends each chunk's framing towrite_buf, flushing only once it crossesCHUNKED_FLUSH_THRESHOLD(64 KiB) so memory stays bounded for large/long-lived bodies.poll_finish_response_withappends the terminator, flushes once, then shrinkswrite_bufback to its default capacity so a connection that served one large response doesn't retain an oversized buffer.Poll::Pending), so incremental/SSE responses are still delivered promptly; only chunks that are available synchronously coalesce.No new per-connection allocation — it reuses the buffer that already exists for building response frames.
Results
oha -c 100 --disable-compression,--profile=release-lite, same build for both columns:Tests
deno_http_h1unit tests (which assert byte-exact chunked output, including partial/fragmented writes and trailers) pass.unit::serve_testandunit::http_testpass.