kqueue: write-side socket error can be dispatched before pending readable data after TCP reset

When a peer sends a response and immediately resets the TCP connection while the local endpoint still has pending writes, the kqueue backend can dispatch the write-side error before pending readable data.

On affected macOS runs, this causes uv__write() to report EPIPE before Node.js can parse an already-sent HTTP response. The write error then marks the Duplex readable side as errored, so the response is never delivered to the HTTP parser.

This issue concerns the Darwin/kqueue ordering found while investigating the Node.js HTTP regression. It may not be the sole cause of the original Linux release boundary between Node.js v24.15 and v24.16.

code example

  'use strict';

  const http = require('node:http');
  const body = Buffer.alloc(2 * 1024 * 1024);

  const server = http.createServer((request, response) => {
    let received = 0;

    request.on('data', function onData(chunk) {
      received += chunk.length;
      if (received <= body.length / 2) return;

      request.off('data', onData);
      response.writeHead(413, { 'content-length': 0 });
      response.end();
      request.destroy();
    });
  });

  server.listen(0, () => {
    const req = http.request({
      method: 'POST',
      port: server.address().port,
      headers: { 'content-length': body.length },
    }, (res) => {
      console.log('response', res.statusCode);
      res.resume();
      res.on('end', () => console.log('response complete', res.complete));
    });

    req.on('error', (err) => {
      console.log({
        code: err.code,
        syscall: err.syscall,
        writableFinished: req.writableFinished,
        hasResponse: Boolean(req.res),
        responseComplete: req.res?.complete,
      });
    });

    req.on('close', () => server.close());
    req.end(body);
  });

On affected Darwin runs, the first observable result is similar to:

code: EPIPE
syscall: write
writableFinished: true
hasResponse: false

A patched kqueue backend showed an EVFILT_WRITE event with EV_EOF and a non-zero fflags socket error being delivered before the corresponding readable event.

The current kqueue mapping (https://github.com/libuv/libuv/blob/v1.52.0/src/unix/kqueue.c#L330-L400) maps that event to POLLOUT. It maps EV_ERROR to POLLERR, but does not account for a runtime socket error carried by EV_EOF and fflags.

Consequently, uv__stream_io() (https://github.com/libuv/libuv/blob/v1.52.0/src/unix/stream.c#L1190-L1238) receives only POLLOUT and calls uv__write() first. The write produces EPIPE.

If the write error is temporarily prevented from disabling the readable side, the same socket subsequently delivers a complete HTTP response. This indicates that response data was still available but had not reached the HTTP parser when the write error was propagated.

Darwin stores the reset error in so_error, closes the TCP state, and wakes both socket directions. Since kqueue represents read and write filters as separate events, their delivery order can differ from epoll, where readiness and error flags for the same descriptor are commonly returned together.

Diagnostic runs:

Raw kqueue instrumentation (https://github.com/Archkon/node/actions/runs/29524179884/job/87708326818)
Delayed readable-side error propagation (https://github.com/Archkon/node/actions/runs/29488284966/job/87588214115)

Possible solution ?

One possible backend normalization is:

  if (ev->filter == EVFILT_WRITE &&
      (ev->flags & EV_EOF) &&
      ev->fflags != 0)
    revents |= POLLERR;

This retains POLLOUT and adds POLLERR. The existing stream handler would therefore attempt uv__read() before uv__write().

Is this normalization appropriate for libuv’s kqueue backend, or is the current platform-dependent ordering considered expected behavior?

And could check this nodejs/node#64507 (comment)

When a peer sends a response and immediately resets the TCP connection while the local endpoint still has pending writes

That's an apparent violation of the TCP standard (and HTTP standard), so this seems to be working as intended in the presence of such a concurrency mistake.

I don't want to research and explain the precise violation, so I am letting Claude do it:

The data loss in this reproduction isn't a kqueue bug — it's the documented, expected consequence of an abortive connection close, and the reliability the test expects is something neither TCP nor HTTP promises across a reset.

The server forces a RST, and the standards say that discards in-flight data.

In the repro the server reads half the body, writes the 413, and then calls request.destroy() while ~1 MB of the request body is still unread (the client keeps sending). Closing a socket with unread inbound data — or receiving more data after close — is exactly the case where TCP is required to abort with a RST rather than close gracefully:

▎ RFC 1122 §4.2.2.13: "If such a host issues a CLOSE call while received data is still pending in TCP, or if new data is received after CLOSE is called, its TCP SHOULD send a RST to show that data was lost."

And a received RST is an abort, not an orderly release:

▎ RFC 9293 §3.10.7 / §3.5.1: receipt of a RST in ESTABLISHED "aborts the connection and advises the user"; "an 'abort' in which one or more RST segments are sent and the connection state is immediately discarded." Segment queues are flushed and outstanding RECEIVEs get reset responses.

The key point: TCP only guarantees reliable, ordered delivery for the graceful FIN close sequence. Once a RST is in play, previously transmitted bytes sitting in the peer's receive buffer may be discarded before the application reads them. So "the 413 was already sent" does not entitle the client to read it. Whether it surfaces is a race — which is precisely the platform-dependent behavior being observed. The behavior isn't wrong; the guarantee being relied on doesn't exist.

HTTP explicitly documents this exact hazard — and puts the fix at the HTTP layer.

RFC 9112 §9.6 (Tear-down) describes this scenario almost verbatim:

▎ "If a server performs an immediate close of a TCP connection, there is a significant risk that the client will not be able to read the last HTTP response. If the server receives additional data from the client on a fully closed connection… the server's TCP stack will send a reset packet
▎ to the client; unfortunately, the reset packet might erase the client's unacknowledged input buffers before they can be read and interpreted by the client's HTTP parser."

The standard's prescribed remedy is a staged/half-close: shutdown(SHUT_WR), then keep draining the client's remaining body until you see its close (or you're confident your response was ACKed), then fully close. The server in this repro does the opposite — it aborts while the client is
still uploading — so the response loss is the documented outcome of not following the tear-down procedure, not a kernel-notification ordering defect.

Why the proposed kqueue normalization doesn't actually fix it.

Adding POLLERR on EVFILT_WRITE | EV_EOF with non-zero fflags so that uv__read() runs before uv__write() only helps in the case where the response bytes happen to already be buffered locally when the error is delivered. It cannot provide the guarantee the report wants, because:

On the wire the RST may arrive before, interleaved with, or after the response segment, and per the RFCs above TCP is permitted to flush that buffered response on RST. No event-ordering change in libuv can make an aborted connection deliver data reliably.

The epoll behavior being treated as the reference ("readiness and error flags commonly returned together") is itself not a guaranteed contract — relying on it is relying on unspecified coincidence.

@vtjnash

When a peer sends a response and immediately resets the TCP connection while the local endpoint still has pending writes

That's an apparent violation of the TCP standard (and HTTP standard), so this seems to be working as intended in the presence of such a concurrency mistake.

I don't want to research and explain the precise violation, so I am letting Claude do it:

The data loss in this reproduction isn't a kqueue bug — it's the documented, expected consequence of an abortive connection close, and the reliability the test expects is something neither TCP nor HTTP promises across a reset.

The server forces a RST, and the standards say that discards in-flight data.

In the repro the server reads half the body, writes the 413, and then calls request.destroy() while ~1 MB of the request body is still unread (the client keeps sending). Closing a socket with unread inbound data — or receiving more data after close — is exactly the case where TCP is required to abort with a RST rather than close gracefully:
▎ RFC 1122 §4.2.2.13: "If such a host issues a CLOSE call while received data is still pending in TCP, or if new data is received after CLOSE is called, its TCP SHOULD send a RST to show that data was lost."
And a received RST is an abort, not an orderly release:
▎ RFC 9293 §3.10.7 / §3.5.1: receipt of a RST in ESTABLISHED "aborts the connection and advises the user"; "an 'abort' in which one or more RST segments are sent and the connection state is immediately discarded." Segment queues are flushed and outstanding RECEIVEs get reset responses.
The key point: TCP only guarantees reliable, ordered delivery for the graceful FIN close sequence. Once a RST is in play, previously transmitted bytes sitting in the peer's receive buffer may be discarded before the application reads them. So "the 413 was already sent" does not entitle the client to read it. Whether it surfaces is a race — which is precisely the platform-dependent behavior being observed. The behavior isn't wrong; the guarantee being relied on doesn't exist.

HTTP explicitly documents this exact hazard — and puts the fix at the HTTP layer.

RFC 9112 §9.6 (Tear-down) describes this scenario almost verbatim:
▎ "If a server performs an immediate close of a TCP connection, there is a significant risk that the client will not be able to read the last HTTP response. If the server receives additional data from the client on a fully closed connection… the server's TCP stack will send a reset packet
▎ to the client; unfortunately, the reset packet might erase the client's unacknowledged input buffers before they can be read and interpreted by the client's HTTP parser."
The standard's prescribed remedy is a staged/half-close: shutdown(SHUT_WR), then keep draining the client's remaining body until you see its close (or you're confident your response was ACKed), then fully close. The server in this repro does the opposite — it aborts while the client is
still uploading — so the response loss is the documented outcome of not following the tear-down procedure, not a kernel-notification ordering defect.

Why the proposed kqueue normalization doesn't actually fix it.

Adding POLLERR on EVFILT_WRITE | EV_EOF with non-zero fflags so that uv__read() runs before uv__write() only helps in the case where the response bytes happen to already be buffered locally when the error is delivered. It cannot provide the guarantee the report wants, because:

On the wire the RST may arrive before, interleaved with, or after the response segment, and per the RFCs above TCP is permitted to flush that buffered response on RST. No event-ordering change in libuv can make an aborted connection deliver data reliably.

The epoll behavior being treated as the reference ("readiness and error flags commonly returned together") is itself not a guaranteed contract — relying on it is relying on unspecified coincidence.

@vtjnash
Thank you for pointing out this behavior, but all rfc document from claude seems just say the response would be not definite to be read by clinet due to kernel buffer clear or drop or lost in transport but I mean that when the client could truly read this data packet and proccessing http parse while kqueue reports EVFILT_WRITE + EV_EOF with a socket error , should libuv normalize the event to include POLLERR, allowing currently readable data to be cotinusouly consumed before the write error is exposed ?

@vtjnash

@vtjnash Apologies for pinging you again. Would you be willing to reopen this issue and discuss my proposal with the other libuv maintainers? I realize this may be asking a lot, and I sincerely apologize for the imposition.
I just meant to solve this case "only helps in the case where the response bytes happen to already be buffered locally when the error is delivered"

No, libuv shouldn't try to adapt to clients that deliberately violate the TCP standard. That way just lies madness (and ever more difficult to diagnose bugs--like the infamous 500 mile email).

You've mentioned the data is definitely in the kernel buffer, but that's what is not safe to assume here. The TCP state diagram shows the data was enroute to that buffer, but the RFC protocol instructs any intermediate handlers to discard that data upon observation of the RST packet. It seems that in your limited testing, that did happen to be already in the kernel buffer, but that makes a lot of timing assumptions that aren't valid for the TCP standard (as detailed in RFC 9112 §9.6).

@vtjnash

You've mentioned the data is definitely in the kernel buffer, but that's what is not safe to assume here. The TCP state diagram shows the data was enroute to that buffer, but the RFC protocol instructs any intermediate handlers to discard that data upon observation of the RST packet. It seems that in your limited testing, that did happen to be already in the kernel buffer, but that makes a lot of timing assumptions that aren't valid for the TCP standard (as detailed in RFC 9112 §9.6).

@vtjnash
Sorry,but I don't assume that "data is definitely in the kernel buffer" ,I just assume "if".I don't know whether there would be some wording problem mislead you ?And RFC 9112 §9.6 just said "the reset packet might erase the client's unacknowledged input buffers before they can be read and interpreted by the client's HTTP parser" so not definitely drop data packet in buffer ?I don't know if I miss something?

@vtjnash

No, libuv shouldn't try to adapt to clients that deliberately violate the TCP standard. That way just lies madness (and ever more difficult to diagnose bugs--like the infamous 500 mile email).

@vtjnash
And I don't mean that let libuv to adapt to tcp client that violate tcp standard
I just to say that as the reference from rfc document, it seems that data packet don't definitely be drop by kernel while write error happend, rfc document seems never said we should't let the data packet be read if that is readable for this conditon?I think if we could let the readable data to be read which not violate what rfc document said ?
I don't know whether my understanding is correct ? I'm happy that you could guide me something I may miss? I

http: defer write errors for pending responses

Navigation Menu

Repository navigation

kqueue: write-side socket error can be dispatched before pending readable data after TCP reset #5196

Possible solution ?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

kqueue: write-side socket error can be dispatched before pending readable data after TCP reset #5196

Description

Possible solution ?

Activity

Archkon commented on Jul 17, 2026

vtjnash commented on Jul 17, 2026

Archkon commented on Jul 17, 2026

Archkon commented on Jul 17, 2026

vtjnash commented on Jul 17, 2026

vtjnash commented on Jul 17, 2026

Archkon commented on Jul 17, 2026

Archkon commented on Jul 17, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions