http: propagate highWaterMark to ClientRequest OutgoingMessage#64653
http: propagate highWaterMark to ClientRequest OutgoingMessage#64653trivenay wants to merge 1 commit into
Conversation
|
Review requested:
|
`http.request({ highWaterMark })` passes the value to the TCP socket
via createConnection() but does not set it on the OutgoingMessage
internal kHighWaterMark. OutgoingMessage._writeRaw() has two mutually
exclusive write paths:
Path A (socket connected): conn.write() — uses socket HWM ✓
Path B (no socket yet): outputSize < this[kHighWaterMark] — uses
OutgoingMessage own default (64 KB) ✗
Because the OutgoingMessage constructor already accepts
options.highWaterMark, the fix is to set kHighWaterMark from the
user options after they are parsed in the ClientRequest constructor.
This resolves two symptoms:
1. write() returning the wrong boolean for pre-socket writes (the
user highWaterMark was silently ignored on all Node versions).
2. A deadlock on Node >= 24.16.0 where the incorrect false return
sets kNeedDrain, but drain never fires because the socket was
never backpressured (introduced by the stricter drain gate in
nodejs#62936).
Signed-off-by: Trivikram Narayanan Kamasamudram <[email protected]>
Fixes: nodejs#64645
Refs: nodejs#62936
b1249d0 to
a008632
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #64653 +/- ##
=======================================
Coverage 90.12% 90.13%
=======================================
Files 741 741
Lines 242091 242142 +51
Branches 45563 45563
=======================================
+ Hits 218180 218250 +70
+ Misses 15430 15394 -36
- Partials 8481 8498 +17
🚀 New features to boost your workflow:
|
This comment was marked as outdated.
This comment was marked as outdated.
pimterry
left a comment
There was a problem hiding this comment.
I think there's a missing case to watch for here with agents or createConnection. In those cases, the request often doesn't directly create or own the socket - an agent might create it independently, it might be a reuse of an existing socket from another request, or it could be a custom non-socket stream from anywhere, and in each case it may have its own independent highWaterMark option already set.
I'm not sure but I strongly suspect this won't properly handle that, since it's assuming that the request owns the socket and controls the HWM setting. I think if you reproduce the HWM mismatch in one of those contexts there's still a route to a deadlock with no drain in some cases.
I won't block this - it is still a clear improvement on the current state in the meantime, but if you have time to investigate and potentially fix that flow en route that would be fantastic.
http.request({ highWaterMark })passes the value to the TCP socket viacreateConnection()but does not set it on theOutgoingMessage's internalkHighWaterMark.OutgoingMessage._writeRaw()has two mutually exclusive write paths:conn.write()— uses socket's HWM (correct)outputSize < this[kHighWaterMark]— uses OutgoingMessage's own default (64 KB) (incorrect)Because the
OutgoingMessageconstructor already acceptsoptions.highWaterMark, the fix is to setkHighWaterMarkfrom the user's options after they are parsed in theClientRequestconstructor.This resolves two symptoms:
write()returning the wrong boolean for pre-socket writes (the user'shighWaterMarkwas silently ignored on all Node versions).falsereturn setskNeedDrain, butdrainnever fires because the socket was never backpressured (introduced by the stricter drain gate in http: emit 'drain' on OutgoingMessage only after buffers drain #62936).Test results (compiled from source)
Minimal reproduction
node -e "require('http').createServer((req,res)=>{req.resume();req.on('end',()=>res.end('ok'))}).listen(9001)"write()returnsfalse(wrong)false(wrong)true(correct)Fixes: #64645
Refs: #62936