When sockets get reused for multiple requests to endpoints that support chunked transfer encoding, the code in node-fetch that's designed to detect malformed chunked responses gets confused and throws false-positive 'premature close' errors.
Sockets get reused in such a way when you use an http.Agent or an https.Agent in ‘keep alive’ mode.
Reproduction
Steps to reproduce the behavior:
- Add this logging to
node-fetch (diff link)
- Create an HTTPS Agent with
keepAlive: true and maxSockets of 5.
- Make 6 requests in a row for an endpoint that supports chunked transfer encoding
In that repro, 6 requests are made with a limit of 5 sockets in keep-alive mode. One of those sockets gets reused for the 6th request. The presence of the 6th request as a pending listener for that reused socket confuses the logic in node-fetch that is designed to detect corrupt chunked responses.
Here's a repro project that you can clone that does all of the above: https://github.com/steveluscher/repro-node-fetch-premature-close-with-agent
Expected behavior
No premature close error should be thrown, since the chunked responses are well-formed.
Your Environment
| software |
version |
| node-fetch |
2.6.8-2.6.12 |
| node |
18+ |
Additional context
This bug is the root of solana-foundation/solana-web3.js#1418.
cc/ @achingbrain.
When sockets get reused for multiple requests to endpoints that support chunked transfer encoding, the code in
node-fetchthat's designed to detect malformed chunked responses gets confused and throws false-positive'premature close'errors.Sockets get reused in such a way when you use an
http.Agentor anhttps.Agentin ‘keep alive’ mode.Reproduction
Steps to reproduce the behavior:
node-fetch(diff link)keepAlive: trueandmaxSocketsof 5.In that repro, 6 requests are made with a limit of 5 sockets in keep-alive mode. One of those sockets gets reused for the 6th request. The presence of the 6th request as a pending listener for that reused socket confuses the logic in
node-fetchthat is designed to detect corrupt chunked responses.Here's a repro project that you can clone that does all of the above: https://github.com/steveluscher/repro-node-fetch-premature-close-with-agent
Expected behavior
No premature close error should be thrown, since the chunked responses are well-formed.
Your Environment
Additional context
This bug is the root of solana-foundation/solana-web3.js#1418.
cc/ @achingbrain.