- Version:
v10.14.2
- Platform:
64-bit Windows
- Subsystem:
http, stream, buffer
Use axios to download a file from HTTPs URL by piping response readable stream into a fs Writable stream. However, the Node.js process will exit unexpectedly with following errors thrown. The errors seem from Node.js core modules related to TLSSocket, stream and buffer.
Another reason I think this should be a Node.js issue is that, with the same code, the issue can be stable reproduced in 10.14.2, but 8.14.0 doesn't have any issues.
buffer.js:269
throw err;
^
RangeError [ERR_INVALID_OPT_VALUE]: The value "2207514624" is invalid for option "size"
at Function.allocUnsafe (buffer.js:291:3)
at BufferList.concat (internal/streams/buffer_list.js:65:24)
at fromList (_stream_readable.js:1067:26)
at TLSSocket.Readable.read (_stream_readable.js:462:11)
at TLSSocket.socketErrorListener (_http_client.js:396:10)
at TLSSocket.emit (events.js:182:13)
at emitErrorNT (internal/streams/destroy.js:82:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
at process._tickCallback (internal/process/next_tick.js:63:19)
My code is very simple. And my dev box network speed is very good reach to 100~200MB/s download speed max, not sure it's related to the root cause.
const rs = response.readableStreamBody;
const ws = fs.createWriteStream(`F:/1TB.download${new Date().getTime()}`);
await new Promise((resolve, reject)=> {
rs.pipe(ws);
ws.on("finish", resolve);
ws.on("error", reject);
rs.on("error", reject);
});Another strange behavior between 10.14.2 and 8.14.0 is that, if I add "data" event listener to the response download stream. The 8.14.0 stables emits buffer with size 16384. However, for 10.14.2, sometimes a very large buffer larger than 2GB will be emitted, then triggers a "end" event. And the big buffer size is not stable, and I'm guessing if the large buffer reaches a limitation to alloc a new buffer, then the error will be triggered. So the question is, why sometimes have such a big buffer emitted?
v10.14.2
64-bit Windows
http, stream, buffer
Use axios to download a file from HTTPs URL by piping response readable stream into a fs Writable stream. However, the Node.js process will exit unexpectedly with following errors thrown. The errors seem from Node.js core modules related to TLSSocket, stream and buffer.
Another reason I think this should be a Node.js issue is that, with the same code, the issue can be stable reproduced in 10.14.2, but 8.14.0 doesn't have any issues.
My code is very simple. And my dev box network speed is very good reach to 100~200MB/s download speed max, not sure it's related to the root cause.
Another strange behavior between 10.14.2 and 8.14.0 is that, if I add "data" event listener to the response download stream. The 8.14.0 stables emits buffer with size 16384. However, for 10.14.2, sometimes a very large buffer larger than 2GB will be emitted, then triggers a "end" event. And the big buffer size is not stable, and I'm guessing if the large buffer reaches a limitation to alloc a new buffer, then the error will be triggered. So the question is, why sometimes have such a big buffer emitted?