Bug Description
Paraphrasing the HTTP spec (RFC 7230 3.3.3), the message length for a normal response body can be defined in a few ways:
- An explicit
content-length header with a fixed length (point 3 in the RFC list)
- Given a
transfer-encoding header, a transfer coding chunk that explicitly ends the body (point 5 in the list)
- If otherwise unspecified, all remaining bytes on the connection until the connection is closed (point 7 in the list)
Undici handles the last case incorrectly, making it impossible to read the HTTP response body for this simple "respond and then close the connection" case.
This doesn't come up much for big fancy servers, which tend to use keep-alive etc and explicit framing wherever they can, but it is common on quick simple HTTP server implementations, which avoid state and complexity by just streaming responses and closing the connection when they're done. It's a legitimate way to send responses according to the HTTP spec, and it is supported correctly by browsers and Node's http module, but not by Undici.
Reproducible By
Running in Node 18.0.0:
const http = require('node:http');
http.createServer((req, res) => {
res.removeHeader('transfer-encoding');
res.writeHead(200, {
// Header isn't actually necessary, but tells node to close after response
'connection': 'close'
});
res.end('a response body');
}).listen(8008);
fetch('http://localhost:8008').then(async (response) => { // Global fetch from Undici
console.log('got response', response.status, response.headers);
try {
const responseText = await response.text()
console.log('got body', responseText); // Should log the body OK here
} catch (e) {
console.log('error reading body', e); // Throws a 'terminated' error instead
}
});The server returns a simple response, just containing the default date header and a connection: close header, then sends the body, then closes the connection.
The connection: close header is for clarity - this should work equally well without that, just using res.socket.end() explicitly instead.
Expected Behavior
The above should print the status, headers, and then the response body, which is everything after the headers until the connection is closed.
This does work using fetch in browsers. To test this, run the script above (which will print the fetch error, but then keep running) then load localhost:8008 in your browser - the string loads successfully.
With that page loaded (for CORS) you can also run the equivalent command in the browser dev console, which also works and prints the response body correctly:
fetch('http://localhost:8008').then(async (response) =>
console.log(await response.text()) // Logs the body correctly, no errors
);This also works using Node's built-in HTTP module:
const http = require('http');
const streamConsumers = require('stream/consumers');
http.get('http://localhost:8008', async (res) => {
const body = await streamConsumers.text(res);
console.log('GET got body:', body); // Logs the body correctly
});Logs & Screenshots
Undici's fetch does not print the response body, instead it throws an error:
error reading body TypeError: terminated
at Fetch.onAborted (node:internal/deps/undici/undici:7881:53)
at Fetch.emit (node:events:527:28)
at Fetch.terminate (node:internal/deps/undici/undici:7135:14)
at Object.onError (node:internal/deps/undici/undici:7968:36)
at Request.onError (node:internal/deps/undici/undici:696:31)
at errorRequest (node:internal/deps/undici/undici:2774:17)
at Socket.onSocketClose (node:internal/deps/undici/undici:2236:9)
at Socket.emit (node:events:527:28)
at TCP.<anonymous> (node:net:715:12) {
[cause]: SocketError: closed
at Socket.onSocketClose (node:internal/deps/undici/undici:2224:35)
at Socket.emit (node:events:527:28)
at TCP.<anonymous> (node:net:715:12) {
code: 'UND_ERR_SOCKET',
socket: {
localAddress: undefined,
localPort: undefined,
remoteAddress: undefined,
remotePort: undefined,
remoteFamily: 'IPvundefined',
timeout: undefined,
bytesWritten: 171,
bytesRead: 90
}
}
}Environment
Ubuntu
- Node v18.0.0 with built-in fetch
- Node v16.14.2 with Undici 5.1.1
Bug Description
Paraphrasing the HTTP spec (RFC 7230 3.3.3), the message length for a normal response body can be defined in a few ways:
content-lengthheader with a fixed length (point 3 in the RFC list)transfer-encodingheader, a transfer coding chunk that explicitly ends the body (point 5 in the list)Undici handles the last case incorrectly, making it impossible to read the HTTP response body for this simple "respond and then close the connection" case.
This doesn't come up much for big fancy servers, which tend to use keep-alive etc and explicit framing wherever they can, but it is common on quick simple HTTP server implementations, which avoid state and complexity by just streaming responses and closing the connection when they're done. It's a legitimate way to send responses according to the HTTP spec, and it is supported correctly by browsers and Node's
httpmodule, but not by Undici.Reproducible By
Running in Node 18.0.0:
The server returns a simple response, just containing the default
dateheader and aconnection: closeheader, then sends the body, then closes the connection.The
connection: closeheader is for clarity - this should work equally well without that, just usingres.socket.end()explicitly instead.Expected Behavior
The above should print the status, headers, and then the response body, which is everything after the headers until the connection is closed.
This does work using fetch in browsers. To test this, run the script above (which will print the fetch error, but then keep running) then load
localhost:8008in your browser - the string loads successfully.With that page loaded (for CORS) you can also run the equivalent command in the browser dev console, which also works and prints the response body correctly:
This also works using Node's built-in HTTP module:
Logs & Screenshots
Undici's fetch does not print the response body, instead it throws an error:
Environment
Ubuntu