Undici incorrectly rejects responses with trailer header but no matching trailer #1418
Description
Bug Description
When a server sends a Trailer: X header, it's saying that it may send an X trailer field after the response. Undici treats this as a commitment, and throws an error if the response has no such trailer, for no apparent reason.
This seems to be an intentional check here discussed originally in #432. There's no reference to any spec there that says this is necessary though (and I can't find one) just discussion suggesting maybe it's odd. Undici should follow the spec, not reject responses that seem weird.
There are real servers (e.g. all IPFS nodes, as noted in that issue at the time) that are broken by this in many cases, since they return trailer: X-Stream-Error with all streaming responses, to indicate that they may add stream errors after the body in some cases.
Browser fetch & node HTTP don't enforce this, and no spec says this is invalid - this check should just be removed imo.
Reproducible By
const http = require('http');
http.createServer((req, res) => {
res.writeHead(200, {
trailer: 'test'
});
res.end('body');
}).listen(8008);
// Global Undici fetch from Node 18:
fetch('http://localhost:8008').then((res) => {
console.log('got status', res.status);
return res.text();
}).then((body) => {
console.log('got body', body);
})
.catch(console.error);This prints:
TypeError: terminated
at Fetch.onAborted (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/fetch/index.js:1900:49)
at Fetch.emit (node:events:527:28)
at Fetch.terminate (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/fetch/index.js:79:10)
at Object.onError (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/fetch/index.js:2034:34)
at Request.onError (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/core/request.js:233:27)
at errorRequest (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/client.js:1728:13)
at Socket.onSocketClose (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/client.js:1002:5)
at Socket.emit (node:events:527:28)
at TCP.<anonymous> (node:net:715:12) {
[cause]: TrailerMismatchError: Trailers does not match trailer header
at Parser.onMessageComplete (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/client.js:864:30)
at wasm_on_message_complete (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/client.js:383:30)
at wasm://wasm/0002afd2:wasm-function[45]:0x8dc
at wasm://wasm/0002afd2:wasm-function[56]:0x1ad3
at wasm://wasm/0002afd2:wasm-function[55]:0xcd7
at wasm://wasm/0002afd2:wasm-function[21]:0x4e4
at Parser.execute (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/client.js:515:22)
at Parser.readMore (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/client.js:484:12)
at Socket.onSocketReadable (/tmp/tmp.EolblgSMPd/node_modules/undici/lib/client.js:925:10)
at Socket.emit (node:events:527:28) {
code: 'UND_ERR_TRAILER_MISMATCH'
}
}Expected Behavior
This should work as normal, as if the trailer field was not present.
This works correctly in Node HTTP, node-fetch & browser fetch, so this produces regressions for everybody using a fetch polyfill with Node 18.
Environment
Ubuntu 20.04, using global from Node 18.0.0 or Node 16.14.2 with Undici 5.1.1.
Additional context
Looks like this was causing errors in the past too, e.g. #880 says it broke all requests to pinterest.