Skip to content

bodyLimit/assertBodySize: stream enforcement instead of pre-buffering the body #1498

Description

@pi0x

Summary

bodyLimit() / assertBodySize() currently verify request body size by pre-buffering the whole body (up to limit) before the handler runs. This adds latency and memory overhead, and blocks streaming handlers. We can keep the exact same byte-accurate security guarantee while removing the pre-buffering, by counting bytes inline as they flow instead of ahead of time.

Current behavior

isBodySizeWithin() (src/utils/body.ts:204-237):

  1. If an honest Content-Length exceeds the limit → fail-fast (rejected before the handler runs). ✅
  2. Content-Length + Transfer-Encoding together → 400 (RFC 7230 anti-smuggling). ✅
  3. Otherwise it always drains a req.clone() stream to completion (up to limit) before returning.

Step 3 is the problem. Because bodyLimit is middleware that awaits assertBodySize before calling next():

// src/utils/middleware.ts
return async (event, next) => {
  await assertBodySize(event, limit);
  return next();
};

...a streaming handler behind bodyLimit() cannot start until the entire body has arrived and been buffered, and up to limit bytes are held in memory per in-flight request.

Enforcement itself is correct — no chunked-encoding bypass; CL+TE rejected with 400. This issue is purely about the latency/memory characteristics.

Why we can't just trust Content-Length (fast-path rejected)

An obvious idea is: if Content-Length <= limit, skip the read. This is unsafe. assertBodySize operates on a web-standard Request, where the content-length header and the body stream are independent fields — nothing guarantees they agree. While a conformant HTTP/1.1 or HTTP/2 server bounds the body to the declared length via transport framing, h3 also receives Requests from adapters, proxies, mockEvent, service workers, and synthetic construction, where the header can claim 5 while the stream delivers gigabytes. Trusting the header would open a bypass in exactly those cases. The guard must stay byte-accurate, not header-based.

Proposed improvement — streaming enforcement (no pre-buffer)

Byte-accuracy does not require pre-buffering — count actual bytes as they flow, not ahead of time:

  • Keep the cheap fail-fast 413 for an honest oversized Content-Length, and keep the CL+TE 400.
  • For everything else, wrap event.req.body in a counting TransformStream that errors the moment the running total exceeds limit, and swap in a new Request carrying the wrapped stream. Bytes are counted as the handler reads them.

Result: no clone(), no pre-buffer, memory bounded by what the handler itself holds, and streaming handlers start immediately. Same security guarantee (real bytes verified; a lying-small Content-Length is still caught mid-stream).

Tradeoffs to accept / document

  1. Overflow surfaces mid-stream, not as a pre-handler 413. For chunked / unknown-length bodies you can't know the size is too big until you've read too much, so an overflow becomes a stream error mid-read (aborted stream) rather than a tidy 413. The fail-fast 413 is preserved wherever an honest Content-Length is present.
  2. Enforcement becomes tied to body consumption. Today bodyLimit reads the clone itself, so it enforces even if the handler ignores the body; streaming enforcement only counts what the handler reads. This is unavoidable — "enforce independently of the handler" is the buffering we're removing. It's benign (an unread body consumes no app memory and is bounded by transport backpressure) but should be documented.

Scope

  • Counting TransformStream wrapper + event.req replacement in assertBodySize / bodyLimit
  • Preserve fail-fast 413 (honest oversized Content-Length) and CL+TE 400
  • Tests: content-length fast-fail, chunked overflow (mid-stream error), CL+TE 400, and a streaming handler that starts before the full body arrives
  • Docs note (the current buffering caveat in docs/2.utils/1.request.md / 9.more.md was added as an interim measure and would be updated)

Happy to open a PR for review.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions