Skip to content

feat(server): stream fs.read responses with HTTP Range support - #53088

Merged
kitlangton merged 4 commits into
v2from
fs-read-range
Oct 7, 2026
Merged

kitlangton merged 4 commits into
v2from
fs-read-range

Conversation

@kitlangton

@kitlangton kitlangton commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Why

GET /api/fs/read/* previously buffered the entire file into memory via fs.readFile and returned a plain 200 OK byte array without Accept-Ranges, Content-Range, ETag, or Last-Modified headers. Native media clients such as iOS AVPlayer could not stream or seek video and had to download whole files upfront, while lexical or symlink path escapes surfaced as unhandled 500 defects instead of 404 responses.

What Changes

FileSystem.read now resolves and confines the target path, stats the file once, and returns its size, mtime, MIME type, and a lazy disk stream without buffering. The fs.read handler builds the response from that one stat: it streams exactly Content-Length bytes (a file that grows mid-read is not over-sent), computes a weak ETag and Last-Modified from the same size and mtime, and answers single-range and conditional requests by comparing headers.

Every 200, 206, and 304 carries Cache-Control: no-cache, no-transform. no-cache keeps browsers and Electron from heuristically caching on Last-Modified, so the app revalidates and sees agent edits immediately; no-transform keeps the compression middleware from re-encoding the body, so Content-Length and byte ranges always describe the file on disk.

Request Response Headers
GET /api/fs/read/<path> 200 OK (streamed) Accept-Ranges: bytes, Content-Length, Content-Type, weak ETag, Last-Modified, Cache-Control
HEAD /api/fs/read/<path> 200 OK (no body, file not opened) Same as GET
Range: bytes=start-end / bytes=start- / bytes=-suffix 206 Partial Content (streamed slice) Content-Range: bytes start-end/size, Content-Length (end - start + 1), plus the 200 headers
Unsatisfiable Range (start >= size, start > end, bytes=-0, or empty file) 416 Range Not Satisfiable Content-Range: bytes */size, Accept-Ranges: bytes
Multi-range (bytes=0-1, 3-4) or malformed Range 200 OK (full stream) Standard 200 OK headers
Matching If-None-Match or If-Modified-Since 304 Not Modified ETag, Last-Modified, Cache-Control
Range with matching If-Range (ETag or Last-Modified) 206 Partial Content As above
Range with mismatched If-Range 200 OK (full stream) Standard 200 OK headers
Path escape (.., symlink outside location) or non-file 404 Not Found JSON FileNotFoundError
flowchart TD
  Req["GET / HEAD /api/fs/read/*"] --> Resolve["FileSystem.read: confine path + one stat"]
  Resolve -- "missing / escape / non-file" --> E404["404 FileNotFoundError"]
  Resolve -- "regular file (size, mtime, stream)" --> Cond{"If-None-Match / If-Modified-Since match?"}
  Cond -- "yes" --> E304["304 Not Modified"]
  Cond -- "no" --> IsGetRange{"GET with Range & matching If-Range?"}
  IsGetRange -- "no (or HEAD / If-Range mismatch)" --> S200["200 OK: stream(offset 0, size bytes)"]
  IsGetRange -- "yes" --> Parse{"parseRange(range, size)"}
  Parse -- "malformed / multi-range" --> S200
  Parse -- "unsatisfiable" --> E416["416 Range Not Satisfiable (bytes */size)"]
  Parse -- "{ start, end }" --> S206["206 Partial Content: stream(start, end - start + 1 bytes)"]
Loading

Scope

This PR owns streaming and single-range HTTP Range / conditional request support for confined location-relative GET /api/fs/read/* (and router-dispatched HEAD) requests, plus returning 404 FileNotFoundError when a path escapes the location or targets a non-file. Absolute-path reads, multipart byte ranges, and signed URLs remain out of scope.

Verification

bun --cwd packages/core test test/location-filesystem.test.ts
bun --cwd packages/server test
bun --cwd packages/server run probe:workerd
bun --cwd packages/client run check:generated
bun --cwd services/www run check:generated
bun run typecheck
bun run lint

# End-to-end curl verification against a standalone server on an ephemeral port
curl -sI -u opencode:secret "$URL"
curl -s -u opencode:secret -H 'Range: bytes=0-1' -D - -o /dev/null "$URL"
curl -s -u opencode:secret -H 'Range: bytes=-128' -D - -o /dev/null "$URL"
curl -s -u opencode:secret -H 'Range: bytes=5000-6000' -D - -o /dev/null "$URL"
curl -s -u opencode:secret -H "If-None-Match: $ETAG" -D - -o /dev/null "$URL"
curl -s -u opencode:secret -H 'Range: bytes=0-99' -H 'If-Range: W/"stale"' -D - -o /dev/null "$URL"
  • packages/core/test/location-filesystem.test.ts: 10 pass, 0 fail.
  • packages/server full suite: 70 pass, 5 skip, 0 fail. test/fs.test.ts covers every Range form, 416, multi-range and malformed fallback, 304, If-Range, HEAD, Cache-Control, path confinement, a gzip-accepting request for a compressible text file that keeps its exact Content-Length, 4 MiB byte-for-byte full and partial reads, and that HEAD leaves no open file handle while an aborted download closes its handle (checked through /proc/self/fd on Linux and lsof on macOS).
  • probe:workerd, both generated-output checks, bun run typecheck (36/36), and bun run lint pass.
  • End-to-end curl output against a 1,000-byte sample.mp4:
=== HEAD (curl -sI) ===
HTTP/1.1 200 OK
cache-control: no-cache, no-transform
etag: W/"3e8-1a113e33c97"
last-modified: Wed, 07 Oct 2026 01:03:41 GMT
accept-ranges: bytes
content-type: video/mp4
content-length: 1000

=== Range: bytes=0-1 (iOS probe) ===
HTTP/1.1 206 Partial Content
cache-control: no-cache, no-transform
etag: W/"3e8-1a113e33c97"
last-modified: Wed, 07 Oct 2026 01:03:41 GMT
accept-ranges: bytes
content-type: video/mp4
content-range: bytes 0-1/1000
content-length: 2

=== Suffix Range: bytes=-128 ===
HTTP/1.1 206 Partial Content
cache-control: no-cache, no-transform
etag: W/"3e8-1a113e33c97"
last-modified: Wed, 07 Oct 2026 01:03:41 GMT
accept-ranges: bytes
content-type: video/mp4
content-range: bytes 872-999/1000
content-length: 128

=== Unsatisfiable Range: bytes=5000-6000 (416) ===
HTTP/1.1 416 Range Not Satisfiable
accept-ranges: bytes
content-range: bytes */1000
transfer-encoding: chunked

=== If-None-Match: W/"3e8-1a113e33c97" (304) ===
HTTP/1.1 304 Not Modified
cache-control: no-cache, no-transform
etag: W/"3e8-1a113e33c97"
last-modified: Wed, 07 Oct 2026 01:03:41 GMT

=== If-Range mismatch (200) ===
HTTP/1.1 200 OK
cache-control: no-cache, no-transform
etag: W/"3e8-1a113e33c97"
last-modified: Wed, 07 Oct 2026 01:03:41 GMT
accept-ranges: bytes
content-type: video/mp4
content-length: 1000

@kitlangton
kitlangton marked this pull request as ready for review October 7, 2026 00:39
Build the response from the size, mtime, and stream of the single stat
FileSystem.read already performs instead of re-statting through
HttpPlatform. Full reads are bounded to the stat size, conditional and
If-Range checks compare headers directly, and Cache-Control: no-transform
replaces the content-encoding workaround for keeping compression away
from Content-Length. This drops the HttpPlatform wiring in routes.
Without Cache-Control, Last-Modified lets browsers and Electron cache
file reads heuristically, so the app could show a stale file for minutes
after an agent edit. no-cache keeps the validators but forces a
conditional request each time.
@kitlangton
kitlangton merged commit 74678c4 into v2 Oct 7, 2026
11 checks passed
@kitlangton
kitlangton deleted the fs-read-range branch October 7, 2026 01:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant