Skip to content

feat(server): add experimental GET /api/experimental/media/:hash and opt-in experimental-media-uri capability - #58657

Merged
kitlangton merged 0 commit into
media-backfillfrom
media-wire
Oct 25, 2026
Merged

kitlangton merged 0 commit into
media-backfillfrom
media-wire

Conversation

@kitlangton

@kitlangton kitlangton commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

What it does

Lets a client skip inline base64 when reading a session and fetch attachment bytes only when it needs them. It's experimental and opt-in at both ends: the client has to ask for refs, and the session's location needs experimental.media_blob on, which is off by default. Every other client and session gets inline base64 exactly as today.

Stack, built on the merged #58656: write (#58741) → backfill (#58742) → wire (this PR) → app (#58743).

Why

With media stored once in blob (#58741), message reads still turn every ref back into base64, so each load of a long session with screenshots re-sends every image, a third larger than the raw bytes. With refs, a read carries a short media:<hash> per file, and a client fetches each image once and caches it. Before → after bytes per read have not been measured yet.

How it works

  1. A client opts in with X-OpenCode-Capabilities: experimental-media-uri. Only session.messages, session.message and session.inbox.list have ref variants. They return stored files as media:<hash> refs with bytes, width and height. No route path is sent or stored.
  2. Whether refs are sent follows the flag in the session's own location config, not the server's working directory. If that config can't be resolved, the read logs a warning and falls back to inline.
  3. GET /api/experimental/media/:hash (generated client: experimental.media.get, OpenAPI tag experimental) serves a blob only while a message or inbox item still references it. A blob that's been released or is waiting to be swept returns 404. Responses carry immutable Cache-Control, an ETag with 304 support, nosniff and a sandbox CSP. Anything other than PNG, JPEG, GIF or WebP downloads as application/octet-stream.
  4. A ref whose blob is gone reads as the [media unavailable: <name> (<mime>)] placeholder instead of a dead link.
  5. A prompt may send a media:<hash> ref back as an attachment. The server re-admits the stored bytes unchanged, without normalizing them again, and rejects a ref nothing references anymore.

Risk

Server, protocol and the generated client. The new route and response shapes are experimental, and clients that don't opt in see no change. These still send media inline on purpose: session.context, the session.prompt response, session export and import, and the runner's model requests. Legacy /api/media/ URLs are not supported, since refs are the only form sent or accepted.

Proof

  • bun test test/media.test.ts test/message.test.ts (packages/server): 4 pass. They cover the ref and inline shape switch; headers, ETag and 304; the missing-blob placeholder; following the flag from the session's project config when the server's own directory has it off; and a 404 plus a rejected prompt for an unreferenced blob.
  • bun test for the core prompt, owned, projector, store and backfill tests: 95 pass. bun test test/promise.test.ts (packages/client): 40 pass.
  • bun run check:generated (protocol, client): clean. bun run typecheck (core, server, protocol): clean. oxlint on changed files: 0 errors.

@kitlangton kitlangton changed the title feat: add media-uri wire capability and /api/media/:hash route feat(server): add /api/media/:hash route and media-uri capability Oct 24, 2026
@kitlangton
kitlangton changed the base branch from media-storage to media-backfill October 24, 2026 16:10
@kitlangton kitlangton changed the title feat(server): add /api/media/:hash route and media-uri capability feat(server): add experimental GET /api/experimental/media/:hash and opt-in experimental-media-uri capability Oct 24, 2026
@kitlangton
kitlangton merged commit 8aea935 into media-backfill Oct 25, 2026
2 checks passed
@kitlangton
kitlangton deleted the media-wire branch October 25, 2026 23:38
@kitlangton
kitlangton restored the media-wire branch October 25, 2026 23:38
@kitlangton

Copy link
Copy Markdown
Contributor Author

Superseded by #59103 (GitHub auto-marked this merged into the stack base during a reorder; nothing reached dev).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant