Skip to content

feat(core): store session media in a content-addressed media_blob table - #56991

Draft
kitlangton wants to merge 7 commits into
attachment-hydrationfrom
media-blob-table
Draft

kitlangton wants to merge 7 commits into
attachment-hydrationfrom
media-blob-table

Conversation

@kitlangton

@kitlangton kitlangton commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #57729 (attachment-hydration). Retarget to beta after it merges.

What it does. Session images live as base64 inside session_message.data, so 85% of a 15 GB opencode.db is image bytes and 19% of those bytes are exact duplicates. This moves media of 1 KiB or more into a content-addressed media_blob table and leaves a blob:sha256:<hash> reference in the row.

How it works.

  1. SessionProjector insert/update calls SessionMediaBlob.externalize inside the projection transaction. User file data and tool file URIs that are exactly data:<mime>;base64,… become refs; identical bytes share one row (INSERT … ON CONFLICT DO NOTHING).
  2. Every read that returns media hydrates first (SessionHistory.decodeMessageRows): model context, store.message, inline store.messages (export, revert), inbox redelivery, title generation and the output-limit check. Fork copies refs as-is.
  3. Reference-mode store.messages (feat(app): hydrate user attachments on demand #57729) swaps user attachments for opencode-attachment: placeholders before hydration, so it never reads their blobs.
  4. The migration only creates the table. Existing rows keep working unchanged; packages/core/script/migrate-media-blobs.ts is an offline, re-runnable backfill (server stopped) with optional VACUUM INTO.

Risk. Server data layer.

  • Blobs are never deleted (revert and session delete leave orphans) until a separate cleanup lands.
  • A missing blob makes that message fail to decode instead of showing a broken image.
  • Reference mode still hydrates assistant tool media (read/execute screenshots); skipping those needs fix(core): omit read media from the experimental timeline #56989's omission on top.
  • A tool or MCP file whose URI literally is blob:sha256:<existing hash> would be hydrated. User files can't hit this (base64 schema rejects :).
  • Uses node:crypto createHash; the workerd bundle probe passes, but it isn't exercised under workerd at runtime.

Proof. Copy of a 22k-row session (store.messages, 40 rows, warm, median of 7):

page mode before backfill after
heavy user images (2.1 MB) inline 32.9 ms 29.8 ms
heavy user images reference 1.5 ms 0.3 ms
heavy tool screenshots (16.8 MB) inline 7.1 ms 4.6 ms
heavy tool screenshots reference 5.6 ms 4.1 ms
  • bun test test/session-projector.test.ts test/session-store.test.ts test/session-title.test.ts test/session-runner-message.test.ts test/session-prompt.test.ts test/database-migration.test.ts in packages/core: pass. Includes a test that drops media_blob and checks reference-mode lists still load, and that inbox redelivery, store.message and inline lists return the original bytes.
  • bun test test/message.test.ts in packages/server: pass.
  • Pre-push bun run check (lint + typecheck, 36 packages): pass. bun run probe:workerd: pass.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant