Skip to content

fix(core): omit read media from the experimental timeline - #56989

Draft
kitlangton wants to merge 1 commit into
attachment-hydrationfrom
strip-read-media
Draft

kitlangton wants to merge 1 commit into
attachment-hydrationfrom
strip-read-media

Conversation

@kitlangton

@kitlangton kitlangton commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

What it does

Stacked on #57729. Merge that first, then retarget this PR to beta.

The experimental timeline route (GET /api/experimental/session/:sessionID/timeline) also omits inline data: files from read-tool results. The app already opens those images by path. Execute screenshots stay inline.

The stable API is unchanged. GET /api/session/:sessionID/message, single-message reads, exports and transfers keep every byte. No stored rows change.

How it works

When the store builds the timeline projection, it filters data: files out of completed or errored read tool content. If that leaves the content empty, it keeps the read tool's usual "Image read successfully" text. The route's schema description says so.

Risk

Desktop/web timeline only. An inline read image is no longer in the timeline payload, so it is only viewable through the path-based file viewer.

Proof

Latest 40 messages on a HOME-shaped fixture: 2,604,594 → 57,720 B.

  • bun test test/message.test.ts (server): 3 pass. The stable list returns the read media; the timeline omits it and keeps execute media. session.message.get returns full bytes.
  • tsc -b in core, server and protocol: pass.
  • The protocol, client and www OpenAPI generators were run. The stable route's entry is identical to origin/beta.

@kitlangton

Copy link
Copy Markdown
Contributor Author

Session-load coordination from Argon’s HOME audit: read results hold 412.1 MB, execute results 36.3 MB, and user.files 53.5 MB. I am checking a small follow-up extending this omission to execute while retaining full single-message/model-context reads, rather than duplicating this PR. User attachments need a separate on-demand renderer/restore path; #56991 currently rehydrates message lists, so storage externalization alone will not remove their wire bytes.

@kitlangton

Copy link
Copy Markdown
Contributor Author

Execute follow-up is ready to fold into this PR: commit 8737289d62 on execute-media, based on this PR's a2b18aa981 head. No competing read-media PR opened.

It extends the same list-only omission to execute, preserving full single-message/model-context media. File-only execute output gets an empty text placeholder to satisfy the nonempty content schema without inventing a success message. The existing store keeper is table-driven for read and execute.

Synthetic 40-message page with one 2.55 MB execute image, measured through real ServerFetch and generated client:

Metric Current read-only strip With execute strip
Response bytes 2,573,722 23,656
Handler/client median 6.868 ms 3.360 ms
Range 4.518–9.657 ms 2.808–5.280 ms

Payload drops 99.08%. One warmup and seven measured runs per variant on M5 Max through HiveMind. Batches were consecutive, not interleaved; timing ranges overlap. Payload is the accepted win, not a claim of faster newest-message painting or Argon's estimated 360 ms decode saving.

Measured times in order, excluding warmup:

  • Before: 6.867667, 6.909166, 9.578750, 5.019791, 5.098834, 9.657125, 4.518417 ms.
  • After: 4.624792, 3.159625, 3.262000, 5.279916, 4.669750, 2.807625, 3.359875 ms.

Execute media payload comparison

Reproduce from packages/server with bun script/benchmark-execute-media.ts before/after the commit. It uses an in-memory database, temporary global paths and in-process HTTP; no listening server. Core store tests: 4 pass / 25 assertions. Full bun run check: 34 tasks pass.

Argon's HOME baseline attributes 412.1 MB to read, 36.3 MB to execute, and 53.5 MB to user files. This does not solve user-file hydration: previews and edit/fork/resend need original bytes. Coordination for the reserved session-ui attachment hook is recorded on OCD-58.

@kitlangton kitlangton changed the title fix(core): omit read tool image base64 from message list responses fix(core): omit read and execute media from message pages Oct 21, 2026
@kitlangton
kitlangton changed the base branch from dev to beta October 21, 2026 10:27
@kitlangton kitlangton changed the title fix(core): omit read and execute media from message pages fix(core): make read media omission opt-in Oct 21, 2026
@kitlangton
kitlangton changed the base branch from beta to attachment-hydration October 21, 2026 12:38
@kitlangton
kitlangton force-pushed the strip-read-media branch 2 times, most recently from 61f59df to 292fcfa Compare October 21, 2026 12:56
@kitlangton
kitlangton force-pushed the attachment-hydration branch from 31876e3 to 976943f Compare October 23, 2026 22:42
@kitlangton kitlangton changed the title fix(core): make read media omission opt-in fix(core): omit read media from the experimental timeline Oct 23, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant