Skip to content

feat(app): hydrate user attachments on demand - #57729

Closed
kitlangton wants to merge 4 commits into
betafrom
attachment-hydration
Closed

kitlangton wants to merge 4 commits into
betafrom
attachment-hydration

Conversation

@kitlangton

@kitlangton kitlangton commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What it does

Opening a session with many pasted images downloads every image's bytes up front. The desktop/web timeline now loads them from a new experimental route, GET /api/experimental/session/:sessionID/timeline (experimental.session.timeline, SDK client.experimental.session.timeline; client.message is unchanged). It returns user attachments as opencode-attachment: references, and the app loads the originals near the viewport or on click.

The stable API is unchanged. GET /api/session/:sessionID/message has no new parameter, and its OpenAPI entry matches beta byte for byte.

How it works

  1. The timeline route takes the same paging params as the stable messages route and shares its handler. The only difference is that the store replaces inline user file bytes with opencode-attachment:<sessionID>/<messageID>/<index> and empty data. The response schema marks that format as experimental.
  2. Only the app's timeline data store calls the new route. Revert, Fork, image hydration and single-message reads keep the stable routes and get full bytes. Exports and model context are untouched.
  3. A per-server LRU (32 messages, 16 MiB) holds the originals for image display, Revert and Fork. Concurrent reads share one request, and a failed read can be retried.
  4. AGENTS.md gets one line: new or uncertain API surface goes under /api/experimental/ first.

#56989 is stacked on this PR and adds read-tool media omission to the same route.

Risk

Desktop/web timeline only. If hydration fails, an image shows a sized placeholder and retries on its next intersection. Explicit clicks surface the error. Other clients never see references.

Proof

200-prompt page on a HOME-shaped fixture: 9,920,082 → 145,676 B. No reliable newest-paint improvement is claimed.

Opt-in page measurements

  • bun test test/message.test.ts (server): 3 pass. The stable list returns inline bytes; the timeline returns references and pages correctly.
  • App attachment/prompt/export units: 16 pass.
  • session-ui hydration component tests: 3 pass.
  • tsc -b in core, server, protocol, client, app and session-ui, plus the full typecheck in the pre-push hook: pass.
  • The protocol, client and www OpenAPI generators were run. The stable route's entry and SessionMessagesResponse are identical to origin/beta.

@kitlangton

Copy link
Copy Markdown
Contributor Author

Final head: 67a0ebd7911757ac7a061adeac1aaab6f9519896. CI run has finished.

  • Linux/Windows builds, unit tests, and session-ui component suites pass.
  • App E2E and app component suites still fail. Categories include cached-timeline focus/disposal, busy-tab activity, pending-steer layout, activity chips, virtualizer reconnect, websearch, colors/artifact and mobile navigation. These overlap earlier independent-branch failures; without a clean-base run, they are not established as pre-existing. The PR remains CI-blocked and unmerged.
  • The first run caught an export keeper expecting the old query arguments. Its new explicit attachments=inline expectation is fixed; six targeted tests pass and both full CI unit jobs now pass.

Remaining newest-paint cost

With all four improvements combined on a separate measurement branch, 21 interleaved pairs still show no reliable newest-paint effect from changing the server projection: 925.9→933.8 ms. Median after-sample: 1.2 ms initiation, 210.8 wait (including 180 ms simulated RTT), 7.5 server, 17.8 body transfer, 0.3 JSON decode, 1.4 store insertion, 232.2 store-to-DOM and 462.5 DOM-to-visible paint.

The remaining 695 ms render/settling slice belongs to timeline/UI. Separate opt-in Chrome profiling attributes 118 ms sampled CPU to markdown and 106 ms to virtualizer stacks, with overlapping native style/layout work. Those CPU figures are not additive wall-time slices. The combined build is held fixed between variants; this is not an all-four-before/after app comparison. No further optimization attempted.

@kitlangton
kitlangton force-pushed the attachment-hydration branch from 31876e3 to 976943f Compare October 23, 2026 22:42
@kitlangton

Copy link
Copy Markdown
Contributor Author

Closing: this was a memory optimization only.

@kitlangton kitlangton closed this Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant