Skip to content

[Bug]: Claude Read images stay inline as base64 in both event and projection rows (1.4 GB, 46% of statev2.sqlite, in one week) #26591

Open

Description

@rgrvlsk

Before submitting

Area

apps/server

Steps to reproduce

  1. Start a Claude thread on current main, or on any nightly after 2026-10-07, and ask it to Read a PNG of a few hundred KB. Repeat a few times, or let an agent work on screenshots for a day.

  2. Open statev2.sqlite read-only and measure the stored item:

    SELECT length(payload_json), json_extract(payload_json, '$.output[0].type'),
           json_extract(payload_json, '$.output[0].source.type'),
           length(json_extract(payload_json, '$.output[0].source.data'))
    FROM orchestration_v2_projection_turn_items
    WHERE type = 'dynamic_tool' AND json_extract(payload_json, '$.toolName') = 'Read'
    ORDER BY length(payload_json) DESC LIMIT 3;
  3. Run the same measurement on the matching turn-item.updated row in orchestration_events.

Expected behavior

Reading an image adds a small row to each table, and the image bytes exist once on disk in the attachments directory, like user uploads and MCP app snapshots. The Read row already carries viewedImagePath for its preview.

Actual behavior

The Read item's output is [{ type: "image", source: { type: "base64", media_type: "image/png", data: "<whole file>" } }]. The server writes that payload unchanged to both orchestration_events and orchestration_v2_projection_turn_items. Reading the same file five times stores it ten times, and event compaction never removes the event copy.

I measured one install read-only with dbstat. It is a macOS desktop install, the rows come from nightlies between 2026-10-19 and 2026-10-25, and the behavior is the same on 0.1.4-nightly.20261028.3824 and on main @ d07c0a7.

statev2.sqlite 2.98 GB
Read image results, 2026-10-19 to 2026-10-25 2,401 rows
Their payload in orchestration_events 685 MiB
Same rows in orchestration_v2_projection_turn_items 685 MiB
Share of the file taken by these two copies 46%
Largest single row 666 KiB, one PNG
Growth on two image-heavy days 315 MB and 438 MB
Growth on days without images 15 to 70 MB
User-uploaded images in the attachments directory 23 MB, none in the database

The server wrote every one of those rows after #16652 merged. This install has no Codex inputImage rows and no sizeBytes-marker rows above 20 KB, so the bloat comes only from the Claude Read path.

SQLite rewrites pages across the whole file, so every file-level backup (Time Machine, iCloud, Dropbox) copies the full multi-GB statev2.sqlite after each session. Attachment files never change after the server writes them.

Impact

Major degradation or frequent failure. Anyone whose agents read screenshots sees the database grow by hundreds of MB per image-heavy day, and it never shrinks.

Version or commit

main @ d07c0a7. Nightlies from 2026-10-19 to 2026-10-25 produced the rows, and 0.1.4-nightly.20261028.3824 reproduces it.

Environment

macOS 26, T3 Code desktop (Nightly) as host, Claude Code provider.

Where the bytes come from

All references are to main @ d07c0a7.

  • outputFromClaudeToolResult (apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.ts:3074) returns toolResult.content for tool_result. An image Read therefore becomes a dynamic_tool item whose output is the raw content array, image block included.
  • ProviderEventIngestor.ts:658 runs stripUnservedToolOutputImageBytes from fix(server): stop storing tool image bytes no client reads #16652. That helper returns served blocks untouched: if (record.type === "image" && isServed(context, record)) return record; in apps/server/src/orchestration-v2/toolOutputImageBytes.ts.
  • A block counts as served when three conditions hold. readToolOutputImage recognises it (packages/shared/src/toolOutput.ts:217), it is among the first MAX_TOOL_OUTPUT_IMAGES = 8 blocks, and its base64 is under MAX_TOOL_OUTPUT_IMAGE_BASE64_LENGTH (10 MiB raw). A Claude tool_result image block meets all three, so the helper keeps the whole file.
  • The helper keeps it because the tool-output-image asset route reads the bytes back out of the projection row. AssetAccess.readToolOutputImage calls orchestrator.getTurnItem and returns toolOutputImages(item.output)[index] (apps/server/src/assets/AssetAccess.ts:240-252).
  • The turn-item.updated event carries the same payload, so the bytes also land in orchestration_events. According to the maintainer triage on [Bug]: V2 stores tool-result image bytes twice per item, and no client reads them #16647, event compaction skips turn-item.updated.

#16652 removed the copies that no client reads. It kept the copy the asset route reads, because the row is the only place that copy exists, and the event log holds a second one.

Proposed fix

The fix is small and server-only. Write served tool-output images to the attachments directory and store a reference in the row. MCP app snapshots already work this way: apps/server/src/mcpApps/McpAppSnapshot.ts:67 writes the HTML to the attachments directory and stores the attachment id in the item output.

  1. Where stripUnservedToolOutputImageBytes runs today, write each served block's bytes to the attachments directory and replace the block with { type: "image", mimeType, attachmentId, sizeBytes }. Ids come from createDeterministicAttachmentId(threadId, sha256(bytes)) (apps/server/src/attachmentStore.ts:91). Rereading the same file in a thread then writes nothing new, and a replayed event keeps its id.
  2. AssetAccess.readToolOutputImage resolves the block by attachmentId with resolveAttachmentPathById when the id is present, and by inline data otherwise. Rows written before the change keep working.
  3. ProjectionStore.getThreadAttachmentIds (ProjectionStore.ts:5196) already collects attachment ids from dynamic_tool rows for the attachment.cleanup effect on thread delete. Adding the new key to that query makes thread deletion remove its images.

Clients do not change. They already request tool-output-image by item and index, and omitToolOutputImageData already keeps the bytes off the wire. After the fix, a Read of any image stores a row under 4 KB in each table. An image-heavy day adds the size of the distinct images read to the attachments directory, instead of about 400 MB to the database.

These are out of scope:

Related

Workaround

No workaround keeps the history. Deleting threads does not free the rows (#17845).

I can submit the PR

I would like to send this as one PR, fix(server): store served tool-output images as attachments. It covers the write at ingest, the read in the asset route, the cleanup query, and these tests:

  • A Claude Read fixture stores a row under 4 KB in both tables.
  • The asset route serves the same bytes back.
  • Reading the same image twice writes one attachment.
  • Deleting the thread deletes the attachment.

The PR changes no contracts and no client code, so the before/after row sizes are the only evidence it needs.

I have one design question before I start. Should the file write live in ProviderEventIngestor, next to the existing strip, where it covers every adapter that emits a served block? Or should it live in ClaudeAdapterV2, where the tool_result first arrives, the way the Codex adapter handles MCP app snapshots? The ingestor is the smaller diff, but I will follow whichever you prefer.

Activity

  1. Alb11747 commented on Nov 1, 2026

    @Alb11747
    Contributor

    Additional evidence for this issue's persistence problem, from a Windows install with image-heavy Codex/MCP tool use rather than only Claude Read. All sizes below are decimal GB. Measurements are from a completed 2026-11-01 SQLite backup and disposable offline copies, not new scans of the live database. Source inspection/reader checks used f8b9be7185e42e8f60fe31ed85b2ad76ec9b5af5 and 73b4ab6dddbeadc43c9e9abd8682b637bd6040cd; equivalence to the installed build was not verified.

    Measurement Result
    statev2.sqlite 25,612,169,216 bytes
    orchestration_events allocated table pages 12.920 GB
    orchestration_v2_projection_turn_items allocated table pages 7.902 GB
    Inline image/base64 payload across events and projections at least 8.35 GB
    Initial recognized-image cohort 64,652 occurrences; 31,617 unique base64 strings
    Decoded image bytes in that cohort 6.253 GB total; 3.102 GB unique; 3.151 GB repeated
    turn-item.updated 773,276 rows; 7,493,817,892 payload bytes

    Method: immutable read-only connections to the backup, dbstat for allocated table pages, UTF-8 payload accounting, recursive image detection including JSON serialized inside tool results, and hashes of decoded image bytes. These categories overlap and must not be added together. The projection alone contains 4.152 GB of direct JPEG/PNG image-block base64, predominantly MCP screenshots; corresponding event payloads retain those results too.

    An offline experiment replacing inline images with 960-pixel-longest-side AVIF thumbnails (quality 50), consistently in events and projections, then running VACUUM, reduced the file to 19,317,690,368 bytes: 6,294,478,848 bytes saved (24.58%). This combines lossy image conversion and SQLite repacking, not a measured lossless deduplication saving. Row counts and non-image fingerprints matched, SQLite quick_check and foreign-key checks passed, and typed decoding read 2,202,114 events. A small coverage pass additionally recognized 18 Claude Read image occurrences.

    This is a storage experiment, not a deployable migration: the inspected tool-image MIME allowlist rejects AVIF, and full projection rebuild, UI rendering and provider resume were not tested. Exact pixels and small text would require retaining originals or an explicit archive policy.

    Running the actual DatabaseMaintenance.compact on a separate already-minified copy deleted 1,149 superseded events but changed 19,317,690,368 to 19,336,548,352 bytes while enabling incremental auto-vacuum. It did not recover multiple GB. Most turn-item bytes here are retained content: an illustrative latest-only estimate saves just 491.7 MB and would discard required first-state/protected snapshots, so this is not evidence that the intermediate-snapshot fix in #19780 failed.

    This supports externalizing served tool images once by content hash and persisting durable attachment references across events, projections and replay, covering MCP/provider image shapes as well as Read. Optional thumbnails can reference retained originals. Snapshot deltas for unchanged large fields are another direction, subject to replay ordering. Historical reclamation needs a separately validated migration.

    Separate backup note: an online-backup freeze is consistent with #23482. The backup script held BEGIN plus a read before copying, and kept that source transaction open through destination validation. Source inspection supports a pinned-reader/WAL-admission-gate mechanism; no runtime WAL trace was captured for this incident, so causality remains unconfirmed. No additional issue is needed for that observation.

  2. EtanHey commented on Nov 10, 2026

    @EtanHey
    Contributor

    Visible symptom of the same root cause: an expanded Claude Read row renders the image twice.

    Expanded Read row showing the image twice
    • Copy 1: the viewedImagePath preview (TimelineWorkEntryRow.tsx ~L514).
    • Copy 2: FetchedToolOutput (V2ItemInspector.tsx ~L169) draws the served output.content[0] image block, followed by the structuredContent JSON.

    Neither path checks for the other. The suggested fix is to prefer the tool-output image, since it's what the model saw, and skip the disk preview. That would also cover #21296 and #27946.

    Seen on 0.1.8-nightly.20261109.4433 (macOS, Claude provider).

    Claude Opus 5.5 via T3 Code triage.

  3. phoduke commented on Nov 14, 2026

    @phoduke

    Another data point from 0.1.10-nightly.20261113.4656 on Linux. I ran the query from the report on a 3.7 GB statev2.sqlite that has been used daily for several weeks:

    • orchestration_v2_projection_turn_items has 764 Read items with an inline image, holding 141 MB of base64.
    • orchestration_events has 211 Read image events, 23 MB in total.

    So the inline copies are still written on this build. Here they are about 4% of the file. The largest payload types in the projection table are command_execution (447 MB) and dynamic_tool (350 MB, which includes the images).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions