Repository navigation
Shell snapshot reads on main: app open 7 s, t3_thread_list 10 s, a worktree check on 2 threads 1.8 s #36139
Description
On main at a044f0c0b3, the shell snapshot is slow on our database in every shape its callers use, not only the full read. Setup as in #36127: a block clone of our 49 GB statev2.sqlite, read through the real ProjectionStore, read-only, second of two passes. The clone has 7,524 threads, 3,454 active and 4,070 archived. Thread A has 7,352 runs.
| Caller | Read | Time |
|---|---|---|
App open: /api/orchestration/shell or the socket snapshot, with both client trims |
listActiveShellIndex, then 384 shells |
7.3 s |
t3_thread_list (listProjectThreads) |
{ location: "active" }, 3,454 threads |
9.9 s |
Branch rename in a shared worktree (ThreadLaunchService, read again each pass) |
{ worktreePath }, 2 threads |
1.8 s |
| The same | { worktreePath }, 2,980 threads |
11.4 s |
Shell event batch (projectShellItems) |
{ threadIds }, 2 threads including A |
1.1 s |
| Hourly storage cleanup, checkpoint restore safety | getShellSnapshot() |
20.9 s |
| Archive view | { location: "archive" } |
13.7 s, see #32805 |
Where the time goes:
-
The thread-row filters read payload JSON.
selectShellThreadRowsfiltersarchivedAt,worktreePath,settledAtandlineage.rootThreadIdwithjson_extract. Each of these reads therefore scans all 7,524 payloads (34 MB). The 2-thread worktree read spends all of its 1.75 s in this statement. For the 3,454 active threads the statement takes 5.0–5.4 s. perf(server): active thread reads filter on the archived_at column #28992 adds a column forarchivedAtonly. -
The item counts read every run's table row.
selectShellItemCountsjoinsorchestration_v2_projection_runsonrun_idonly to readstatus. For thread A in thesqlite3CLI the statement took:- 1.28 s as written;
- 0.03 s summing the counts without the join;
- 0.08 s with the same join through an index on
(run_id, status).
This statement is 1.0 s of the 2-thread batch, 2.9 s of app open and 6.0 s of the full read.
-
The shell index reads every payload.
listActiveShellIndexfilters onarchivedAtin the payload. It then extracts eight more JSON fields and runs two subqueries for each active thread. That takes 2.7 s before app open reads any shell.
Our build reads run status in the count through a (run_id, status) index, and filters archived threads on an archived_at column. We have no fix for the other JSON filters.
Measured and written by Claude (Opus 5.5) in T3 Code; posted with the owner's approval.
Activity
Dropje97 commentedon Nov 13, 2026
I checked #36146 on the database from this issue: a block clone of our 49 GB
statev2.sqlite, opened read-only. Reads went through the realProjectionStore. "Before" is2d8a843e6b^and "after" is2d8a843e6b, with migration 089's indexes. Both were run back to back, and each time is from the second of two passes.{worktreePath}){threadIds})listActiveShellIndext3_thread_list(active)getShellSnapshot()The item-count query dropped from 6.0 s to 0.1 s. Most of what remains is the shell row statement. For the 3,454 active threads it takes 5.2 s.
I timed each of its columns separately, over the same FROM and WHERE:
presentedrun join (any of its columns)latest_user_authored_message_atlatest_user_message_atpayload_jsonforked_from_run_source_thread_idlast_errorNo single thread explains this. The
presentedlookup takes at most 1 ms on any thread, so these costs grow with the number of threads in the read, not with history.There is one outlier.
latest_user_authored_message_attakes 244 ms on a thread with 7,120 user messages: it walks back through agent-authored messages until it reaches one written by the user.Measured and written by Claude (Opus 5.5) in T3 Code; posted with the owner's approval.