Repository navigation
fix(opencode): huge OpenCode user messages no longer crash the usage scan - #27679
juliusmarminge wants to merge 4 commits into
Conversation
…can pages Co-Authored-By: Claude Opus 5.5 <[email protected]>
OpenCode stores user messages of up to 150 MB (mostly summary diffs), and the usage reader selected the full `data` column for every row, so one 64-row page could OOM the server on every Usage page load. Filter non-assistant and malformed rows in SQL (guarded by json_valid) and return only the usage fields as a small json_object, so the bytes a page materializes no longer scale with message bodies. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
ApprovabilityVerdict: Approved at Macroscope's review found this PR approvable — The OpenCode usage scan now filters messages and extracts only usage fields in SQLite, preventing huge message bodies from overwhelming the JavaScript heap. Read-only access and existing scan safeguards remain intact, with reported large-history verification supporting the limited risk. You can add or adjust custom eligibility rules. Learn more. |
Thread transfer impact✅ Thread transfer remains within every enforced ceiling.
Baseline: Scenario and decoded snapshot size10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.
Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed. |
|
No actionable comments were generated in the recent review.
|
| Layer / File(s) | Summary |
|---|---|
Filter and project usage rows packages/provider-opencode/src/server/usage.ts, packages/provider-opencode/src/server/usage.test.ts |
The query retains rows with no role or an assistant role, and projects selected usage fields. The test adds large user-message patches and checks the assistant usage result and largest raw query-result size. |
Priority:
Estimated code review effort: 2 (Simple) | ~10 minutes
Change: Bug fix · Severity of issue fixed: Medium
Suggested labels: size:XS
Suggested reviewers: t3dotgg, macroscopeapp, yash-singh1
Merge Risk: 
Minimal · up to bee4d
-
Usage scans return compact records without an identified merge-blocking risk.
Pre-merge checks: 
4
-
Full details: Description check
-
Explanation
The description includes the required Problem, Change, Scope and approval, and Verification sections. It explains the failure, implementation, linked issue, focused tests, runtime results, limitations, and agent details.
Full details: Linked Issues check
-
Explanation
Issue
#27667requires the OpenCode usage scan to avoid materializing large user-message bodies while preserving assistant usage.usage.tsnow filters valid text JSON to rows with no role orrole = 'assistant', and projects only usage fields withjson_objectbefore SQLite returns rows. The new test inserts 32 large user diffs beside an assistant row, verifies 1,500 tokens, and bounds the largest raw result below 6 MiB. The existing parser, paging, and legacy paths remain intact in the reviewed diff.
Full details: Out of Scope Changes check
-
Explanation
The reviewed changes are limited to the OpenCode usage SQL query and a focused regression test. The filter, field projection, malformed-row guard, and test instrumentation directly support issue
#27667. No unrelated product behavior or database mutation is introduced.
-
Finishing Touches
-
📝 Generate docstrings
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Comment @coderabbitai help to get the list of available commands.
OpenCode usage scans materialized complete message bodies and queried empty rowid ranges. Select in-window rowids first and project usage fields in SQLite, keeping message bodies off the JS heap. Read each database in one snapshot and record scan row and page counts. On a read-only snapshot of a 3.32 GB real database, five fresh-process runs measured 2.33x faster 30-day reads and 2.82x faster 90-day reads with identical records. These are cold application-cache measurements; OS caches were not cleared. Local check:changed and CI passed. Adapted SQL projection from #27679 and window selection/snapshot handling from #26613. Model and harness: GPT 6.1 Sol through Codex in T3 Code. Co-authored-by: Julius Marminge <[email protected]> Co-authored-by: Tymoteusz Bielski <[email protected]> Co-authored-by: maria-rcks <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]>
|
Thanks, Julius, for the SQL projection and the thorough reproduction. #33612, merged as bbdc7dca32, incorporates this fix: it filters and projects usage fields in SQLite so large user message bodies stay off the JS heap. Closing as covered. |
) OpenCode usage scans materialized complete message bodies and queried empty rowid ranges. Select in-window rowids first and project usage fields in SQLite, keeping message bodies off the JS heap. Read each database in one snapshot and record scan row and page counts. On a read-only snapshot of a 3.32 GB real database, five fresh-process runs measured 2.33x faster 30-day reads and 2.82x faster 90-day reads with identical records. These are cold application-cache measurements; OS caches were not cleared. Local check:changed and CI passed. Adapted SQL projection from pingdotgg#27679 and window selection/snapshot handling from pingdotgg#26613. Model and harness: GPT 6.1 Sol through Codex in T3 Code. Co-authored-by: Julius Marminge <[email protected]> Co-authored-by: Tymoteusz Bielski <[email protected]> Co-authored-by: maria-rcks <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit bbdc7dc)
…4ff2a2 (#2836) * fix(server): Codex /goal no longer tries to create the goal twice (#33260) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit 1a43f4870d4d27e6514b7485a8742027e2f48b34) * fix(server): steering during a Claude compaction shows it stopped, not failed (#33291) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit 2b65ffb641f152fadf443096bad7f4a0677caae4) * fix(server): price Antigravity Gemini Flash effort variants (#13809) Co-authored-by: Julius Marminge <[email protected]> Co-authored-by: Claude Fable 5.1 <[email protected]> Co-authored-by: maria-rcks <[email protected]> (cherry picked from commit 9a3f8db9b39704cd1adb522babda23c21de8e1e4) * fix(server): Antigravity install no longer fails on a stale Windows Crashpad pipe (#26374) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit 12085f0b0c1229198bbcbf47c63135f4931936f6) * fix(desktop): agents see visible:true while the desktop shows their tab (#23589) Co-authored-by: PR Batch Tester <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit 5ae957848a3bac1324213f68dc3e091b23f8a69b) * fix(server): Claude threads can allow an MCP app for the session (#33707) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit 4c361199f47f6ea32a2c14835df392640d0f7c63) * fix(server): detaching a session no longer strands its pending approval (#33535) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit 577c524089368176e00b331a8217b642fd6d06a5) * fix(server): quitting no longer restarts the relay client mid-shutdown (#33628) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit 5636b5666d3fb9b2b280dbe8a0c5cb42874f1bcd) * fix(server): usage prices Claude Mythos 5.2 from its Bedrock rate (#33721) (cherry picked from commit 0ca73a19f06bc82d729f6d1c1c8903aa22d78868) * fix(acp-registry): commands past the 128th are no longer dropped (#33768) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit 7171fdd7716b05dc1b4be40c26ac78e8bb35f80c) * perf(opencode): bound usage reads to in-window messages (#33612) OpenCode usage scans materialized complete message bodies and queried empty rowid ranges. Select in-window rowids first and project usage fields in SQLite, keeping message bodies off the JS heap. Read each database in one snapshot and record scan row and page counts. On a read-only snapshot of a 3.32 GB real database, five fresh-process runs measured 2.33x faster 30-day reads and 2.82x faster 90-day reads with identical records. These are cold application-cache measurements; OS caches were not cleared. Local check:changed and CI passed. Adapted SQL projection from pingdotgg/t3code#27679 and window selection/snapshot handling from pingdotgg/t3code#26613. Model and harness: GPT 6.1 Sol through Codex in T3 Code. Co-authored-by: Julius Marminge <[email protected]> Co-authored-by: Tymoteusz Bielski <[email protected]> Co-authored-by: maria-rcks <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit bbdc7dca320ee4fbc7131498aaffef2fa1eb3d41) * fix(opencode): long command turns no longer fail when the command request times out (#27677) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit eff45664fae739f0efadaee13acc145ddf0d0b66) * fix(acp): OMP threads show html_render pages (#33677) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit d47591f613a28818d583bd3003f950ccd5060951) * fix(acp): OMP device writes resolve GOAT's MCP server name Upstream: d47591f613 (#33677) matched T3's server key; GOAT injects its server as goat. The device parsing moves beside the shared fallback-input parser to keep runtimeModel.ts within its code budget. * fix(server): desktop-shown browser tabs report visible from GOAT's capacity check Upstream: 5ae957848a (#23589) relied on an isVisible helper from upstream's idle-tab capacity change; GOAT's equivalent lives in ServerBrowserCapacity, which now exposes the visibility it already used for idle tabs. The upstream test moves to the capacity test file that owns the shown-tab fixture. * fix(server): opening the branch menu no longer wipes PR status A local-only refresh cached `pr: null` for the current branch, so readers treated the branch as read with no pull request. Leave the remote cache untouched and read the parts instead. Upstream: a22068f3711 (#33263) * fix(server): one finished turn looks up its branch's pull request once A turn end commits several checkpoint and run events at once, and each queued its own branch lookup. Coalesce requests per thread so the batch shares one lookup, while a request that arrives after the lookup starts still queues a later refresh. Upstream: f8fbc199ae1 (#33752) * refactor(server): move git branch-list and action-summary helpers into their own modules Upstream: none * fix(server): Commit, push & PR no longer pushes the default branch into itself (#33695) GOAT checks the push destination before a commit, push & PR when origin/HEAD is stale, and publishes a branch that tracks the default branch under its own name. Upstream: ad91636f0b48fe41119c888ebe937acb5167f0ee (#33695) * fix(forgejo): find merged pull requests whose head branch was deleted (#33675) A deleted head branch arrives as refs/pull/<N>/head with the name in label; read it when the repository is known and the pull request is not an AGit topic. Upstream: b37c7f6fa41e36f259ce3b8ae7821e0a3a22793c (#33675) * fix(server): stop PR watch flapping while workflows await approval (#24599) A check awaiting approval is no longer a failure. The watch records each approval hold until its workflow run finishes, announces it once, and an unreadable approval list neither passes nor reports missing checks. Upstream: f6a32dbc7f9047cce5e6a83062aa1b8031a03081 (#24599) * fix(server): background command results reach a running Codex goal Steer server-raised command notices into a running turn whose provider thread has an active goal, so a background result is seen instead of waiting behind the goal run. Other notices keep queueing. Upstream: 4c8eb1dfd5 (#33268) Implemented with deepseek-v4.1-flash-free through OpenCodex. * fix(server): a steer that restarts a completion wake no longer strands later subagent results (#33469) Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit 8d54c8b1a7ab58528ec7fcddb30fdc68a6e68ea0) * feat(models): add Claude Fable 5.2 (#33500) Co-authored-by: Claude Opus 5.5 <[email protected]> * fix(antigravity): resolve model names from usage metadata (#33661) (cherry picked from commit e2aa223229e42dd742b6dd05a9fad61775231d88) * perf(cursor): improve account usage scanning (#33636) Normalize Cursor usage pages as they arrive, retain bounded raw page data, and preserve count-proven boundary deduplication and stable event keys. Use direct hex hashing, limit concurrent account refreshes to two, reuse unchanged cached records, and persist retention pruning. Validation: check:changed passed with 61 focused tests; 40 differential reader cases matched the original output. CI and agentic reviews passed on the merged head. GPT 6.1 Sol via Codex in T3 Code. * fix(ssh): tunnel readiness no longer counts ssh's connection time (#33554) Upstream: 4d9caf14ee (#33554) * fix(ssh): Intel Mac hosts no longer try to download a missing goat archive (#33325) Gate the remote runner's archive download on the supported release platform keys, so Darwin x86_64 fails before fetching instead of 404ing. Upstream: 56064fc4f6 (#33325) * fix(server): allow requested WebMCP writes (#26570) preview_evaluate's guidance said page tools only read. Say they write on request, and keep results on html_render. Upstream: 0c832589a0 (#26570) * fix(server): thread ids with parentheses stay readable in MCP links (#33255) A thread id can carry parentheses, and a raw Markdown destination ends at the first one. Labeled links now bracket a destination that holds them, and the parser reads angle-bracketed destinations back. Upstream: cd87cce900 (#33255) * fix(server): report update download diagnostics Upstream: 633438d54f (#33239) * fix(server): fail Claude turns when the query exits early Upstream: bcee0f5d2872ba193c66ed9ba979bbe424f89ff9 (#33008) * docs(chase): record cycle 261110 and close cycle 261109 * chore(server): keep chase helpers module-private and type the completion id allocator Knip flagged five exports only their own modules use, and the delegated-completion module typed the allocator's error channel as unknown. * refactor(server): keep chase ports within code budgets Cherry-picked upstream tests and helpers grew six budgeted files. Move the new and self-contained tests into focused test files and testkits, move the detach thread-id helper, compaction outcome and MCP elicitation options into neighbouring modules, and narrow the SSH readiness test to the ready case. * fix(server): keep chase ports bounded and re-announce later approval waits Review of #2836 found two ports that removed a bound GOAT keeps on server memory: the ACP command probe now stops at 1,024 distinct commands (up from upstream's old 128), and the OpenCode usage reader pages rowids by keyset instead of listing the whole window. A PR watch also re-announces a workflow approval wait once earlier holds on the same head have settled. * Revert "fix(server): Claude threads can allow an MCP app for the session (#33707)" The session grant keyed only on server name and message, so a different app or a changed request schema reused it without a card, and the grant set had no bound. Deferred to #2832 with the review findings. The cycle card now counts 73 ported and 68 deferred. * docs(server): state the bound on queued thread pull request refreshes * Revert "fix(server): stop PR watch flapping while workflows await approval (#24599)" An approval hold stayed until a finished check for that exact run appeared, so a cancelled or vanished gated run left a hold that never cleared, had no count limit and blocked checks-passed. The re-announce fix built on it goes too. Deferred to #2832; the cycle card now counts 69 ported and 72 deferred. * test(server): computer-use goal replay expects the new /goal start prompt Upstream 1a43f4870d changed the prompt a /goal turn starts with; GOAT's computer-use replay still expected the old text, which failed goat-test-server on #2836. --------- Co-authored-by: Theo Browne <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]> Co-authored-by: Ephraim <[email protected]> Co-authored-by: Julius Marminge <[email protected]> Co-authored-by: maria-rcks <[email protected]> Co-authored-by: shivam <[email protected]> Co-authored-by: PR Batch Tester <[email protected]> Co-authored-by: Yash Singh <[email protected]> Co-authored-by: Tymoteusz Bielski <[email protected]>
) OpenCode usage scans materialized complete message bodies and queried empty rowid ranges. Select in-window rowids first and project usage fields in SQLite, keeping message bodies off the JS heap. Read each database in one snapshot and record scan row and page counts. On a read-only snapshot of a 3.32 GB real database, five fresh-process runs measured 2.33x faster 30-day reads and 2.82x faster 90-day reads with identical records. These are cold application-cache measurements; OS caches were not cleared. Local check:changed and CI passed. Adapted SQL projection from pingdotgg#27679 and window selection/snapshot handling from pingdotgg#26613. Model and harness: GPT 6.1 Sol through Codex in T3 Code. Co-authored-by: Julius Marminge <[email protected]> Co-authored-by: Tymoteusz Bielski <[email protected]> Co-authored-by: maria-rcks <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]> (cherry picked from commit bbdc7dc)
Problem
The OpenCode usage reader pages OpenCode's
messagetable 64 rows at a time, but it selects the wholedataJSON of every row. OpenCode can store user messages of 11–150 MB, almost all of itsummary.diffs[].patch. When several of those land in one 64-row page,StatementSync.all()pulls hundreds of MB to GBs of strings onto the V8 heap. Each scan then crashes the server withFATAL ERROR: Ineffective mark-compacts near heap limitinnode::sqlite::ExtractRowValues, and the client sees "reconnecting" in a loop.Reproduced on main at 616c426 with a synthetic
opencode.db: one assistant row plus 32 user rows of 150 MB each, all in one 64-rowid page, read withOPENCODE_DATA_DIR=<dir> vp run devand the Usage page opened. The server aborts with the reporter's stack. The same 150 MB rows spread one per page scan fine, so what kills the server is the total per page.Change
readOpenCodeUsagenow does the role check and field extraction in SQL, so a page only returns usage-sized data:WHEREterm keeps rows whose JSON has noroleor hasrole = 'assistant'. ACASE WHEN typeof(data) = 'text' AND json_valid(data)guard skips malformed and non-text rows the way the oldJSON.parsecatch did, without failing the page. The guard comes after the time and rowid predicates, so rows outside the window are rejected before their bodies are parsed.json_objectoftokens,cost,model,modelID,time,idandsessionIDin place ofdata.parseOpenCodeMessage, the rowid paging, thesession_messagepath and the legacy JSON-file path are unchanged.SQLite still reads each large body into native memory while it checks the row. That memory is per row and short-lived, and it is not on the V8 heap.
Scope and approval
Fixes #27667
Triaged via the automated issue flow.
Verification
readOpenCodeUsage > does not materialize user message diffs that share a page with usageinpackages/provider-opencode/src/server/usage.test.tsputs one assistant row and 32 user rows, each with a 1 MiB diff patch, in one page:expected 33558178 to be below 6291456: the bytesStatementSync.allreturns per page.vp run check:changedpasses: format, lint, knip, typecheck of 10 packages, and tests.role: null, malformed JSON, JSON5, BLOB and NULLdata, nestedmodelobjects and largecostvalues. The only differences are duplicaterolekeys and nesting deeper than 1000 levels.ExtractRowValues. This branch returns the usage, and its largest page result is 153 bytes.Runtime transcript (real dev server, synthetic data)
Driver: drivers/make-db.sh
builds an OpenCodeopencode.dbwith oneassistant row (1,200 input + 300 output tokens) plus N user rows whose data is
{"role":"user",...,"summary":{"diffs":[{"file":"gen.js","patch":"<x * size>"}]}}.Each run:
T3CODE_DEV_SEED=0 OPENCODE_DATA_DIR=<dir> vp run dev(fresh dev database), open /usage in the web client.Node v26 (heap_size_limit 4192 MB), Linux x86_64.
Run 1 log excerpt (main)
Run 3 (fix)
Driver: make-db.sh (synthetic OpenCode db, not committed)
Model and harness: Claude Opus 5.5 (claude-opus-5-5) in Claude Code via T3 Code; reviews by GPT-6.1 Sol (Codex) and Claude Opus 5.5.
🤖 Generated with Claude Code