fix(core): keep Code Mode catalog updates small and stable by rekram1-node · Pull Request #62176 · anomalyco/opencode

rekram1-node · 2026-11-05T20:04:57Z

Problem

Noisy updates. The catalog shared one 2k budget across all servers, filled cheapest-first. A server connecting or leaving reshuffled which tools the others showed, so most updates restated the whole catalog (~2.2k tokens each), and every restatement stays in history.
Wasted work. Picking the cheapest tools meant rendering every tool's signature on every rebuild.

Change

Same format and 2k cap as today, as small edits to the existing code.

Which tools show (catalog.ts): the namespaces still take turns, one tool per round, but in path order instead of cheapest first. Only the signatures it selects are rendered, plus the next one it tries; tool.ts makes signatures lazy for that. The stored catalog also keeps every tool's path, so an update can tell a new tool from one that just wasn't shown.
The prompt is unchanged: the complete version when every tool is listed, the partial one with search otherwise.
Updates (instructions.ts), capped at ~1k tokens while the catalog is partial, in the existing wording:
- a new namespace, or one whose description changed: "New tools are available:" with its first tools that fit, plus a count;
- new tools in a known namespace: "New tools are available in addition to those previously listed:" with their signatures, then "N more; search for them";
- a signature the model was shown that changed, even if it's no longer shown: "Changed tool listings supersede the previously listed ones:";
- removed tools and removed namespaces: the existing one-line notices;
- when the catalog stops listing every tool, a one-time note that introduces search, instead of restating the catalog.
No update when a namespace shows more or fewer tools because another came or went, or when a tool is pinned or unpinned.
The full catalog is restated only around an empty catalog, or when that is shorter than the update.

The Code Mode tool catalog has changed.

New tools are available in addition to those previously listed:
  - tools.github.actions_cancel({ owner: string, repo: string, run_id: number }): Promise<unknown> // Cancel a workflow run

Changed tool listings supersede the previously listed ones:
  - tools.github.actions_get({ … }): Promise<unknown> // Get details about GitHub Actions resources

The following tools are no longer available and must not be called: tools.github.actions_list.

Results

Updates over a replayed session (GitHub, Playwright, Context7, Filesystem; a late server, the browser attaching and detaching twice, GitHub gaining a tool):

	`dev`	This PR
Starting catalog	2,218 tokens	2,322
Late server connects	2,235 (full restatement)	24
Browser attaches (×2)	2,254 (full restatement)	802
Browser detaches (×2)	2,235 (full restatement)	27
GitHub gains a tool	42	236
All updates	11,255	1,918 (−83%)

Rebuild time, only when tools change: 2–12 ms typical, ~70 ms with Cloudflare's 3,767 tools.

Evals, dev vs this PR (* beyond noise):

	`dev`	This PR
Unseen tools (`codemode-novel`, 480 trials): pass rate	0.97	0.98
Unseen tools: steps	4.43	3.34 *
Unseen tools: input tokens	34.7k	23.3k *
Unseen tools: seconds	11.4	6.3 *
Known tools (`codemode-catalog`, 256 trials): pass rate	0.86	0.84
Known tools: seconds	13.8	10.1 *
Known tools: input tokens	53.3k	57.1k

Real servers (codemode-real: GitHub's MCP server read-only, and Playwright on real pages, 11 tasks), plus the fully fake tracker suite. 9 models, 2 repetitions:

	`dev`	This PR
Real servers: pass rate, 8 models without `gpt-5-nano`	0.98	0.97
Real servers: steps	4.1	3.9
Real servers: input tokens	42.3k	41.3k
Fake tracker: pass rate	0.88	0.89
Fake tracker: steps	5.2	4.6

The one regression is gpt-5-nano on real servers (7 of 22 passed vs 2 of 22). It tried a plain web request instead of the tools, then gave up, once saying the GitHub tools had no issue lookup because the catalog showed only the first GitHub tools alphabetically. Stronger models searched.

Evals: anomalyco/evals#21. The older codemode-catalog suite mixed real tool lists with fake responses and was replaced by codemode-real and codemode-tracker.

Servers connecting mid-session (codemode-arrival: one small server, a first question, then a small and a ~100-tool server connect and the catalog stops listing every tool; 10 models, 480 trials each):

	`dev`	This PR
Pass rate	0.99	1.00
Steps	3.8	3.1
Input tokens	36.2k	24.5k
Seconds	6.8	5.2

On tasks whose tool was not listed, models searched 97% of the time after the one-time note and passed every time.

Known limits

In a group too big to show in full, the shown tools are the alphabetically first ones, not the most useful. search covers the rest.
Sessions from before the upgrade restate the catalog once.
Weak models may conclude a tool doesn't exist instead of searching for it (see gpt-5-nano above).

Verification

packages/core: bun typecheck, bun run typecheck:tsc, and the full suite. On this machine 29 shell, plugin and image tests time out on dev too, plus the known isolates global home and XDG roots failure.
Root bun run check.
Existing catalog and instruction tests edited in place, with new cases for path order, rendering only what is selected, each update shape, the 1k cap, no update when only the shown tools move, and a changed signature on a tool that is no longer shown.
One end-to-end scenario: a new namespace, a removed one, a new tool, compaction and a fork.

Visual

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 240 72" width="240" height="72" fill="none" stroke="currentColor">
  <style>
    .new{animation:grow 3s ease-in-out infinite}
    @keyframes grow{0%,20%{opacity:0;transform:translateY(4px)}40%,80%{opacity:1;transform:none}100%{opacity:0}}
    @media (prefers-reduced-motion:reduce){.new{animation:none;opacity:1}}
  </style>
  <rect x="20" y="8" width="200" height="56" rx="6" stroke-width="1.5" opacity=".5"/>
  <g stroke-width="2" opacity=".7">
    <path d="M32 20 H70"/><path d="M40 28 H150"/>
    <path d="M32 38 H64"/><path d="M40 46 H130"/>
  </g>
  <g class="new" stroke-width="2.5">
    <path d="M32 56 H72"/><path d="M80 56 H160"/>
  </g>
</svg>

Added by the opencode bot (AI-generated).

Fixes #55529

opencodejr

The arrival and departure deltas work, but the catalog is not stable yet: it changes again on the turn after a namespace arrives.

How I checked: I ran this PR's catalog.test.ts against the base. The stickiness tests and the complete↔partial delta tests fail there, so the old reshuffle and full restatement reproduce; they pass here. In packages/core, bun test test/codemode test/session-runner.test.ts passes (277), and bun run typecheck:tsc passes. I also chained summarize calls with the same setup as the new tests (opencode 13 + slack 27 tools, budget 300, then a browser namespace with a description arrives):

Arrival turn: the update is just New tool namespaces are available: - browser …, and all 16 listings are kept. 👍
Next turn, same tools: the catalog drops to 15 listings and sends another update: The following tools are no longer listed: tools.opencode.tool0 ….

So each arrival into a full partial catalog still produces two updates, and the second one evicts tools the model was relying on. Details are inline.

Public API: unchanged. InstructionState.value, the previous option and the exported CodeModeInstructions.key are internal to packages/core, and the stored catalog format is the same.

Fixes #55529

opencodejr · 2026-11-05T20:08:05Z

      .reduce((total, listing) => total + cost(listing.line), 0)
+  // A namespace that arrived since the previous catalog is always shown, but its header does not cost the kept
+  // listings their place; only pinned tools added since can, and they drop the costliest first.
+  const known = new Set(options.previous?.namespaces.map((namespace) => namespace.name))


This skips the new namespace's header cost only on the turn it arrives. On the next turn browser is in known, so arrived is 0 and remaining is still negative. The loop below then evicts kept listings and sends a second update ("no longer listed: tools.opencode.tool0") even though no tools changed. I reproduced it by calling summarize again with the same inventory and previous: joined (opencode 13 / slack 27 / browser 45 with a description, budget 300): 16 listings drop to 15. A compaction or fork that rebuilds from the stored catalog would hit the same eviction.

One fix: allow overage up to what the previous catalog already cost, or carry the excused header cost forward, so kept listings are evicted only when newly pinned tools or grown signatures need the room. A regression test could run the same inventory twice after an arrival and expect update to have nothing to send the second time.

Fixed in the latest commit, thanks. Kept listings are no longer evicted at all while their tools exist, so the over-budget case can't drop them on a later step. A fixed-point test now builds the catalog again with the same tools after an arrival, a departure and new pins, and expects no change. It fails on the previous commit with exactly this eviction. The catalog can run over budget, bounded by the headers of namespaces that arrived since and any newly pinned tools; pinned tools already exceed the budget in a fresh catalog.

opencodejr

The eviction problem from my last review is fixed. Kept listings are no longer dropped, so building the catalog again from the stored one with the same tools reproduces it exactly. I checked this with my earlier repro (opencode 13 and slack 27 tools at a 300-token budget, with a 45-tool browser namespace attached and detached four times): a second build with the same tools changed nothing after each attach and each detach, and each detach notice stayed small. test/codemode and session-runner.test.ts pass, and so does typecheck:tsc.

Public API is unchanged. Nothing else to raise.

rekram1-node · 2026-11-05T20:28:11Z

A review pass found no case where the stored catalog and what the model saw can diverge (compaction, nested forks, reverts, moves, agent switches, Code Mode off and on, undecodable values, preview). It also confirmed that rebuilding with unchanged tools is a fixed point. Fixed from that pass:

Removed tools now always say "no longer available". The "may have been removed; search" wording dated from the first commit's eviction.
Namespace lines whose tool count or shown count changes are restated, so counts don't go stale.
The complete → partial notice says search only works inside execute.
Tests: the undecodable-value test now compares the reads, the delta tests build on the previous catalog, the revert scenario compares the baseline to a fresh render, and there is a new complete → partial → compaction scenario.

The arrival-order trade-off is now described in the PR body.

opencodejr

My earlier eviction thread stays fixed: rebuilding the catalog with the same tools still reproduces it exactly. I checked three attach/detach cycles of a 45-tool browser namespace against opencode and slack at a 300-token budget. Each attach sent only the new-namespace block, each detach sent only a one-line removal, and building again with the same tools never produced an update. The new header restatement and the "no longer available" wording are consistent with tools now leaving only when they no longer exist. test/codemode and session-runner.test.ts pass (282), and so does typecheck:tsc.

Public API is unchanged. Nothing new to raise.

rekram1-node · 2026-11-05T21:25:36Z

The approach has changed. The earlier sticky selection (and the review findings about it) is gone. Each namespace now gets its own catalog section, built only from that namespace's tools: names with required parameters, full signatures only for pinned tools, and a size cap per namespace. Updates name only the namespaces and tools that changed, and there's no new session state. The PR description has the details, a session-replay token comparison (−92% on catalog updates) and an A/B eval against dev.

opencodejr

The per-namespace sections work as described. I ran the changed tests (test/codemode, codemode.test.ts, session-runner, session-generate, test/instructions; 291 pass) and typecheck:tsc (passes). In a scratch check with opencode and slack, attaching a 45-tool browser namespace sends only its section (~420 tokens), detaching sends one line (~32), and summarizing the same tools again gives an identical catalog. My earlier eviction thread no longer applies.

One concern: the catalog now has no overall size limit (inline). Older sessions whose stored catalog uses the previous format fall back to one fresh render through the instructions decode path, so nothing breaks.

Public API is unchanged; the stored catalog value is internal.

opencodejr · 2026-11-05T21:31:49Z

 const CHARACTERS_PER_TOKEN = 4
-const INLINE_BUDGET = 2_000
+// Enough for every tool of a typical MCP server; a larger one lists what fits and the rest is found with search.
+const SECTION_BUDGET = 1_500


With a per-namespace cap and no overall one, the catalog grows with every connected server. Eight servers with 60 tools each render to about 4,600 tokens, against the base's ~2,000-token total, and users with many large MCP servers could pay 10k+ tokens on every request. Could the catalog keep an overall ceiling, for example by shrinking every section's cap evenly once the total is exceeded? That would keep each section independent of its neighbours' contents, so updates stay small, while still bounding the total.

opencodejr

The new commits look sound. Marked partial-update lines, not reporting a tool that only moved between pinned and listed by name as gone, bare names for inputs that aren't plain objects, and listing a tool at its namespace's own path all match their tests. The changed tests pass (test/codemode including the new parameters.test.ts, codemode.test.ts, session-runner, session-generate, test/instructions; 298 pass), and so does typecheck:tsc.

The point about the catalog having no overall size limit isn't changed in code; it's now documented as a known limit, so that thread stays open. Nothing new to raise.

Public API is unchanged.

opencodejr

The detail levels for each namespace work as described. Updates still restate only the namespace that changed: in a scratch check, adding one tool sent ~95 tokens, a 45-tool browser namespace arriving sent only its own section, and leaving sent one line. Signature rendering runs only on namespaces of 75 tools or fewer, and the catalog inventory is cached across steps, so the extra rendering is paid once per tool set. Tests pass in packages/core (test/codemode, codemode.test.ts, session-runner, session-generate, test/instructions; 300) and packages/codemode (1,501), and typecheck:tsc passes in both.

The open size thread still applies: by the description's own figures, the starting catalog roughly doubled. Nothing new to raise.

Public API: unchanged for users. ToolDescription.compactSignature is an additive field inside @opencode/codemode.

opencodejr

The global 2k cap is back and the catalog depends only on the current tools, which resolves both earlier threads. In a scratch check, a group arriving sends only its section, leaving sends one line, and other groups showing fewer tools sends nothing. test/codemode, codemode.test.ts, session-runner, session-generate and test/instructions pass (295), and so does typecheck:tsc.

One new gap: a signature the model was shown can change without the model being told when the tool also drops out of the shown set (inline).

Public API is unchanged.

packages/core/src/codemode/instructions.ts:97: This only reports a changed signature when the tool is still shown in current. If a tool the model saw changes signature in the same step it drops out of the shown set (because another group arrived), nothing is said. Later, when it's shown again, previous.listings no longer has it, so the change is never reported. I reproduced it with opencode 60 / slack 80 tools at the default budget: slack.tool58 gains force: boolean as a 3-tool browser group arrives. The update only announces browser, and when browser leaves the update only says it's gone. The model keeps the old signature until the next compaction. Comparing each previous.listings entry against the current inventory through lookup(path), not against current.listings, would catch the case where a change and a hide happen in the same step.

… no longer shown

opencodejr

The stale-signature gap is fixed. Comparing every listing the model was shown against the current inventory catches a changed signature even when the tool is no longer shown. My repro (a shown slack tool changing signature in the same step a new group pushes it out of view) now reports it under Changed:, and rebuilding with the same tools sends nothing. The changed tests pass (296), and so does typecheck:tsc. The earlier eviction and size threads are addressed by the current design.