Repository navigation
Commit 95bc33f
feat: add the AIR sessionIndex extension for fast session lists (#590)
## Summary
Adds the opt-in AIR `sessionIndex` extension, so that an AIR client can
show the latest sessions of a project quickly, with their state, and can
rename, archive and unarchive them over ACP. The branch also adds
recovery from a crashed Codex app-server, cheaper tool call update
dedup, and fixes to token usage and `session/fork`.
The extension follows these ACP RFDs. The transport differs
(`_meta.jetbrains.air` request options and row fields, `_session/*`
methods, AIR capability names), and so does the `archived` list filter:
- **Session list extensions**
(agentclientprotocol/agent-client-protocol#2323): `limit`,
`updatedAt`-desc order, `includeWorktrees`, and the row fields
`lastPromptAt`, `model`, `forkedFrom`, `state` and `lastTurnEndedAt`.
The RFD's `list_changed` hint is replaced: the adapter pushes changed
rows through a subscription (below).
- **Session archive** (agentclientprotocol/agent-client-protocol#2161):
- `archived` on every row and `session_info_update` on a change;
- `-32002` for unknown sessions;
- the list filter `archived` is a string with three values:
`"unarchived"` (the default for omitted or `null`), `"archived"`
(archived only) and `"all"`. #2161 has a boolean, whose `false` and
`true` are `"unarchived"` and `"all"`.
- **Archiving stops a session that is open here**, as `session/close`
does. #2161 leaves execution alone; the session list extensions RFD
proposes this change to #2161.
- **Client-set titles**
(agentclientprotocol/agent-client-protocol#1987): `_session/rename`.
Everything is gated on the client declaring `sessionIndex` in
`clientCapabilities._meta.jetbrains.air.capabilities`. The agent then
lists these in `_meta.jetbrains.air.capabilities` of `initialize`:
- `sessionIndex`;
- `sessionArchive`: `_session/archive` and `_session/unarchive` (#2161's
`archive` capability);
- `sessionRename`: `_session/rename` (#1987's rename capability,
`setTitle` there);
- `sessionListSubscribe`: `_session/list/subscribe`.
The client does not declare the last three.
Clients that don't declare `sessionIndex`, meaning non-AIR clients and
released AIR builds, keep main's session index behaviour:
- the same `session/list` path, auth check and `thread/list` params
(covered by a snapshot test);
- `session/delete` still archives;
- the same initialize capabilities;
- no watchers and no notifications.
## Session index
- **`session/list`**:
- One `thread/list` per archive side with `cwd`, `limit` ≤ 100,
`sortKey: "recency_at"`, `sourceKinds: []`, `modelProviders: []` and
`useStateDbOnly: true`.
- With `includeWorktrees`, it also lists the same relative subdirectory
in each linked Git worktree, read from
`<git-common-dir>/worktrees/*/gitdir` as the Codex TUI does.
- `"unarchived"` and `"archived"` each read one Codex list. `"all"`
reads both in parallel and merges them by recency, with ties unarchived
first.
- The adapter cursor holds the Codex cursor of each side and the last
rows returned. It is bound to `cwd`, `archived` and `includeWorktrees`;
a cursor from another scope fails with `-32602`.
- A relative `cwd` gathers several Codex pages up to `limit`, and stops
a scan without matches after 300 ms, answering with a cursor.
- It doesn't wait for auth.
- **Row fields** (`_meta.jetbrains.air`):
- `archived`, always present;
- `lastPromptAt` (`Thread.recencyAt`, the order key);
- `model`;
- `forkedFrom`;
- `state`;
- `lastTurnEndedAt`.
`updatedAt` is `Thread.updatedAt`. There is no `cost`, and rows carry no
token usage.
- **`forkedFrom`** is `Thread.forkedFromId` when Codex gives it.
`thread/list` doesn't, so otherwise the adapter reads
`session_meta.payload.forked_from_id` from the first line of the
rollout:
- the read runs in the background, is bounded, and happens once per
thread (negative answers are cached too);
- a list never waits for it;
- a subscription gets the row again when the read ends.
- **Row `state`** is set only for a thread loaded here. The first that
holds wins:
- `requires_action`;
- `reviewing`: Codex review mode, during a turn;
- `running`;
- `error`: the last turn failed, not a user cancel, or `Thread.status`
is `systemError`. A new turn, a close or unload, or an app-server
restart clears it.
- `idle`.
Threads that aren't loaded omit `state`.
- **`_session/rename`** → `thread/name/set`. It is serialized with
automatic titles per session, so an explicit title is never overwritten,
also across close and reload. A failed rename keeps automatic titles on.
Renaming an archived thread fails with `data.reason: "archived"`.
- **`_session/archive`** → `thread/archive`, after closing a session
that is open on this connection:
- The close works as `session/close` does: it interrupts the running
turn, ends the pending prompt and its permission and elicitation
requests, and unsubscribes the thread.
- `session_info_update` with `archived: true` is then the last update of
that session.
- A later prompt fails as for a closed session.
- If `thread/archive` fails, the session stays closed and unarchived.
- **`_session/unarchive`** → `thread/unarchive`. It loads nothing.
Archive and unarchive are idempotent: the adapter checks `Thread.path`
with `thread/read`.
- **`session/delete`** → `thread/delete`, closing a loaded session
first.
- **Errors**:
- a thread held by another process fails with `data.reason:
"thread_active_writer"`;
- an unknown thread fails with `-32002`;
- invalid params or an invalid cursor fail with `-32602`.
Documented in `docs/air-extensions.md`, with an extension → RFD mapping
table and the remaining differences.
## Session list subscription
The methods:
- `_session/list/subscribe {cwd}` → `{subscriptionId}`. `cwd` is
required and absolute, else `-32602`. A connection can hold at most 128
subscriptions; the next one fails with `-32602` and `data.reason:
"too_many_subscriptions"`.
- `_session/list/unsubscribe {subscriptionId}` → `{}`. It is idempotent.
- Notification `_session/list/changes {subscriptionId, sessions,
removed}`.
How it behaves:
- **Scope**: threads of the cwd, of its canonical path, and of the same
subdirectory in the primary checkout and each linked worktree, from
interactive sources, archived or not.
- **Rows**: each change carries the full row exactly as `session/list`
answers it. Subscribing reads no rows, only a small `thread/list` per
archive state for the first marks.
- The first change of each session after subscribing is sent in full.
Later ones are sent only when `title`, `lastPromptAt`, `state`,
`lastTurnEndedAt`, `model`, `forkedFrom` or `archived` differ.
- `removed` lists deleted threads.
- At most one change per thread per second, and nothing before the
subscribe answer.
- Best effort: no resync and no sequence numbers.
- **Detection**:
- The adapter's own app-server notifications are handled at once.
- Other processes are seen through `fs.watch` of `state_<n>.sqlite-wal`
and CODEX_HOME, with a 30 s fallback check. Each burst of writes leads
to an `updated_at` scan per archive state.
- Renames come from `session_index.jsonl` and archive moves from
`archived_sessions/`.
- A delete by another process of an unarchived thread is not seen.
- **Recovery**: subscriptions survive an app-server crash.
New modules: `SessionIndex*.ts` (incl. `SessionIndexWorktrees.ts`),
`SessionListSubscriptions.ts`, `CodexHomeWatcher.ts`,
`SessionNameLog.ts`, `SessionForkOrigins.ts`, `SessionWriteQueue.ts`.
### Measurements
Docker (`node:22`, 2 CPUs, 2 GB), 10k threads, first page of 50:
| Query (cold / warm p50 / p95, ms) | Before | After |
|---|---|---|
| Big project | 47 / 25 / 68 | 42 / 21 / 23 |
| `archived: "all"` | 213 / 104 / 260 | 41 / 22 / 26 |
| `archived: "all"` + worktrees | 167 / 70 / 203 | 125 / 26 / 30 |
Under churn from another Codex app-server (a rename, archive or
unarchive every 200 ms for 300 s):
- a change reached the subscription with p50 158 ms and p95 162 ms,
against 1.03 s and 2.0 s for the old `_session/list_changed` hint;
- 0 changes were missed and 0 duplicated;
- adapter plus app-server CPU was about 0.02 cores.
## Recover from a crashed Codex app-server
Code is in `src/app-server-recovery/`. In Docker load tests, a cold
`session/resume` of a 638 MB rollout got the app-server OOM-killed. Main
then failed every later request with "Connection is disposed.", and a
running prompt never ended.
- **`CodexAppServerSupervisor`** owns the child:
- It records the code, signal, uptime and stderr of every exit, and
tells crashes from intentional stops.
- It drains the output before disposing the connection.
- It never SIGKILLs the `codex.js` wrapper.
- A replacement starts with the same startup args as the first
app-server, including main's hook trust override.
- **`AppServerRecovery`** starts the app-server again lazily, on the
next request that needs it:
- It reuses the client's `initialize` handshake and the provider
routing.
- Backoff is 0/1/2/4 s, and it stops after 5 crashes in 5 min
(`CODEX_ACP_APP_SERVER_CRASH_LIMIT` and
`CODEX_ACP_APP_SERVER_CRASH_WINDOW_MS`).
- An open session is resumed on its next use, with the model, effort,
modes, directories and MCP servers that the adapter kept.
- A session that the app-server died opening twice in 30 min is refused
for the rest of that time.
- **Request in flight**: fails with code 1001 and the cause, for example
"killed by SIGKILL, which usually means it ran out of memory", plus the
stderr tail.
- An app-server that exits before its `initialize` handshake keeps
main's wording and promises no restart: `Codex process has exited with
code N:` plus stderr. An example is one that rejects an invalid hooks
config.
- The crash-loop error also carries the stderr of the last crash.
- **Prompt in flight**: ends at once.
- An AIR client gets `end_turn` with the `transport_lost` session
failure; other clients get JSON-RPC 1001.
- Open tool calls become `failed`, and permission and elicitation
requests are cancelled.
- A prompt the client already cancelled answers `cancelled`.
- **What restarts a crashed app-server**: `session/list`, the
`_session/*` requests and `_codex/hooks/list` and `_codex/hooks/trust`
do. Close, cancel and shutdown never do.
- **No crash**: behaviour is unchanged, and the fast paths add no
`await`.
## Other changes
- **Tool call update dedup**: `ToolCallReports` compares fields as a
shared tree (`src/JsonSnapshot.ts`) instead of running `JSON.stringify`
on every update. The same updates are sent and suppressed, checked by a
differential test over about 300k random updates. Example: comparing 6
MB of diffs went from 7.4 ms to 0.2 ms.
- **App-server logging**: `attachLogs` formats traffic only when
`APP_SERVER_LOGS` is set.
- **Token usage**:
- Cache writes are subtracted from input and reported as
`cachedWriteTokens`.
- `PromptResponse.usage` and `_meta.quota` report the whole turn, not
only the last model request.
- A native subagent's usage no longer overwrites the parent session's
usage.
- **`session/fork`**: the fork stays subscribed. Before, a prompt sent
to the fork before a `session/load` never ended.
## Test plan
- [x] `npm run typecheck`, `npm test` (1483 passed), `npm run
bundle:all`
- [x] `RUN_E2E_TESTS=true` `e2e/hooks.test.ts`: the invalid-hooks
`initialize` error matches main's
- [x] Snapshot test that the legacy `thread/list` params are unchanged
without `sessionIndex`
- [x] Watcher tests on macOS and Linux (Docker `node:22`)
- [x] Merge pagination fuzzers (empty pages, mutations, budget)
- [x] Crash recovery unit tests with fake app-servers that die at
controlled points, plus Docker SIGKILL reproduction
- [x] End-to-end against a real app-server in a throwaway `CODEX_HOME`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>1 parent 51f78d6 commit 95bc33f
65 files changed
Lines changed: 11863 additions & 217 deletions
File tree
- docs
- src
- __tests__
- CodexACPAgent
- data
- app-server-recovery
- app-server-recovery
- subagents
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| 15 | + | |
| 16 | + | |
15 | 17 | | |
16 | 18 | | |
17 | 19 | | |
| |||
102 | 104 | | |
103 | 105 | | |
104 | 106 | | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
166 | 166 | | |
167 | 167 | | |
168 | 168 | | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
169 | 185 | | |
170 | 186 | | |
171 | 187 | | |
| |||
183 | 199 | | |
184 | 200 | | |
185 | 201 | | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
186 | 210 | | |
187 | 211 | | |
188 | 212 | | |
| |||
213 | 237 | | |
214 | 238 | | |
215 | 239 | | |
| 240 | + | |
216 | 241 | | |
217 | 242 | | |
218 | 243 | | |
| |||
226 | 251 | | |
227 | 252 | | |
228 | 253 | | |
| 254 | + | |
229 | 255 | | |
230 | 256 | | |
231 | 257 | | |
| |||
267 | 293 | | |
268 | 294 | | |
269 | 295 | | |
| 296 | + | |
270 | 297 | | |
271 | 298 | | |
272 | 299 | | |
| |||
276 | 303 | | |
277 | 304 | | |
278 | 305 | | |
279 | | - | |
280 | | - | |
281 | | - | |
282 | | - | |
283 | | - | |
284 | | - | |
285 | | - | |
286 | | - | |
287 | | - | |
288 | | - | |
289 | | - | |
290 | | - | |
291 | | - | |
292 | | - | |
293 | | - | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
294 | 322 | | |
295 | | - | |
296 | | - | |
297 | | - | |
298 | | - | |
299 | | - | |
300 | | - | |
301 | | - | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
302 | 330 | | |
303 | | - | |
304 | | - | |
305 | | - | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
306 | 341 | | |
307 | 342 | | |
308 | 343 | | |
| |||
383 | 418 | | |
384 | 419 | | |
385 | 420 | | |
| 421 | + | |
386 | 422 | | |
387 | 423 | | |
388 | 424 | | |
| |||
663 | 699 | | |
664 | 700 | | |
665 | 701 | | |
666 | | - | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
667 | 706 | | |
668 | 707 | | |
669 | 708 | | |
| |||
675 | 714 | | |
676 | 715 | | |
677 | 716 | | |
| 717 | + | |
678 | 718 | | |
679 | 719 | | |
680 | 720 | | |
| |||
1196 | 1236 | | |
1197 | 1237 | | |
1198 | 1238 | | |
1199 | | - | |
1200 | | - | |
| 1239 | + | |
| 1240 | + | |
1201 | 1241 | | |
1202 | 1242 | | |
1203 | 1243 | | |
1204 | 1244 | | |
1205 | 1245 | | |
| 1246 | + | |
1206 | 1247 | | |
1207 | 1248 | | |
1208 | 1249 | | |
| |||
0 commit comments