Skip to content

Commit 6ec2e52

Browse files
feat: report the token usage of each thread in the session list rows
A sessionIndex row now carries _meta.jetbrains.air.usage = {inputTokens, cachedReadTokens, cachedWriteTokens, outputTokens, reasoningTokens, model, subagents: [{sessionId, model, ...counts}]}: the cumulative total of the last token_count record of the thread's rollout (Thread.path), with fresh input apart from cache reads and writes as in PromptResponse.usage. A fork counts only its own work: its first record holds the inherited total plus its first request, so that total minus its last_token_usage is subtracted. Subagents are the threads named by the SubAgentActivity items of the thread's own rollout, each read with thread/read and counted apart; thread/list leaves out the about half of them that have no user message of their own. Reading rollouts with the subagent lookup made a cold first page of 50 rows three to four times slower, so usage is read in the background: a row carries the usage read for its thread, rows that are not read yet get it through _session/list/changes once it is, and unchanged threads and rollouts are not read again. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 3ec572b commit 6ec2e52

9 files changed

Lines changed: 902 additions & 26 deletions

‎docs/air-extensions.md‎

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1047,9 +1047,40 @@ Every row carries these fields in `_meta.jetbrains.air`; all but `archived` only
10471047
| `forkedFrom` | `Thread.forkedFromId`. |
10481048
| `state` | Only for a thread that this adapter has loaded: `running`, `requires_action` (waiting for an approval or for user input), or `idle`. Omitted otherwise, never `unknown`. |
10491049
| `lastTurnEndedAt` | ISO time of the last `turn/completed` that this adapter saw for a session of this connection. Forgotten when the thread is deleted. |
1050+
| `usage` | The token usage of the thread, see [Token usage](#token-usage). Omitted until the adapter has read it, and for a thread whose rollout has no token count. |
10501051

10511052
The adapter sends no `cost`: Codex reports none.
10521053

1054+
#### Token usage
1055+
1056+
```json
1057+
{ "inputTokens": 704734, "cachedReadTokens": 31875072, "cachedWriteTokens": 0, "outputTokens": 61728, "reasoningTokens": 27252,
1058+
"model": "gpt-6.1-sol",
1059+
"subagents": [ { "sessionId": "01a0f43d-…", "model": "gpt-6.1-sol", "inputTokens": 257928, "cachedReadTokens": 13756160, "cachedWriteTokens": 0, "outputTokens": 20889, "reasoningTokens": 6150 } ] }
1060+
```
1061+
1062+
- The counts are cumulative for the thread. They come from `info.total_token_usage` of the last `event_msg`
1063+
`token_count` record in the thread's rollout, the file at `Thread.path`. The adapter reads the last 16 KB of it,
1064+
and up to 1 MB when that holds no record.
1065+
- `inputTokens` is fresh input only: Codex's input minus the cache reads and the cache writes, at least 0, as
1066+
`PromptResponse.usage` counts it. `cachedReadTokens` and `cachedWriteTokens` are the cache reads and writes.
1067+
`outputTokens` includes `reasoningTokens`.
1068+
- `model` is `Thread.model`, the thread's latest model.
1069+
- A fork counts only its own work. The first record of a fork carries the total of the history it inherited
1070+
plus the usage of its own first request, so the adapter subtracts that first total minus its
1071+
`last_token_usage`. A fork whose first record is not in the first 8 MB of its rollout gets no `usage`.
1072+
- `subagents` lists the threads that the thread spawned, in the order it spawned them, each with its own counts
1073+
and `model`; they are not part of the thread's counts. The adapter finds them in the thread's rollout, from the
1074+
`agent_thread_id` of its `SubAgentActivity` items, and reads each one with `thread/read` for its rollout and
1075+
model. A subagent without a token count is left out; a thread that spawned none has `subagents: []`.
1076+
- The adapter reads usage in the background, so a list never waits for it. A row carries the usage read for its
1077+
thread; the first list of a thread has none, and a thread that changed keeps the usage of the last read until
1078+
the new one is done. A subscription gets the row again when its usage was read, see
1079+
[Session list subscription](#session-list-subscription). A thread whose `updatedAt` and rollout did not change is
1080+
not read again, and a rollout is read again only when its size or time changed.
1081+
- The usage of a subagent is read again when its parent thread changes: a subagent that works while its parent
1082+
waits shows its new counts once the parent writes again.
1083+
10531084
### Requests
10541085

10551086
| Method | Params | Codex call |
@@ -1107,7 +1138,7 @@ still re-reads the list now and then, and a lost change shows up there.
11071138
archived or not. The agent applies no archive filter: the client filters by the `archived` field of the row.
11081139
- `_session/list/changes` carries, for each thread in scope that appeared or changed, its whole row exactly as
11091140
`session/list` answers it. A row counts as changed when `title`, `lastPromptAt`, `state`, `lastTurnEndedAt`,
1110-
`model`, `forkedFrom` or `archived` differs from the row the subscription last got. A change of `updatedAt` alone
1141+
`model`, `forkedFrom`, `archived` or `usage` differs from the row the subscription last got. A change of `updatedAt` alone
11111142
sends nothing; the new `updatedAt` comes with the next change. Archive and unarchive are row changes.
11121143
`removed` lists the ids of deleted threads. Both arrays are always present, and one of them is not empty.
11131144
- The rows the client is assumed to have when it subscribes are the 50 most recently updated threads of the scope,
@@ -1126,7 +1157,9 @@ How the agent sees changes:
11261157

11271158
- Threads of its own Codex app-server: at once, from the app-server notifications `thread/started`,
11281159
`thread/status/changed`, `thread/name/updated`, `thread/archived`, `thread/unarchived`, `thread/deleted`,
1129-
`thread/closed`, `turn/started` and `turn/completed`. The agent reads the thread with `thread/read`.
1160+
`thread/closed`, `turn/started`, `turn/completed` and `thread/tokenUsage/updated`. The agent reads the thread
1161+
with `thread/read`.
1162+
- Token usage that the agent read in the background: the row is sent again when its usage differs.
11301163
- Threads of other Codex processes: every change of a thread writes the state DB WAL,
11311164
`<CODEX_HOME>/state_<n>.sqlite-wal`. The agent watches the WAL files and CODEX_HOME for WAL files that appear or
11321165
go, and checks the WAL size and time every 30 s in case the file system drops an event. 150 ms after the last

‎src/RolloutUsage.ts‎

Lines changed: 236 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,236 @@
1+
/**
2+
* Reads the token usage and the spawned subagents of a thread from its rollout file.
3+
*
4+
* Codex writes `event_msg` `token_count` records into the rollout of a thread; `info.total_token_usage` of the
5+
* last one is the running total of the thread, `info.last_token_usage` the usage of the request it follows.
6+
* A thread that spawns a subagent records a `SubAgentActivity` item with the `agent_thread_id` of the child.
7+
*/
8+
9+
import fs from "node:fs/promises";
10+
11+
export interface SessionUsageTokens {
12+
/** Fresh input: neither read from nor written to the prompt cache. */
13+
inputTokens: number;
14+
cachedReadTokens: number;
15+
cachedWriteTokens: number;
16+
/** Output, reasoning included. */
17+
outputTokens: number;
18+
reasoningTokens: number;
19+
}
20+
21+
/** The fields of a Codex `token_count` total, as the rollout writes them. */
22+
export interface RawTokens {
23+
input: number;
24+
cached: number;
25+
cacheWrite: number;
26+
output: number;
27+
reasoning: number;
28+
}
29+
30+
/**
31+
* The tail windows read in turn for the last `token_count`. In real rollouts the last record is within 16 KB of
32+
* the end for 98% of threads and within 1 MB for all.
33+
*/
34+
const TAIL_WINDOWS = [16 * 1024, 128 * 1024, 1024 * 1024];
35+
/** How much of the start of a fork's rollout is read for its first `token_count`, and in what chunks. */
36+
const HEAD_LIMIT = 8 * 1024 * 1024;
37+
const HEAD_CHUNK = 256 * 1024;
38+
39+
const TOKEN_COUNT_MARKER = "\"token_count\"";
40+
const TOKEN_COUNT_BYTES = Buffer.from(TOKEN_COUNT_MARKER);
41+
const NEWLINE = 0x0a;
42+
/** How far into a `token_count` line its marker can be. */
43+
const MAX_TOKEN_COUNT_PREFIX = 4096;
44+
45+
/** The `token_count` total and last usage of a rollout line, or `null` for any other line. */
46+
function tokenCountOf(line: string): {total: RawTokens, last: RawTokens | null} | null {
47+
if (!line.includes(TOKEN_COUNT_MARKER)) return null;
48+
let record: unknown;
49+
try {
50+
record = JSON.parse(line);
51+
} catch {
52+
return null;
53+
}
54+
const payload = field(record, "payload");
55+
if (field(record, "type") !== "event_msg" || field(payload, "type") !== "token_count") return null;
56+
const info = field(payload, "info");
57+
const total = rawTokens(field(info, "total_token_usage"));
58+
return total === null ? null : {total, last: rawTokens(field(info, "last_token_usage"))};
59+
}
60+
61+
function rawTokens(value: unknown): RawTokens | null {
62+
const count = (key: string): number | null => {
63+
const number = field(value, key);
64+
return typeof number === "number" && Number.isFinite(number) ? number : null;
65+
};
66+
const input = count("input_tokens");
67+
const output = count("output_tokens");
68+
if (input === null || output === null) return null;
69+
return {
70+
input,
71+
cached: count("cached_input_tokens") ?? 0,
72+
cacheWrite: count("cache_write_input_tokens") ?? 0,
73+
output,
74+
reasoning: count("reasoning_output_tokens") ?? 0,
75+
};
76+
}
77+
78+
function field(value: unknown, key: string): unknown {
79+
return value !== null && typeof value === "object" ? (value as Record<string, unknown>)[key] : undefined;
80+
}
81+
82+
export function minus(left: RawTokens, right: RawTokens): RawTokens {
83+
return {
84+
input: Math.max(0, left.input - right.input),
85+
cached: Math.max(0, left.cached - right.cached),
86+
cacheWrite: Math.max(0, left.cacheWrite - right.cacheWrite),
87+
output: Math.max(0, left.output - right.output),
88+
reasoning: Math.max(0, left.reasoning - right.reasoning),
89+
};
90+
}
91+
92+
/** The usage fields of a total. Fresh input leaves out cache reads and writes, as `toTokenCount` does. */
93+
export function usageTokens(tokens: RawTokens): SessionUsageTokens {
94+
return {
95+
inputTokens: Math.max(0, tokens.input - tokens.cached - tokens.cacheWrite),
96+
cachedReadTokens: tokens.cached,
97+
cachedWriteTokens: tokens.cacheWrite,
98+
outputTokens: tokens.output,
99+
reasoningTokens: tokens.reasoning,
100+
};
101+
}
102+
103+
/** The total of the last `token_count` in the rollout, read from its end in growing windows. */
104+
export async function readLastTokenTotal(file: string): Promise<RawTokens | null> {
105+
const handle = await fs.open(file, "r");
106+
try {
107+
const size = (await handle.stat()).size;
108+
for (const window of TAIL_WINDOWS) {
109+
const start = Math.max(0, size - window);
110+
const buffer = Buffer.alloc(size - start);
111+
await handle.read(buffer, 0, buffer.length, start);
112+
let end = buffer.length;
113+
while (end > 0) {
114+
const marker = buffer.lastIndexOf(TOKEN_COUNT_BYTES, end - 1);
115+
if (marker < 0) break;
116+
const lineStart = buffer.lastIndexOf(NEWLINE, marker) + 1;
117+
// A line cut by the window start is read whole with the next window.
118+
if (lineStart === 0 && start > 0) break;
119+
const lineEnd = buffer.indexOf(NEWLINE, marker);
120+
const count = tokenCountOf(buffer.toString("utf8", lineStart, lineEnd < 0 ? buffer.length : lineEnd));
121+
if (count !== null) return count.total;
122+
end = lineStart;
123+
}
124+
if (start === 0) break;
125+
}
126+
return null;
127+
} finally {
128+
await handle.close();
129+
}
130+
}
131+
132+
/** The first `token_count` of the rollout, read from its start, or `null` within {@link HEAD_LIMIT}. */
133+
export async function readFirstTokenCount(file: string): Promise<{total: RawTokens, last: RawTokens | null} | null> {
134+
const handle = await fs.open(file, "r");
135+
try {
136+
let pending = Buffer.alloc(0);
137+
// Inside a long line that is no `token_count`: its rest is skipped.
138+
let skipping = false;
139+
let position = 0;
140+
while (position < HEAD_LIMIT) {
141+
const chunk = Buffer.alloc(HEAD_CHUNK);
142+
const {bytesRead} = await handle.read(chunk, 0, HEAD_CHUNK, position);
143+
if (bytesRead === 0) break;
144+
position += bytesRead;
145+
let fresh = chunk.subarray(0, bytesRead);
146+
if (skipping) {
147+
const lineEnd = fresh.indexOf(NEWLINE);
148+
if (lineEnd < 0) continue;
149+
fresh = fresh.subarray(lineEnd + 1);
150+
skipping = false;
151+
}
152+
const buffer = pending.length === 0 ? fresh : Buffer.concat([pending, fresh]);
153+
let from = 0;
154+
for (;;) {
155+
const marker = buffer.indexOf(TOKEN_COUNT_BYTES, from);
156+
if (marker < 0) break;
157+
const lineEnd = buffer.indexOf(NEWLINE, marker);
158+
if (lineEnd < 0) break;
159+
const lineStart = buffer.lastIndexOf(NEWLINE, marker) + 1;
160+
const count = tokenCountOf(buffer.toString("utf8", lineStart, lineEnd));
161+
if (count !== null) return count;
162+
from = lineEnd + 1;
163+
}
164+
// The last line goes on in the next chunk. The marker is near the start of a `token_count` line,
165+
// which is a few KB long, so a longer line without it is none.
166+
pending = buffer.subarray(buffer.lastIndexOf(NEWLINE) + 1);
167+
if (pending.length > MAX_TOKEN_COUNT_PREFIX && pending.indexOf(TOKEN_COUNT_BYTES) < 0) {
168+
pending = Buffer.alloc(0);
169+
skipping = true;
170+
}
171+
}
172+
return tokenCountOf(pending.toString("utf8"));
173+
} finally {
174+
await handle.close();
175+
}
176+
}
177+
178+
const SPAWN_MARKER = Buffer.from("\"agent_thread_id\":\"");
179+
const THREAD_ID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
180+
const SCAN_CHUNK = 1024 * 1024;
181+
182+
/** Where a scan of a rollout for spawned threads stopped, and what it found. */
183+
export interface SpawnScan {
184+
/** The end of the last complete line read. */
185+
offset: number;
186+
/** The spawned thread ids, in the order of their first record. */
187+
threadIds: string[];
188+
}
189+
190+
/**
191+
* The threads that a thread spawned, from the `agent_thread_id` of the `SubAgentActivity` items that it records
192+
* under its own `thread_id`: a fork can carry such items of the thread it was forked from. A rollout only grows,
193+
* so a scan goes on from where the last one stopped; one of a file that is shorter now starts over.
194+
*/
195+
export async function scanSpawnedThreads(file: string, threadId: string, previous: SpawnScan | null): Promise<SpawnScan> {
196+
const ownRecord = `"thread_id":"${threadId}"`;
197+
const handle = await fs.open(file, "r");
198+
try {
199+
const size = (await handle.stat()).size;
200+
const scan: SpawnScan = previous !== null && previous.offset <= size
201+
? {offset: previous.offset, threadIds: [...previous.threadIds]}
202+
: {offset: 0, threadIds: []};
203+
const seen = new Set(scan.threadIds);
204+
let position = scan.offset;
205+
while (position < size) {
206+
const length = Math.min(SCAN_CHUNK, size - position);
207+
const buffer = Buffer.alloc(length);
208+
const {bytesRead} = await handle.read(buffer, 0, length, position);
209+
if (bytesRead === 0) break;
210+
const lastNewline = buffer.lastIndexOf(NEWLINE, bytesRead - 1);
211+
if (lastNewline < 0) {
212+
// A line that Codex is still writing is read by the next scan.
213+
if (position + bytesRead >= size) break;
214+
// The middle of a line longer than a chunk: no spawn record is that long.
215+
position += bytesRead;
216+
scan.offset = position;
217+
continue;
218+
}
219+
// Only complete lines: the rest is read with the next chunk.
220+
const view = buffer.subarray(0, lastNewline + 1);
221+
for (let at = view.indexOf(SPAWN_MARKER); at >= 0; at = view.indexOf(SPAWN_MARKER, at + SPAWN_MARKER.length)) {
222+
const id = view.toString("latin1", at + SPAWN_MARKER.length, at + SPAWN_MARKER.length + 36).toLowerCase();
223+
if (!THREAD_ID_PATTERN.test(id) || seen.has(id)) continue;
224+
const line = view.toString("utf8", view.lastIndexOf(NEWLINE, at) + 1, view.indexOf(NEWLINE, at));
225+
if (!line.includes(ownRecord) || !line.includes("\"SubAgentActivity\"")) continue;
226+
seen.add(id);
227+
scan.threadIds.push(id);
228+
}
229+
position += view.length;
230+
scan.offset = position;
231+
}
232+
return scan;
233+
} finally {
234+
await handle.close();
235+
}
236+
}

‎src/SessionIndex.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import type {ServerNotification} from "./app-server";
1515
import type {Thread, ThreadListParams, ThreadListResponse, ThreadStatus} from "./app-server/v2";
1616
import {AIR_META_KEY, JETBRAINS_META_KEY, withAirMeta} from "./AirExtension";
1717
import {normalizeSessionTitle} from "./SessionTitle";
18+
import type {SessionUsage} from "./SessionUsage";
1819
import {logger} from "./Logger";
1920

2021
export const AIR_SESSION_INDEX_KEY = "sessionIndex";
@@ -38,6 +39,7 @@ export const AIR_LAST_TURN_ENDED_AT_KEY = "lastTurnEndedAt";
3839
export const AIR_LAST_PROMPT_AT_KEY = "lastPromptAt";
3940
export const AIR_MODEL_KEY = "model";
4041
export const AIR_FORKED_FROM_KEY = "forkedFrom";
42+
export const AIR_USAGE_KEY = "usage";
4143

4244
export const DEFAULT_SESSION_INDEX_LIMIT = 50;
4345
/** The largest page that the adapter asks Codex for. */
@@ -429,6 +431,7 @@ export function sessionIndexSessionInfo(
429431
thread: Thread,
430432
archived: boolean,
431433
activity: SessionActivity | null,
434+
usage: SessionUsage | null = null,
432435
): acp.SessionInfo {
433436
const airFields: Record<string, unknown> = {[AIR_ARCHIVED_KEY]: archived};
434437
// Codex moves `recencyAt` when a turn starts and orders threads by it: the time of the last prompt.
@@ -438,6 +441,7 @@ export function sessionIndexSessionInfo(
438441
// `state` is omitted when unknown, never "unknown".
439442
if (activity?.state !== undefined) airFields[AIR_STATE_KEY] = activity.state;
440443
if (activity?.lastTurnEndedAt !== undefined) airFields[AIR_LAST_TURN_ENDED_AT_KEY] = activity.lastTurnEndedAt;
444+
if (usage !== null) airFields[AIR_USAGE_KEY] = usage;
441445
let meta: Record<string, unknown> | undefined;
442446
for (const [key, value] of Object.entries(airFields)) {
443447
meta = withAirMeta(meta, key, value);
@@ -517,7 +521,7 @@ export class SessionIndexActivity {
517521

518522
/**
519523
* What the client shows of a row, without `updatedAt`: a row whose signature did not change is not sent again.
520-
* `title` and the AIR fields `lastPromptAt`, `state`, `lastTurnEndedAt`, `model`, `forkedFrom` and `archived`.
524+
* `title` and the AIR fields `lastPromptAt`, `state`, `lastTurnEndedAt`, `model`, `forkedFrom`, `archived` and `usage`.
521525
*/
522526
export function sessionIndexRowSignature(row: acp.SessionInfo): string {
523527
const air = asRecord(asRecord(asRecord(row._meta)[JETBRAINS_META_KEY])[AIR_META_KEY]);
@@ -529,6 +533,7 @@ export function sessionIndexRowSignature(row: acp.SessionInfo): string {
529533
air[AIR_MODEL_KEY] ?? null,
530534
air[AIR_FORKED_FROM_KEY] ?? null,
531535
air[AIR_ARCHIVED_KEY] ?? null,
536+
air[AIR_USAGE_KEY] ?? null,
532537
]);
533538
}
534539

0 commit comments

Comments
 (0)