Repository navigation
Commit e4f0ea9
authored
fix(codex): count spawned subagent requests stamped at the fork instant (ccusage#2051)
* fix(codex): count spawned subagent requests stamped at the fork instant
When a forked rollout's parent log is unavailable, the parser falls back
to skipping the "rewritten burst": the run of usage events Codex writes
back to back at the fork instant when it copies the parent's history.
The burst only ends at a pause of more than a second, because the
heuristic assumes the child's own turns follow the replay after a real
delay.
Some Codex VS Code rollouts (0.164.0-alpha.11.1) carry the session_meta
timestamp on every line, including the child's own token_count events.
With no pause anywhere, the burst runs to the end of the file and the
whole rollout counts as zero, even though its token_usage_record lines
have response IDs that exist in no other rollout.
Timestamps cannot separate replay from the child's own usage in such a
file, but the records can. Since TokenUsageRecord was introduced
(openai/codex#41912), Codex drops the parent's records from the history
it hands a spawned subagent ("child threads inherit model context, not
the parent's cumulative usage state"). So in a session whose session_meta
names a source.subagent.thread_spawn parent, a token_usage_record is a
request the child sent itself, and nothing after it can be replayed
history. The replay filter now ends at that record, whether it was
matching the parent prefix or skipping a rewritten burst.
User forks (thread/fork) copy every rollout item, records included, so
for them a record proves nothing and the existing timestamp handling is
unchanged. A regression test pins that double-count protection.
To carry the spawn signal, the replay plan now hands the parser a
CodexForkReplay (prefix plus whether records end the replay), and the
replay state machine moves into a small filter so the main loop can
observe records alongside usage events.
Closes ccusage#2046
* fix(codex): keep identical requests from one rollout distinct
Codex deduplicates usage that one rollout copied from another, because
replayed parent history repeats the parent's token_count events. Without
a response ID to compare, both dedupe paths (the event loader and the
report aggregation) keyed a usage event on its timestamp, model, and
token counts alone. The key carried no record of which file the event
came from, so two requests in the same rollout with identical usage at
the same timestamp were treated as a copy and merged.
That is exactly the shape from ccusage#2046: a spawned subagent whose every
line carries the fork instant. Its two identical requests were counted
once, reporting 8,000 non-cached input where 16,000 was spent.
A rollout records each request once, so identical events within one file
are distinct requests, while a copy is the same request seen in another
file. Each dedupe path now numbers the identical usage events of a file
in order and puts that ordinal in the key. A copy repeats the original's
ordinals and still collapses onto it, so copied history is counted once,
and a parent's repeated identical requests stay distinct when copied
together. Response-ID keys are unchanged.
In the report path the ordinal also keeps recorded service tiers per
request, because each distinct request now owns its own dedupe record.
The loader keeps events grouped by file until dedupe, so it has the same
provenance as the aggregation.
Measured with release builds over 2.0G of real Codex logs: daily runs in
3.58s against 3.88s on main (within noise). Totals rise by 29.7M tokens
in 14 sessions, none fall. All 14 are spawned subagents with one
timestamp throughout, and none of their response IDs appears in any of
the other 3,980 rollouts, so the added usage is the child's own.
Refs ccusage#2046
* test(codex): keep a spawned child's own usage before its first record
A spawned subagent never inherits the parent's token_usage_record lines,
so a record proves that everything after it is the child's own usage.
It does not prove the converse. Codex releases before openai/codex#41912
wrote no records, and a child created by one of them and resumed by a
later release holds its own record-less turns ahead of the first record.
A real rollout has this shape: a thread_spawn child created by Codex
0.149 that spent 2.8M tokens over several minutes, then was resumed two
days later by a release that writes records. Skipping everything before
the first record would drop all of that usage, so a record may end a
replay skip but never start one. This test pins that boundary.
* docs(codex): document fork replay boundaries and copy dedupe
The adapter README listed only the MultiAgent V2 markers and said replay
boundaries never come from session_meta. Spawned subagents now end a
replay skip at their first token_usage_record, which depends on knowing
from session_meta that the rollout came from thread_spawn.
That rule targets cli_version, which resume re-records. The fork source
identifies how the rollout was created and does not change on resume,
so the README now says the version rule covers version fields only. It
also describes each fork boundary, why a record never starts a skip,
and how usage dedupe keys on an event's ordinal within its own file.
* test(codex): cover the ccusage#2046 rollout in every Codex report mode
The adapter tests drive the event loader and the session aggregation,
but each Codex report mode groups usage on its own path, and the copy
dedupe runs inside that aggregation. A regression in how one mode keys
or groups the two identical requests would pass the adapter tests.
Run the issue's exact rollout through the CLI for daily, monthly, and
session (Codex has no weekly report). Assert JSON rows and totals at
16,000 input, 4,000 cache read, and 200 output, and snapshot each table.
* test(codex): keep the post-record event inside the burst window
The resumed spawned-child test pins that a token_usage_record never
starts a replay skip. Its post-record token_count sat 1.86s after the
record, outside the 1s rewritten-burst window. A broken rule that
started a burst skip at the record would therefore still count that
event, and the test would pass.
Move the event to 800ms after the record, so it falls inside that
window. Checked against two deliberately broken parsers: one starting a
burst skip at the record and one skipping all usage before the first
record. The test now fails under both; with the original timestamp it
passed under the first.
Reported by Pullfrog on ccusage#2051.1 parent 908daef commit e4f0ea9
11 files changed
Lines changed: 640 additions & 164 deletions
File tree
- rust
- adapters/codex/src
- crates/ccusage/tests
- fixtures/codex
- snapshots
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
56 | | - | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
57 | 64 | | |
58 | 65 | | |
59 | 66 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
42 | 98 | | |
43 | 99 | | |
44 | 100 | | |
| |||
243 | 299 | | |
244 | 300 | | |
245 | 301 | | |
| 302 | + | |
246 | 303 | | |
247 | 304 | | |
248 | 305 | | |
249 | | - | |
| 306 | + | |
250 | 307 | | |
251 | 308 | | |
252 | 309 | | |
| |||
261 | 318 | | |
262 | 319 | | |
263 | 320 | | |
264 | | - | |
| 321 | + | |
265 | 322 | | |
266 | 323 | | |
267 | 324 | | |
| |||
293 | 350 | | |
294 | 351 | | |
295 | 352 | | |
| 353 | + | |
296 | 354 | | |
297 | 355 | | |
298 | 356 | | |
299 | | - | |
| 357 | + | |
300 | 358 | | |
301 | 359 | | |
302 | 360 | | |
| |||
310 | 368 | | |
311 | 369 | | |
312 | 370 | | |
313 | | - | |
| 371 | + | |
314 | 372 | | |
315 | 373 | | |
316 | 374 | | |
| |||
323 | 381 | | |
324 | 382 | | |
325 | 383 | | |
326 | | - | |
| 384 | + | |
327 | 385 | | |
328 | 386 | | |
329 | 387 | | |
330 | 388 | | |
331 | 389 | | |
332 | 390 | | |
333 | 391 | | |
334 | | - | |
| 392 | + | |
335 | 393 | | |
336 | 394 | | |
337 | 395 | | |
| |||
351 | 409 | | |
352 | 410 | | |
353 | 411 | | |
354 | | - | |
| 412 | + | |
355 | 413 | | |
356 | 414 | | |
357 | 415 | | |
358 | 416 | | |
359 | 417 | | |
360 | 418 | | |
361 | 419 | | |
362 | | - | |
| 420 | + | |
363 | 421 | | |
364 | 422 | | |
365 | 423 | | |
366 | 424 | | |
367 | 425 | | |
368 | 426 | | |
369 | 427 | | |
370 | | - | |
| 428 | + | |
371 | 429 | | |
372 | 430 | | |
373 | 431 | | |
| |||
697 | 755 | | |
698 | 756 | | |
699 | 757 | | |
700 | | - | |
| 758 | + | |
701 | 759 | | |
702 | | - | |
| 760 | + | |
703 | 761 | | |
704 | 762 | | |
705 | 763 | | |
| |||
710 | 768 | | |
711 | 769 | | |
712 | 770 | | |
713 | | - | |
| 771 | + | |
714 | 772 | | |
715 | 773 | | |
716 | 774 | | |
| |||
782 | 840 | | |
783 | 841 | | |
784 | 842 | | |
| 843 | + | |
785 | 844 | | |
786 | 845 | | |
787 | 846 | | |
| |||
986 | 1045 | | |
987 | 1046 | | |
988 | 1047 | | |
989 | | - | |
| 1048 | + | |
990 | 1049 | | |
991 | 1050 | | |
992 | 1051 | | |
| |||
1864 | 1923 | | |
1865 | 1924 | | |
1866 | 1925 | | |
| 1926 | + | |
| 1927 | + | |
| 1928 | + | |
| 1929 | + | |
| 1930 | + | |
| 1931 | + | |
| 1932 | + | |
| 1933 | + | |
| 1934 | + | |
| 1935 | + | |
| 1936 | + | |
| 1937 | + | |
| 1938 | + | |
| 1939 | + | |
| 1940 | + | |
| 1941 | + | |
| 1942 | + | |
| 1943 | + | |
| 1944 | + | |
| 1945 | + | |
| 1946 | + | |
| 1947 | + | |
| 1948 | + | |
| 1949 | + | |
| 1950 | + | |
| 1951 | + | |
| 1952 | + | |
| 1953 | + | |
| 1954 | + | |
| 1955 | + | |
| 1956 | + | |
| 1957 | + | |
| 1958 | + | |
| 1959 | + | |
| 1960 | + | |
| 1961 | + | |
| 1962 | + | |
| 1963 | + | |
| 1964 | + | |
| 1965 | + | |
| 1966 | + | |
| 1967 | + | |
| 1968 | + | |
| 1969 | + | |
| 1970 | + | |
| 1971 | + | |
| 1972 | + | |
| 1973 | + | |
| 1974 | + | |
| 1975 | + | |
| 1976 | + | |
| 1977 | + | |
| 1978 | + | |
| 1979 | + | |
| 1980 | + | |
| 1981 | + | |
| 1982 | + | |
| 1983 | + | |
| 1984 | + | |
| 1985 | + | |
| 1986 | + | |
| 1987 | + | |
| 1988 | + | |
| 1989 | + | |
| 1990 | + | |
| 1991 | + | |
| 1992 | + | |
| 1993 | + | |
| 1994 | + | |
| 1995 | + | |
| 1996 | + | |
| 1997 | + | |
| 1998 | + | |
| 1999 | + | |
| 2000 | + | |
| 2001 | + | |
| 2002 | + | |
| 2003 | + | |
| 2004 | + | |
| 2005 | + | |
| 2006 | + | |
| 2007 | + | |
| 2008 | + | |
| 2009 | + | |
| 2010 | + | |
| 2011 | + | |
| 2012 | + | |
| 2013 | + | |
| 2014 | + | |
| 2015 | + | |
| 2016 | + | |
| 2017 | + | |
| 2018 | + | |
| 2019 | + | |
| 2020 | + | |
| 2021 | + | |
| 2022 | + | |
| 2023 | + | |
| 2024 | + | |
| 2025 | + | |
| 2026 | + | |
| 2027 | + | |
| 2028 | + | |
| 2029 | + | |
| 2030 | + | |
| 2031 | + | |
| 2032 | + | |
| 2033 | + | |
| 2034 | + | |
| 2035 | + | |
| 2036 | + | |
| 2037 | + | |
| 2038 | + | |
| 2039 | + | |
| 2040 | + | |
| 2041 | + | |
| 2042 | + | |
| 2043 | + | |
| 2044 | + | |
| 2045 | + | |
| 2046 | + | |
| 2047 | + | |
| 2048 | + | |
| 2049 | + | |
| 2050 | + | |
1867 | 2051 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| |||
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
56 | | - | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
57 | 60 | | |
58 | 61 | | |
59 | 62 | | |
| |||
0 commit comments