Skip to content

Commit 89d6a03

Browse files
miiiiiiichclaude
andauthored
Codex: dedupe fork-replayed history by content, not position (#37)
* Codex: dedupe fork-replayed history by content, not position Fork/spawn child rollouts replay the parent session's history with every copied line's timestamp rewritten to the fork instant, so the positional dedup key (session, timestamp, line index) counted replayed token_count events again — inflating token totals, LIMITS samples, and MODES effort mix (issue #36). - Usage events: keyed on (session, last_token_usage vector, total_token_usage vector). The cumulative is non-decreasing between compactions, so the pair identifies one turn's consumption across every file it appears in; re-emissions with an unchanged cumulative collapse too. Logs without the cumulative fall back to the positional key. - Rate-limit samples: same usage fingerprint plus the snapshot's own fields (used_percent, window_minutes, resets_at) so a verbatim replay collapses while a genuine re-notification survives. - Effort events: keyed by turn_context.payload.turn_id, which replays copy verbatim; positional fallback for logs predating turn_id. - merge_into: keyed duplicates keep the earliest observed timestamp — scan order (archived_sessions sorts before sessions wholesale) can't let a replay shift an event's day attribution. The five dedup loops fold into a shared dedupe_into helper. - CACHE_VERSION 9 -> 10: cached FileEvents carry the old positional keys, so parsing semantics changed even though the layout did not. Known remainder (follow-up): session touches and task durations from replayed lines are still counted. Co-Authored-By: Claude Fable 5 <[email protected]> * Codex: harden semantic keys against malformed payloads Address second-model review of the GH-36 fix: - Gate the semantic-key path on a fingerprintable usage object (total_tokens parses as an integer) — total_token_usage null/{} would fingerprint as all-zero and falsely collide distinct turns. Same gate on the rate-limit key's last_token_usage. - Clamp used_percent before keying so an out-of-range raw value can't mint a second key for the same stored sample value. - Soften the CHANGELOG attribution claim: earliest-timestamp-wins holds when both original and replay are scanned; an out-of-window original leaves the replay's fork-instant timestamp (known remainder). - Pin the review-raised invariants in tests: malformed-cumulative fallback, rate-limit re-notification survival, cross-turn re-emission collapse (why turn_id is not in the usage key), same-timestamp distinct turn_ids, cache_write discrimination, and Claude fragment merge (larger volume + earliest timestamp + merged metadata). Co-Authored-By: Claude Fable 5 <[email protected]> * Codex: skip the fork replay burst structurally Layer a parse-time skip under the content-based keys (GH-36) — the approach ccusage took for the same bug (ccusage/ccusage#950) — with a broader trigger: a second session_meta marks a fork-shaped rollout (catches Desktop thread branches that carry no thread_spawn marker), and event lines stamped in the same second as that meta are the replay burst. - Replayed history now contributes nothing even when the parent rollout is outside the scan (deleted or past the mtime window), where cross-file dedup has no original to collapse into and previously the replay was counted once at the fork instant. - Durations (unkeyed) and session touches from replayed lines stop counting, closing two known remainders. - Content-based keys stay as the safety net for burst lines that leak past the fork second, whole-file copies, and re-emissions. Real-data check: same-instant A/B with and without the skip yields byte-identical Codex totals on an all-parents-in-scan corpus — the skip only changes outcomes where originals are missing from the scan. Co-Authored-By: Claude Fable 5 <[email protected]> --------- Co-authored-by: Claude Fable 5 <[email protected]>
1 parent 86769ac commit 89d6a03

3 files changed

Lines changed: 695 additions & 104 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,25 @@ All notable changes to this project are documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [Unreleased]
9+
10+
### Fixed
11+
12+
- Codex fork/spawn child rollouts replay the parent session's history with
13+
timestamps rewritten to the fork instant, and the positional dedup key
14+
(session, timestamp, line index) counted the replayed events again —
15+
inflating token totals, LIMITS samples, and MODES effort mix ([#36]).
16+
Usage, rate-limit, and effort events are now deduplicated by content
17+
(usage vectors + running cumulative; `turn_id` for effort), re-emitted
18+
`token_count` events whose cumulative did not advance collapse too, and
19+
keyed duplicates keep the earliest observed timestamp, so scan order
20+
can't decide attribution. Fork-shaped rollouts (second `session_meta`)
21+
additionally skip the replay burst at the fork instant wholesale — so
22+
replayed history contributes nothing (tokens, rate-limit samples,
23+
efforts, durations, session touches) even when the parent rollout is
24+
outside the scan window, and Desktop thread branches without a
25+
`thread_spawn` marker are covered too.
26+
827
## [0.10.0] - 2026-07-13
928

1029
### Changed
@@ -262,6 +281,8 @@ First public release with the codename system and the shareable stats card.
262281

263282
Initial npm packaging.
264283

284+
[#36]: https://github.com/miiiiiiich/agent-walker/issues/36
285+
[Unreleased]: https://github.com/miiiiiiich/agent-walker/compare/v0.10.0...HEAD
265286
[0.10.0]: https://github.com/miiiiiiich/agent-walker/releases/tag/v0.10.0
266287
[0.9.0]: https://github.com/miiiiiiich/agent-walker/releases/tag/v0.9.0
267288
[0.8.0]: https://github.com/miiiiiiich/agent-walker/releases/tag/v0.8.0

0 commit comments

Comments
 (0)