Repository navigation
feat(codex): include archived sessions - #1230
Conversation
Load archived_sessions next to a Codex home sessions directory so archived Codex JSONL files remain visible in focused and unified reports. Keep direct JSONL directory CODEX_HOME entries working by only falling back to the home path when neither sessions nor archived_sessions exists. Document the additional Codex source and cover path discovery with a fixture-backed unit test.
|
@coderabbitai please review this PR. It adds archived Codex session discovery and keeps the existing direct JSONL directory fallback intact. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThis PR extends the Codex session discovery logic to recognize ChangesCodex archived_sessions support
sequenceDiagram
participant CLI
participant codex_usage_paths_from_homes
participant FS
CLI->>codex_usage_paths_from_homes: call with CODEX_HOME entries
codex_usage_paths_from_homes->>FS: check "<home>/sessions/" existence
codex_usage_paths_from_homes->>FS: check "<home>/archived_sessions/" existence
codex_usage_paths_from_homes->>CLI: return deduplicated list of usage paths
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ccusage-guide | b01fd43 | Commit Preview URL Branch Preview URL |
Jun 08 2026, 12:37 AM |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Fetch PR details and diff
gh pr view 1230 --repo ryoppippi/ccusage --json title,body,files,additions,deletions,headRefNameLength of output: 2457 🏁 Script executed: #!/bin/bash
# Get the diff for the PR
gh pr diff 1230 --repo ryoppippi/ccusageLength of output: 4041 🏁 Script executed: #!/bin/bash
# Read the codex adapter file
cat rust/crates/ccusage/src/adapter/codex.rsLength of output: 402 🏁 Script executed: #!/bin/bash
# Read the codex loader file
cat rust/crates/ccusage/src/codex_loader.rsLength of output: 399 🏁 Script executed: #!/bin/bash
# Check for any tests related to archived sessions
rg -n "archived" --type rustLength of output: 988 🏁 Script executed: #!/bin/bash
# Check the AGENTS.md file in adapter directory
cat rust/crates/ccusage/src/adapter/AGENTS.mdLength of output: 2303 🏁 Script executed: #!/bin/bash
# Check openclaw paths.rs for test patterns comparison
cat rust/crates/ccusage/src/adapter/openclaw/paths.rsLength of output: 2774 🏁 Script executed: #!/bin/bash
# Check the full codex paths.rs for context
cat rust/crates/ccusage/src/adapter/codex/paths.rsLength of output: 2191 🏁 Script executed: #!/bin/bash
# Check ccusage_test_support Fixture API
fd -t f "lib.rs" --search-path rust/crates/ccusage-test-support/
fd -t f "*.rs" --search-path rust/crates/ccusage-test-support/Length of output: 751 🏁 Script executed: #!/bin/bash
cat rust/crates/ccusage-test-support/src/lib.rsLength of output: 2457
Review✅ LogicThe
The
|
Add fixture-backed coverage for sessions-only, archived-sessions-only, and direct JSONL directory fallback path discovery. This addresses the CodeRabbit review gap while preserving the archived session discovery behavior added in the prior commit.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/guide/codex/index.md`:
- Line 26: Update the paragraph to reflect the actual discovery logic from
paths.rs: state that both sessions/ and archived_sessions/ are discovered
independently (an entry with only archived_sessions/ adds that directory rather
than the parent), that archived_sessions/ is not contingent on the presence of
sessions/, and that the JSONL-directory fallback (reading the entry directly as
a JSONL directory) only happens when neither sessions/ nor archived_sessions/
exist; reference CODEX_HOME and the directories sessions/, archived_sessions/,
and the JSONL fallback so readers can map this to the implementation.
In `@rust/crates/ccusage/src/adapter/codex/paths.rs`:
- Around line 55-70: Add four new fixture-backed tests for
codex_usage_paths_from_homes: (1)
falls_back_to_home_when_neither_sessions_nor_archived_exists — create only
"codex" and assert result == vec![fixture.path("codex")]; (2)
includes_only_sessions_when_archived_does_not_exist — create "codex/sessions"
and assert result == vec![fixture.path("codex/sessions")]; (3)
includes_only_archived_when_sessions_does_not_exist — create
"codex/archived_sessions" and assert result ==
vec![fixture.path("codex/archived_sessions")]; and (4)
deduplicates_across_multiple_homes — create "codex/sessions", call
codex_usage_paths_from_homes with vec![path.clone(), path.clone()] and assert
result == vec![fixture.path("codex/sessions")]; use the existing Fixture::new()
and Fixture.create_dir_all calls and the same assertion style as the existing
includes_archived_sessions_next_to_sessions test to validate fallback and
seen-HashSet deduplication.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 455ca464-b984-4c5f-89f4-4a851e6611b4
📒 Files selected for processing (3)
docs/guide/codex/index.mdrust/crates/ccusage/src/adapter/codex/README.mdrust/crates/ccusage/src/adapter/codex/paths.rs
|
@coderabbitai addressed your test coverage gap by adding fixture-backed cases for sessions-only, archived-sessions-only without direct fallback, and direct JSONL directory fallback. Validation passed: direnv exec . cargo test -p ccusage adapter::codex and pre-push hooks including clippy and cargo test. |
|
Tip For best results, initiate chat on the files or code changes.
🐇🎉 🧠 Learnings usedIf you found this review helpful, would you consider giving us a shout-out on X? Thank you for using CodeRabbit! |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — adds archived_sessions/ discovery to the Codex path scanner so archived session JSONL files are included in usage reports.
- Add
archived_sessions/tocodex_usage_paths— the path scanner now checks forarchived_sessions/alongsidesessions/under eachCODEX_HOMEdirectory. The direct JSONL fallback (for savedcodex exec --jsonoutput) now only triggers when neithersessions/norarchived_sessions/exists. Extractedcodex_usage_paths_from_homesas a pure helper for testability. - Document the archived source — updated the Codex guide's Data Source section and the adapter
README.md.
DeepSeek Pro (free via Pullfrog for OSS) ｜ 𝕏
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 3 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
ccusage
@ccusage/ccusage-darwin-arm64
@ccusage/ccusage-darwin-x64
@ccusage/ccusage-linux-arm64
@ccusage/ccusage-linux-x64
@ccusage/ccusage-win32-arm64
@ccusage/ccusage-win32-x64
commit: |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
Clarify Codex home discovery documentation so sessions, archived_sessions, and the direct JSONL fallback match the implementation. Add fixture-backed coverage for duplicate CODEX_HOME entries to pin the seen-set behavior requested in review.
|
@coderabbitai please review the latest push b01fd43. It addresses the docs discovery wording and adds the duplicate CODEX_HOME path dedupe test requested in your review. |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Fetch the specific commit diff
gh api repos/ryoppippi/ccusage/commits/b01fd43b --jq '.files[] | {filename: .filename, status: .status, additions: .additions, deletions: .deletions, patch: .patch}'Length of output: 2130 🏁 Script executed: #!/bin/bash
# Also get the current state of paths.rs and docs
cat rust/crates/ccusage/src/adapter/codex/paths.rsLength of output: 3523 🏁 Script executed: #!/bin/bash
# Check the docs file
cat docs/guide/codex/index.mdLength of output: 6841
✅ Docs wording (
|
| Test | Scenario |
|---|---|
includes_archived_sessions_next_to_sessions |
both subdirs present |
uses_sessions_without_missing_archived_sessions_path |
sessions-only |
uses_archived_sessions_without_direct_path_fallback |
archived-only, no fallback |
falls_back_to_direct_path_when_no_session_directories_exist |
neither subdir → direct fallback |
deduplicates_usage_paths_across_repeated_homes (new) |
repeated CODEX_HOME entry deduped |
All five cases are fixture-backed. This PR is ready to merge. 🎉
🐇 ✨
[approve]
🧠 Learnings used
Learnt from: ryoppippi
Repo: ryoppippi/ccusage PR: 0
File: :0-0
Timestamp: 2026-05-22T15:24:46.102Z
Learning: In `ryoppippi/ccusage`, the `load_groups` function in `rust/crates/ccusage/src/adapter/codex.rs` has a single-CODEX_HOME fast path guarded by `paths.len() == 1 && !wants_json(shared)`. The worker-local aggregation fast path is intentionally restricted to human-table output only; JSON/jq output always uses the global-dedupe aggregation path to avoid cross-chunk fallback cost after all chunks are processed.
Learnt from: CR
Repo: ryoppippi/ccusage PR: 0
File: rust/crates/ccusage/src/adapter/AGENTS.md:0-0
Timestamp: 2026-05-23T23:21:58.299Z
Learning: Applies to rust/crates/ccusage/src/adapter/**/*.rs : Add Rust fixture-backed tests for path discovery, parser behavior, aggregation totals, and important legacy compatibility.
Learnt from: ryoppippi
Repo: ryoppippi/ccusage PR: 0
File: :0-0
Timestamp: 2026-05-22T14:31:40.598Z
Learning: In `ryoppippi/ccusage`, the worker-local aggregation fallback in `aggregate_files_parallel_local` (rust/crates/ccusage/src/adapter/codex.rs) is intentionally left without observability instrumentation. The local-dedupe fast path is optimized for the common single-`CODEX_HOME` case where cross-chunk duplicates are rare; the fallback re-runs the full global-dedupe pass to preserve correctness without adding per-event retained state to the hot path. This is a deliberate design trade-off by the maintainer.
Learnt from: ryoppippi
Repo: ryoppippi/ccusage PR: 0
File: :0-0
Timestamp: 2026-05-22T17:26:33.070Z
Learning: In `ryoppippi/ccusage`, `load_groups_from_directory` in `rust/crates/ccusage/src/adapter/codex.rs` no longer has a worker-local parallel aggregation path. As of commit 695a46e, the parallel branch always uses `create_dedupe_shards()` + `aggregate_files_parallel` (global dedupe in one pass). The previous `aggregate_files_parallel_local` function (with its cross-chunk reread fallback) was removed entirely.
Learnt from: ryoppippi
Repo: ryoppippi/ccusage PR: 0
File: :0-0
Timestamp: 2026-05-22T14:48:37.050Z
Learning: In `ryoppippi/ccusage`, the `codex_line_usage_kind` classifier in `rust/crates/ccusage/src/codex_loader.rs` uses compact JSON key-value finders (`"type":"turn_context"`, `"type":"event_msg"`, `"type":"token_count"`) rather than bare substring finders. A line is classified as a Codex session row only when `"type":"turn_context"` is present, or when both `"type":"event_msg"` AND `"type":"token_count"` appear together. This was tightened in commit 9b83895 to prevent headless rows with `token_count` in arbitrary content fields from being misrouted to the session parser.
Learnt from: ryoppippi
Repo: ryoppippi/ccusage PR: 1062
File: rust/crates/ccusage/src/claude_loader.rs:919-929
Timestamp: 2026-05-19T13:00:16.948Z
Learning: In `rust/crates/ccusage/src/claude_loader.rs`, `extract_project()` returns only the first path segment after `projects/`. As a result, composite/non-segment project filters (e.g. "project-a/session-a") intentionally yield empty results because `entry.project` (a single segment) will never match a multi-segment filter string. The `usage_files` function's full-discovery fallback for non-segment filters preserves this pre-existing behavior to maintain parity with bunx ccusage. Do not suggest bypassing the exact-match filter for non-segment inputs, as that would break parity.
Learnt from: ryoppippi
Repo: ryoppippi/ccusage PR: 1040
File: apps/ccusage/src/commands/index.ts:175-207
Timestamp: 2026-05-17T18:49:01.449Z
Learning: In ryoppippi/ccusage, the top-level agent commands in `apps/ccusage/src/commands/index.ts` (claude, codex, opencode, amp, pi) are intentionally wired to their respective daily leaf runners (e.g., `codexDailyCommand`, `opencodeDailyCommand`) as the parent command's handler. Invoking `ccusage <agent>` without a report subcommand deliberately defaults to showing the daily report — this preserves the pre-PR behavior of `normalizeAgentCommandArgs` (which mapped `agent [--flags]` → `agent:daily [--flags]`). Do not flag the absence of an `ctx.omitted` guard or a dedicated parent-only command in these entries; the default-daily behavior is intentional and covered by cli-output.test.ts.
Learnt from: ryoppippi
Repo: ryoppippi/ccusage PR: 0
File: :0-0
Timestamp: 2026-05-22T16:02:17.988Z
Learning: In `ryoppippi/ccusage`, `visit_codex_session_file` in `rust/crates/ccusage/src/codex_loader.rs` uses bounded-memory streaming via `BufReader::with_capacity(128 * 1024, file)` + `read_until(b'\n', &mut line)` with `line.clear()` each iteration (restored in commit ebf035f). The `fs::read` / `byte_lines` (full-file-in-memory) approach was deliberately reverted to avoid memory spikes on large JSONL files.
Learnt from: ryoppippi
Repo: ryoppippi/ccusage PR: 0
File: :0-0
Timestamp: 2026-05-22T14:48:37.050Z
Learning: In `ryoppippi/ccusage`, session timestamps in `rust/crates/ccusage/src/codex_loader.rs` use a dedicated `codex_session_timestamp` helper (introduced in commit 9b83895) that fast-paths string timestamps (returned trimmed as-is, no normalization overhead) and only delegates numeric timestamps to `normalize_codex_timestamp`. This was added to avoid the normalization cost on the common string-timestamp case in the hot session parsing path.
Learnt from: ryoppippi
Repo: ryoppippi/ccusage PR: 0
File: :0-0
Timestamp: 2026-05-22T17:26:33.070Z
Learning: In `ryoppippi/ccusage`, malformed headless Codex rows in `rust/crates/ccusage/src/codex_loader.rs` are handled by a two-tier fallback (added in commit 695a46e): first attempt typed `CodexLogEntry<'_>` deserialization; on failure, fall back to `add_codex_exec_event_from_value` which parses via `serde_json::Value` and uses `_value` helpers (`normalize_value_timestamp`, `codex_model_from_result_value`, `normalize_headless_codex_usage_value`) that mirror the typed path. The `usage_from_value` helper uses `serde_json::from_value(v.clone()).ok()` to reuse the `CodexRawUsage` deserializer including the `total_tokens: 0` guard.
Learnt from: CR
Repo: ryoppippi/ccusage PR: 0
File: rust/crates/ccusage/src/adapter/AGENTS.md:0-0
Timestamp: 2026-05-23T23:21:58.299Z
Learning: Applies to rust/crates/ccusage/src/adapter/rust/crates/ccusage/src/**/*.rs : Add or update CLI JSON assertions and table snapshots for affected report modes.
Learnt from: CR
Repo: ryoppippi/ccusage PR: 0
File: rust/crates/ccusage/src/adapter/AGENTS.md:0-0
Timestamp: 2026-05-23T23:21:58.299Z
Learning: Applies to rust/crates/ccusage/src/adapter/**/*.rs : Add skipped local-data smoke tests when real user log directories are useful for catching schema drift.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — refined the Codex guide paragraph to describe independent sessions//archived_sessions/ discovery and added a deduplication test for repeated CODEX_HOME entries.
- Refined the Codex Data Source paragraph in
docs/guide/codex/index.md— the new wording describessessions/andarchived_sessions/discovery as independent so an entry with onlyarchived_sessions/clearly contributes logs. - Added
deduplicates_usage_paths_across_repeated_homestest inpaths.rs— verifies theFxHashSetdeduplication prevents double-counting when the sameCODEX_HOMEdirectory appears multiple times.
DeepSeek Pro (free via Pullfrog for OSS) ｜ 𝕏
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
Rebase the remaining ccusage#1176 behavior on top of the archived_sessions support that landed in ccusage#1230. Collect Codex JSONL files through scoped usage sources so an active sessions/ file wins over an archived_sessions/ file with the same relative path within the same CODEX_HOME. Keep different CODEX_HOME roots independent, and use PathBuf keys instead of lossy string conversion for relative paths.
Keep active sessions/ JSONL files when archived_sessions/ has the same relative path within one CODEX_HOME, while preserving separate CODEX_HOME roots and non-lossy PathBuf relative keys.\n\nRebased and scoped from #1176 after #1230 landed the basic archived_sessions discovery.\n\nCo-authored-by: yashau <[email protected]>
The generated Codex fixture is mostly zstd-compressed rollouts, some in `archived_sessions/`. Releases before #1890 skip `.jsonl.zst` files (and before #1230 skip archived sessions), so a history run across that change compared an old ref reading a fraction of the input with a new ref reading all of it, and reported the difference as a CPU and RSS regression. The generator gains `--codex-layout legacy`, which writes the same rollouts uncompressed under `sessions/`. bench-history keeps the current layout when every ref is a commit containing #1890, and otherwise generates the legacy layout and says so at the top of the report; binary paths count as unknown. With a custom Codex fixture that holds `.zst` files and a ref that cannot read them, and for `--real` runs across the change, the report leads with a warning instead. The PR comparison always builds refs that read the current layout, so CI keeps compressed coverage. Reported by CodeRabbit on #2061.
The compatibility check only looked for `.jsonl.zst` files, but releases before #1230 also skip `archived_sessions/`, so a custom fixture with archived plain rollouts got no warning and a history run across #1230 compared different input. Replace the single zstd commit with a table of Codex input classes taken from the Codex loader's history (`paths.rs`, `reader.rs`): archived sessions (a9b12c4, #1230, v20.0.7) and zstd-compressed rollouts (44a3aa8, #1890, v20.0.33). Each ref is checked per class by git ancestry; the generated fixture switches to the legacy layout when any ref skips any class, and a custom fixture or `--real` run warns per class with the refs that skip it. Claude discovery has scanned `projects/` recursively throughout v20 and the date-window skips (#1665, #1794) do not apply to unbounded reports, so they are not classes. Probing each ref's source for the reader code was rejected: the same strings appear in tests and comments. Reported by Cubic on #2061.
#2061) * build(perf): shape the large fixture like current Claude Code and Codex logs The CI performance comparison missed the double parse that #1719 added to every Claude line containing `null`, because the generated fixture never contained `null`: each line was a minimal assistant usage record padded with a single text block of `x` characters. Real Claude Code lines now carry `stop_sequence`, `container`, `stop_details` and other null-valued fields, nested `usage.iterations` and cache-creation TTL splits, and most lines are not usage records at all. On the old fixture v20.0.20 -> v20.0.21 moved CPU time by +8%; on real-shaped lines it is +55%. The generator now writes sessions whose line types, per-type byte share, and field shapes follow aggregate statistics taken from real logs (structure only; every value is synthesized from a seeded PRNG, so the output is deterministic): - Claude: user/tool-result, attachment, api-request(-blob) and metadata lines around assistant responses written one line per content block with repeated usage, null-valued fields, `usage.iterations` including occasional advisor iterations, 5m/1h cache-creation split, escaped and non-ASCII text, 48 projects, sidechain `subagents/agent-*.jsonl` files that replay parent message IDs, and legacy files with `progress` lines wrapping AgentProgress assistant messages. - Codex: dated `sessions/YYYY/MM/DD/rollout-*.jsonl` files with session_meta, turn_context, reasoning, tool call/output and item_completed lines, plus token_count (with null-heavy rate limits) and token_usage_record per response. As in real Codex homes, rollouts older than two weeks are zstd-compressed (`.jsonl.zst`) and some sit in `archived_sessions/`; on real data about 90% of Codex bytes are now read through zstd, a path the old fixture never touched. The file-size distribution (of uncompressed bytes) and the CLI are unchanged. * build(perf): build base and PR from source and judge them by CPU time The PR performance comment could not show a regression or a real improvement: - Unequal artifacts. Base ran the published `ccusage` npm package (Node launcher spawning the native binary) while the Rust leg ran the bare native binary, so every PR showed a ~2.6x "win" on the committed fixtures that was only launcher startup. - One sample. The large fixture ran with 0 warmups and 1 run, and the base and PR runs were not interleaved, so runner noise dominated. - The committed fixtures are a few KiB and measure process startup. - No verdict. Nothing said whether a number was a regression. The job now builds both binaries with `nix build .#ccusage-static`, the derivation the Linux x64 native package ships, on the same runner: the PR head from the checkout and the base from `github.event.pull_request.base.sha`, the same base commit as before. The walk back over pricing-only commits only existed to find a commit with a pkg.pr.new package, so it goes. The job reuses the build-native-packages sticky disk for dependency artifacts and no longer waits for pkg.pr.new, so the JS launcher leg, the committed-fixture section, and the package diagnostics are gone too. The harness runs `claude daily`, `claude session`, `codex daily` and the unified `daily` over the realistic generated fixture in interleaved rounds (each round one hyperfine invocation with both binaries, order rotated), recording wall time, CPU time, peak RSS and, where the platform exposes it, instructions. It reports medians with min-max spread and per-round PR/base ratios. A command is flagged when the median CPU-time or peak-RSS ratio moves past 10% and at least four in five rounds agree; the verdict line and a GitHub warning annotation say so. Wall time is shown but not judged: it moves with runner load, while CPU time paired within a round does not. The check warns rather than fails: a failing check needs a calibrated false-positive rate across many runs on these runners and an escape hatch for intentional trade-offs, and neither exists yet. * build(perf): add `just bench-history` to compare refs Finding which release introduced a slowdown meant building each tag by hand and timing it ad hoc. `just bench-history <ref|binary>...` builds each ref in release mode into `rust/target/bench-history` (one shared cargo target dir, binaries cached by commit), generates the realistic fixture once per generator revision and size, and runs every report command over all refs in the same interleaved rounds as the PR comparison. Each ref is judged against the previous one with the same CPU-time/peak-RSS rule, so a regression shows up at the step that introduced it. `--real` adds a section over the logs the current environment points at. It reuses the PR harness modules instead of adding a parallel tool, and the profile skill now points at it. * fix(perf): print the regression annotation on stdout GitHub Actions reads workflow commands only from stdout, but the `::warning` line was printed while `*out*` was bound to `*err*`, so a flagged regression never produced an annotation. Print it on stdout and pass the GitHub Actions check in, so a test can assert the stream. Reported by Pullfrog on #2061. * fix(perf): reject bench-history runs below the minimum `parse-history-cli` validated a copy with `:runs` raised to 3 but returned the original options, so `--runs 2` passed validation and then measured two rounds, too few for the four-in-five agreement rule. Validate the options that are returned, so fewer than three rounds is rejected as in the PR comparison, and say so in both `--runs` helps. Reported by Pullfrog on #2061. * fix(perf): run fixture benchmarks in an empty environment The fixture runs pointed Claude and Codex at the fixture and moved HOME and XDG_* to an empty directory, but every other agent has its own data-dir variable (AMP_DATA_DIR, OPENCODE_DATA_DIR, PI_AGENT_DIR, GEMINI_DATA_DIR and a dozen more), and a value set in the caller's shell still pointed the unified `daily` report at real logs. Listing every variable would rot as agents are added, so the fixture commands now run under `env -i` with only the variables the run needs; with XDG_* unset the agents fall back to the empty HOME. The local-logs mode keeps the caller's environment on purpose. Reported by Cubic on #2061. * fix(perf): make bench-history safe to point at binaries and run twice Four ways `just bench-history` could measure the wrong thing or fail: - Binary-path variants were labelled by basename, so two builds at `a/ccusage` and `b/ccusage` collided in hyperfine's results. Label them by the argument as given, and reject a ref or path passed twice. - `--fixture-dir` alone silently reused the Claude logs as the Codex fixture. Require both directories or neither. - The cached binary was copied without attributes; copy them so the executable bit is kept on every platform. - Every run checked refs out into the shared `<cache>/src` worktree and removed it on exit, so a second run sharing the cache deleted the first run's checkout mid-build. Each run now uses its own temporary worktree; the shared cargo target is safe because cargo locks it. The first three were reported by Cubic on #2061; the fourth showed up while bisecting with two runs at once. * fix(perf): report memory-only improvements and history instruction ratios The overall verdict counted only CPU improvements, so a PR that cut peak RSS read "no significant change" above a table saying "less memory". Count CPU and RSS improvements alike and name what improved per command. The history tables also dropped the instruction-count ratio the PR comparison shows; add it as a column whenever hardware counters are available. Reported by Cubic on #2061. * test(perf): restore duplicate-name and time-unit hyperfine cases The hyperfine reader rewrite kept the missing-name and malformed-output cases but lost the duplicate-name and non-second-unit ones, so a change that accepted ambiguous or mis-scaled samples would pass. Restore both against the new per-run reader. Reported by Cubic on #2061. * fix(perf): print the generated fixture's size on disk With zstd-compressed Codex rollouts the generator's `Size` line showed the uncompressed bytes, about four times what it wrote. Print the size on disk from `Bun.write` and the uncompressed size separately. Reported by Cubic on #2061. * fix(ci): measure the PR merged onto its base commit The job built the PR side from `pull_request.head.sha` and the base from `pull_request.base.sha`. When a branch is behind its base, the two binaries differ by the base's newer commits as well as the PR's own change, and those commits show up as PR regressions or improvements. Build the checked-out merge commit (`github.sha`) instead. It is the PR applied on top of the base commit, so the diff measured is exactly the PR. A step fails the job if the base is not an ancestor of the built commit, so the report's "merged onto Base:" line always names what was actually built. BASE_SHA/HEAD_SHA fall back to `merge_group.base_sha` and `github.sha`, and the PR-comment step is limited to pull_request events, so the job also works if a merge_group trigger is added. Reported by CodeRabbit on #2061. * fix(perf): cache built bench-history binaries atomically `build-commit!` copied the release binary straight to the cache path that later runs trust when it exists, so an interrupted copy left a truncated binary that every later run reused. Copy to a `.partial` sibling and rename it into place with an atomic move. Reported by CodeRabbit on #2061. * fix(perf): start fixture generation from an empty directory A cached fixture without its `.complete` marker was regenerated into the same directory, so files from an interrupted run could remain next to the new output. Delete the directory first, so the marker always describes exactly what the generator wrote. Reported by CodeRabbit on #2061. * fix(perf): order Codex usage records like real rollouts and add compactions The generator wrote each response's `token_count` before its `token_usage_record`. Real rollouts write the record first, and the parser settles a pending record only when a later `token_count` advances the cumulative total, so every synthetic record stayed in the compaction pending map until the end of its file. That inflated peak RSS in a way real logs never do: on the corrected fixture `codex daily` peaks at 56 MiB on main instead of 69 MiB. Write the record before the `token_count`, as Codex does, and add the remote compactions real rollouts contain: about one turn in 30 ends with a usage record whose usage is not in the cumulative total, followed by a `compacted` marker naming it and carrying a large replacement history. In real logs every marker names an earlier record. Reported by Cubic on #2061. * fix(ci): gate the perf job on preflight and keep its token read-only The old perf job depended on the pkg.pr.new publish job and so, through it, on `preflight` (secret scan and workflow lint); building from source dropped that dependency, letting a PR that fails the scan run the benchmark anyway. Depend on `preflight` directly. The job also ran PR-built binaries, the fixture generator and the harness with `pull-requests: write` and `issues: write`. Split it: `ccusage-perf` builds and measures with a read-only token and uploads the report, and `ccusage-perf-comment` downloads it and only runs the comment upsert with the write token. Reported by Cubic on #2061. * fix(perf): lock bench-history builds and fixture generation Runs sharing the cache could interleave writes: one build could replace `target/release/ccusage` in the shared cargo target before another run copied it into its commit's cache slot, and two runs could delete and regenerate the same fixture under each other. Build-and-copy now runs under a cache-wide file lock, and each fixture key under its own, both re-checking the cache once they hold the lock. A per-commit target dir would avoid the lock but rebuild every dependency for every ref. The lock is a FileChannel lock released by closing the channel, and a test runs three processes against it and checks their critical sections never overlap. Reported by Cubic on #2061. * fix(ci): drop the perf job's unreachable merge_group fallbacks The perf job only runs on `pull_request`, and #2062, which adds the `merge_group` trigger, keeps the perf comment PR-only, so the `merge_group.base_sha` and `github.sha` fallbacks and the comment promising merge_group support could never apply. Remove them. Reported by Cubic on #2061. * fix(perf): give bench-history a Codex fixture every compared ref reads The generated Codex fixture is mostly zstd-compressed rollouts, some in `archived_sessions/`. Releases before #1890 skip `.jsonl.zst` files (and before #1230 skip archived sessions), so a history run across that change compared an old ref reading a fraction of the input with a new ref reading all of it, and reported the difference as a CPU and RSS regression. The generator gains `--codex-layout legacy`, which writes the same rollouts uncompressed under `sessions/`. bench-history keeps the current layout when every ref is a commit containing #1890, and otherwise generates the legacy layout and says so at the top of the report; binary paths count as unknown. With a custom Codex fixture that holds `.zst` files and a ref that cannot read them, and for `--real` runs across the change, the report leads with a warning instead. The PR comparison always builds refs that read the current layout, so CI keeps compressed coverage. Reported by CodeRabbit on #2061. * fix(perf): check every Codex input class a compared ref may skip The compatibility check only looked for `.jsonl.zst` files, but releases before #1230 also skip `archived_sessions/`, so a custom fixture with archived plain rollouts got no warning and a history run across #1230 compared different input. Replace the single zstd commit with a table of Codex input classes taken from the Codex loader's history (`paths.rs`, `reader.rs`): archived sessions (a9b12c4, #1230, v20.0.7) and zstd-compressed rollouts (44a3aa8, #1890, v20.0.33). Each ref is checked per class by git ancestry; the generated fixture switches to the legacy layout when any ref skips any class, and a custom fixture or `--real` run warns per class with the refs that skip it. Claude discovery has scanned `projects/` recursively throughout v20 and the date-window skips (#1665, #1794) do not apply to unbounded reports, so they are not classes. Probing each ref's source for the reader code was rejected: the same strings appear in tests and comments. Reported by Cubic on #2061. * fix(ci): run the perf comment upsert from the base commit The comment job holds `pull-requests: write` and `issues: write`, but it checked out the PR merge commit and ran that tree's setup action and `upsert-pr-comment.nu`, so a PR that edited either ran them with the write token. Check out `pull_request.base.sha` instead, still without persisted credentials. The PR's report reaches the job only as an artifact the script reads as text. This PR does not change `upsert-pr-comment.nu`, and the base version reads the same inputs (COMMENT_FILE as raw markdown, COMMENT_MARKER, PR_NUMBER, GITHUB_REPOSITORY, GITHUB_TOKEN), so the first runs after merge need no fallback. The workflow file itself still comes from the PR on `pull_request` events, so this removes the PR's scripts from the write job rather than making the job a hard trust boundary. Reported by CodeRabbit on #2061. * fix(perf): match Codex input classes to the rollouts the loader reads The input-class matchers looked at absolute paths: any `.zst` file counted as a compressed rollout, and any path containing `/archived_sessions/`, including a parent of the fixture, counted as an archived session. Match paths relative to the Codex home against what the loader reads: `*.jsonl` and `*.jsonl.zst` (reader.rs) under `sessions/` and `archived_sessions/` (paths.rs). Tests cover non-rollout `.zst` files, rollouts outside those directories, non-rollout files in `archived_sessions/`, and a fixture under a directory named `archived_sessions`. Reported by Cubic on #2061.
Loads Codex JSONL files from archived_sessions next to each CODEX_HOME sessions directory so archived Codex usage remains included in reports.
The direct JSONL directory fallback is preserved for saved codex exec --json outputs, and the Codex guide now documents the archived source.
Testing:
Summary by cubic
Include archived Codex sessions in usage reports by reading JSONL from
archived_sessionsalongsidesessions. Keeps the direct JSONL fallback and avoids duplicate reads whenCODEX_HOMEentries repeat.sessions/andarchived_sessions/independently for eachCODEX_HOME.CODEX_HOMEentries.Written for commit b01fd43. Summary will update on new commits.
Summary by CodeRabbit
New Features
Documentation