Repository navigation
perf: speed up CI and local builds - #1984
Conversation
The unified report also scans opencode under XDG_DATA_HOME, so a developer's real opencode database leaked into the snapshot. Point it at an empty fixture directory like the other home directories.
Every Nix CI job unpacked the whole 2.7 GB BerriAI/litellm repository (7-25s per job), and pricing bumps re-lock it every few commits. The input is now a single-file URL pinned to a commit; update-litellm-lock.nu rewrites the commit in that URL before re-locking, and non-Nix cargo builds read the locked URL directly.
…y layer The test derivation ran cargo test in the release profile on top of the release layers, which hold neither dev-dependencies nor the test-snapshots feature variants, so ~50s of dependencies recompiled on every run; hawk compiled the whole workspace uncached (~25s). A dev-profile deps-only layer now serves tests and clippy, hawk gets its own layer, and main's test job roots both on the sticky disk.
buildDepsOnly runs cargo check before cargo build, but only the host layer's check artifacts are used (by clippy and hawk). The static Linux and darwin-x64 layers only need the build output, which saves ~30s whenever dependencies change.
preflight (~45s) sat ahead of the slowest build on the critical path. ci-gate still requires it, and the pkg.pr.new publish now waits for it so a leaked secret is never published.
The rustup install (44s) and Cargo cache extraction (36s) were dominated by Defender and NTFS small-file costs. CARGO_HOME, RUSTUP_HOME and the target directory now live on a ReFS Dev Drive. The cache paths change, so the first run after merge starts cold.
lld links the test binaries an edit relinks faster than the Nix cctools ld64 (about 7-10% on leaf and mid-depth rebuilds). sccache shares compiled dependencies, C build scripts included, across worktrees: a fresh worktree's cargo test --no-run went from ~60s to ~45s wall (CPU 315s to 200s) with a warm cache. Set RUSTC_WRAPPER, even empty, to opt out.
The build command already compiles the test binaries; crane's check phase only added another cargo test --no-run for the ccusage bin.
setup-dev-drive needs the Hyper-V VHD cmdlets, which the release workflow's windows-11-arm runner is not known to provide, so arm64 falls back to the default Cargo and target locations.
|
Caution Review failedThe pull request is closed. Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ccusage-guide | 23c2fc2 | Commit Preview URL Branch Preview URL |
Oct 27 2026, 07:46 PM |
…Drive Blacksmith's Windows image lacks the Hyper-V cmdlets setup-dev-drive needs (New-VHD), so the Dev Drive step failed. Defender path exclusions for the Cargo home, rustup home and workspace target the same small-file scan cost without Hyper-V, and the step is a no-op where Defender is absent.
ccusage
@ccusage/ccusage-darwin-arm64
@ccusage/ccusage-darwin-x64
@ccusage/ccusage-linux-arm64
@ccusage/ccusage-linux-x64
@ccusage/ccusage-win32-x64
commit: |
ccusage performance comparisonPR SHA: Performance comparison skipped. Base package URL was not ready before 300.000s. Fixture performance comparison requires a base package when --base-dir is not provided. Base package: |
ccusage performance comparisonPR SHA: Performance comparison skipped. Base package URL was not ready before 300.000s. Fixture performance comparison requires a base package when --base-dir is not provided. Base package: |
Pricing snapshot commits are pushed to main with GITHUB_TOKEN, so they trigger no CI and never get a pkg.pr.new package. A PR based on one made both perf comment jobs wait out the full 5-minute package timeout and then skip the base comparison. Walk back over pricing-only commits to the nearest one with a package; any other commit without one keeps the old wait.
Blacksmith's Windows image has no Defender (Add-MpPreference fails with 'Invalid class'), so the step only cost ~7s; the slow cache extraction is not a Defender scan cost there.
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
The perf baseline walk-back misses one pricing-owned path, so half the pricing commits still hit the 5-minute wait this PR sets out to fix.
Reviewed changes
- Slimmed the
litellmflake input to a single-file URL and updatedbuild.rs, dev-shell, packaging, docs, and the lock-update script to match. - Added a dev-profile
nix/test-artifacts.nixdependency layer shared by tests, clippy, and hawk, and skipped crane's redundantcheckpass in cross/static dep layers. - Unblocked
testandbuild-native-packagesfrompreflightwhile keepingpreflightrequired inci-gateand the pkg.pr.new publish job. - Added a perf-comment step that walks back over pricing-only commits to the nearest packaged base SHA.
- Enabled lld on macOS and sccache sharing in the dev shell, and isolated the copilot CLI snapshot from host
XDG_DATA_HOME.
muse-spark-1.3-contributor (free via Pullfrog for OSS) ｜ 𝕏
| break | ||
| fi | ||
| if git diff --name-only "$sha^" "$sha" \ | ||
| | grep -qvE '^(flake\.(lock|nix)|rust/crates/ccusage-pricing-snapshots/src/[^/]+\.json)$'; then |
There was a problem hiding this comment.
The pricing-only allowlist omits rust/adapters/codex/src/codex-auto-review-fallbacks.json, which the models.dev leg also commits. A base commit touching only that file plus the allowlisted paths stops the walk and keeps an unpackaged base.
| | grep -qvE '^(flake\.(lock|nix)|rust/crates/ccusage-pricing-snapshots/src/[^/]+\.json)$'; then | |
| | grep -qvE '^(flake\.(lock|nix)|rust/crates/ccusage-pricing-snapshots/src/[^/]+\.json|rust/adapters/codex/src/codex-auto-review-fallbacks\.json)$'; then |
| | first | ||
| ) | ||
| open --raw flake.nix | ||
| | str replace --regex 'BerriAI/litellm/[0-9a-f]{40}/' $"BerriAI/litellm/($rev)/" |
There was a problem hiding this comment.
If git ls-remote fails or prints nothing, $rev is empty and the rewrite corrupts flake.nix before nix flake update runs. Validate a 40-hex rev first.
Upstream tip: 30b653f (2026-11-06, chore(pricing): update models.dev snapshot) Previous fork base: c76874e (the upstream commit this fork was synced to) Synced 128 upstream commits on top of c76874e: 70 automated pricing snapshot refreshes and the following meaningful ones: b24e7a7 docs: remove LINE MAN sponsor logo (ccusage#1910) a93d2fe chore: release v20.0.38 (ccusage#1938) d91f296 docs: add ccx to community projects (ccusage#1943) aa7c675 chore: release v20.0.39 (ccusage#1944) 39fefa2 fix(cli): accept --start-of-week on top-level weekly (ccusage#1945) 600b3c9 chore: release v20.0.40 (ccusage#1946) baeb991 fix(antigravity): map model ID 1402 to claude-opus-5-5 (ccusage#1948) 06347d3 chore: release v20.0.41 (ccusage#1949) 5649632 test(hermes): derive DeepSeek provider pricing from embedded snapshot (ccusage#1951) d0847af chore(deps): update node.js to >=24.22.1 (ccusage#1950) 91dfe85 chore(deps): update pullfrog/pullfrog action to v0.1.100 (ccusage#1953) f335280 chore: release v20.0.42 (ccusage#1952) b86527d chore(deps): update rust crate rustls-webpki to 0.103.16 (ccusage#1954) 5c85ca2 chore(deps): update pullfrog/pullfrog action to v0.1.101 (ccusage#1958) f268798 fix(cli): apply source-specific config in unified reports (ccusage#1959) 37a51da chore: release v20.0.43 (ccusage#1955) e302366 chore(deps): update github/codeql-action action to v4.39.1 (ccusage#1960) 7af0db6 chore(deps): update pnpm to v12.15.0 (ccusage#1961) 730c336 chore(deps): update rust crate ureq to 3.5.0 (ccusage#1964) bd1a21a fix(release): publish platform packages before the wrapper (ccusage#1965) ca33842 chore: release v20.0.44 (ccusage#1962) b2c8f8b chore: remove unused npm registry gate test (ccusage#1966) 19e71d0 chore(deps): update songmu/tagpr action to v1.21.4 (ccusage#1969) 2a84385 fix(codex): count repeated first-response snapshots once (ccusage#1971) 60227f2 chore(deps): update rust crate smallvec to 1.16.4 (ccusage#1973) 3e31edd chore(deps): update hustcer/setup-nu action to v3.29 (ccusage#1974) d93c260 feat(pricing): add gpt-6.1-terra fast/flex multipliers (ccusage#1975) 154e13b chore: release v20.0.45 (ccusage#1967) f1b50e6 chore(deps): update pullfrog/pullfrog action to v0.1.102 (ccusage#1980) 4f1f470 perf: speed up CI and local builds (ccusage#1984) 0bd3d19 refactor(rust): move binary-only report helpers out of ccusage-core (ccusage#1985) 65c5837 fix(antigravity): skip unreadable conversation databases (ccusage#1992) 6372cac chore(deps): update dependency vitepress to v2.0.0-alpha.21 (ccusage#1994) 9a42066 chore(deps): update node.js to >=24.22.2 (ccusage#1995) 87fa788 chore(deps): update pnpm to v12.16.0 (ccusage#1996) 8cb2532 fix(codex): price recorded Ultrafast requests at Ultrafast rates (ccusage#1989) 128d6a0 chore(deps): update pnpm to v12.16.1 (ccusage#1997) 5aa9d3f chore: release v20.0.46 (ccusage#1982) 4712880 chore(deps): update rust crate unicode-width to 0.2.3 (ccusage#1998) 3d59863 chore(deps): update node.js to v26 (ccusage#1999) 96e98c6 chore(deps): update dependency @types/node to v26.6.6 (ccusage#2002) 60fac54 chore(deps): update pullfrog/pullfrog action to v0.1.103 (ccusage#2003) de83d19 fix(config): let space-form command sections override generic report sections (ccusage#2006) 72b8ad4 fix(all): count OpenCode Claude Agent turns once in unified reports (ccusage#2007) 7db7a30 test(pricing): stop the alias-precedence test depending on the LiteLLM snapshot (ccusage#2009) e7a7ee4 fix(all): keep sessions last active on the --until day (ccusage#2008) 4a3da73 chore: release v20.0.47 (ccusage#2001) 8e56c11 chore(deps): update pullfrog/pullfrog action to v0.1.104 (ccusage#2010) e2eddf7 chore(deps): update pullfrog/pullfrog action to v0.1.107 (ccusage#2012) e3d2dcb chore(deps): update pullfrog/pullfrog action to v0.1.108 (ccusage#2014) 8995f3b chore: release v20.0.48 (ccusage#2011) 3977fd0 chore(deps): update pullfrog/pullfrog action to v0.1.110 (ccusage#2017) 546ae51 fix(nix): make `nix flake show` work again on nixpkgs 26.11 (ccusage#2019) 1b5e8cc chore(deps): update kesin11/actions-timeline action to v3.2.1 (ccusage#1983) 23c5d62 chore(deps): update rust crate jiff to 0.2.39 (ccusage#2024) 7b7cec0 chore(deps): update rust crate ureq to 3.5.1 (ccusage#2025) a1e4a03 chore(deps): update pullfrog/pullfrog action to v0.1.111 (ccusage#2028) e84d142 chore(deps): update pnpm to v12.17.0 (ccusage#2029) Fork wins kept: the SQLite on-disk cache with its per-namespace ledger, --live-only and clear-cache, the pi adapter with replay suppression and per-path namespaces, the OpenCode V2 loader (now wrapped as LinkedEntry), the FxHash/Cow hot paths, parallel per-file loads with their single_thread seam, the table renderer, the fork CI and tag-only release workflow, and the pricing freshness, refresh and offline precompute paths. Hand-resolved conflicts: rust/adapters/claude/src/lib.rs: upstream's DedupedEntries/pruned dedupe lands beside the fork's collapse_replays and requestless replay rules; load_daily_summaries routes through load_daily_summaries_with_detection. rust/adapters/opencode/{parser,loader}.rs: upstream's V2 session_message parsing is wrapped as the fork's LinkedEntry, and the native Claude session link is read from the payload so it survives a cache hit. rust/adapters/antigravity/src/loader.rs: upstream's rewrite is taken, keeping the fork's per-database outcome reporting and file order. rust/crates/ccusage-core/src/pricing.rs: the fork's OnceLock embedded loader sits beside upstream's models.dev fallback and alias resolution. rust/crates/ccusage/src/main.rs: upstream dispatch plus the fork's ClearCache command. rust/crates/ccusage-core/src/cache.rs: the ledger is now generic over a LedgerPayload trait, so it covers plain entries and OpenCode's entry-plus-link pairs alike. rust/Cargo.toml: the fork's exact `=` pins stay; jiff 0.2.39, smallvec 1.16.4, ureq 3.5.1, unicode-width 0.2.3, rustls-webpki 0.103.16. .github/workflows/{ci,release}.yaml: the fork's design is kept. packages/ccusage-*/package.json (6 files) and rust/adapters/claude/src/daily.rs: stay deleted. rust/crates/ccusage-core/src/{home,last_window}.rs: upstream's crate split moved them to ccusage-cli and ccusage-report, with their snapshots. apps/ccusage/config-schema.json: regenerated from the merged Rust source (32 numeric minimums `0.0` -> `0`, new pi description), plus three snapshots regenerated in the fork layout for the codex and qwen tests. Fixes made while landing this sync, folded in so the merge stands green: fix(rust): decode cache blobs written before `data_session_id`. postcard encodes struct fields positionally, so a field added mid-struct made every blob the released 20.0.36 binary wrote undecodable. A failed decode is not merely slow: it drops the ledger rows of deleted source files, which no later run can rebuild. In an isolated fixture the released binary still reported 101,570,002 pi tokens after the source files were deleted; a warm build without the decoders reported 0. LegacyCachedEntry and LegacyLedgerEntry decode the old layout and reconstruct `data_session_id` as None; decode_exact requires the whole buffer, so a newer blob can no longer "decode" into an older struct as garbage. Four regression tests cover both layouts and the tail rule. fix(rust): warn once per table when a cached blob decodes as no known layout. The files table is rebuildable; the ledger table is not, so the warning says to save a copy before clear-cache. test(opencode): isolate the Claude-agent pairing test from the developer's real cache; it had been writing 'agent'/'plain' ledger rows into the opencode-db namespace. Version: take upstream's 20.0.48 in package.json, apps/ccusage/package.json and docs/package.json; the Rust workspace keeps version 0.0.0. DO NOT push a v20.0.48 tag from this fork: release.yaml would publish fork bytes under upstream identity.
Summary
Speeds up CI and local Rust builds. Baseline CI critical path:
preflight(~45s) →build-windows-x64(~200–250s: rustup install 44s, cache extraction 36s, cargo 90s) → publish → e2e.CI
litellmflake input was the whole 2.7 GB repository, unpacked on every Nix job (7–25s each). It is now a single-file URL pinned to a commit;update-litellm-lock.nurewrites the commit and re-locks, and non-Nix cargo builds readlocked.url.ccusage-testsran in the release profile and recompiled dev-dependencies andtest-snapshotsvariants every run (~50s); hawk compiled the workspace uncached (~25s). Newnix/test-artifacts.nix; main's test job roots both layers on the sticky disk.cargo checkpass in the static Linux and darwin-x64 deps layers (~30s whenever deps change).ci-gatestill requires it, and the pkg.pr.new publish now depends on it so a leaked secret is never published.GITHUB_TOKEN, trigger no CI and never get a pkg.pr.new package, so a PR based on one made both perf jobs wait out the full 5-minute timeout (seen on this PR's first green run: 332s vs ~45s). The job now walks back over pricing-only commits to the nearest packaged one.Local (dev shell)
cargo test --workspace --no-run~60s → ~45s wall (CPU 315s → 200s). No change for incremental rebuilds. SetRUSTC_WRAPPER(even empty) to opt out.XDG_DATA_HOME.Risks / to verify on this PR's CI
New-VHD) nor Defender, so neither a Dev Drive nor Defender exclusions apply there.debug_assertions.Test plan
nix fmt,nix flake check --no-buildnix build .#ccusage-testsplus clippy and hawk checks,nix build .#ccusageon aarch64-darwincargo test --workspacein the dev shell (1081 passed, with lld + sccache)fetch-litellm-pricingcargo build downloads from the locked URLccusage-testsbuild compiles only the 28 workspace crates