Skip to content

ci: shorten the Windows native build critical path - #1924

Merged
ryoppippi merged 3 commits into
ci/fix-macos-nix-cachefrom
ci/speed-up-windows-build
Oct 18, 2026
Merged

ryoppippi merged 3 commits into
ci/fix-macos-nix-cachefrom
ci/speed-up-windows-build

Conversation

@ryoppippi

@ryoppippi ryoppippi commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Stack created with GitHub Stacks CLI • Give Feedback 💬

Summary

Shortens the build-windows-x64 / build-windows-arm64 legs in ci.yaml and release.yaml without changing what rustc produces: same toolchain version, profile, target, flags and command.

What Changed

  • Minimal toolchain on Windows: the Windows action reads the channel from rust-toolchain.toml and runs rustup toolchain install <channel> --profile minimal. It pins the build to that channel with RUSTUP_TOOLCHAIN, which rustup prefers over the toolchain file, so cargo no longer installs clippy, rustfmt and llvm-tools. Only rustc, cargo and rust-std (3 of the 6 components) are installed now.
  • Cargo cache keyed on dependencies, saved only from main:
    • The key is a hash of rust-toolchain.toml, .cargo/config.toml, rust/Cargo.lock and rust/**/Cargo.toml instead of github.sha.
    • Restore and save are separate actions/cache steps (same pinned SHA). The save runs only on refs/heads/main, never for pull_request or pull_request_target events, and only when that key has no entry yet.
    • Pull requests only restore what main wrote.
    • ~/.cargo/git is dropped from the cached paths (there are no git dependencies).
  • Release: the x64 leg only restores the cache CI saves. The arm64 leg, which CI never builds, still saves its own cache, but only when the dependency key changes.
  • Pruning: before a save, the workspace crate outputs (ccusage*, deps/*ccusage*, .fingerprint/ccusage*, build/ccusage*) are deleted from rust/target/release.
  • The single-use setup-native-build-cache action is folded into build-windows-native-package.

Why

From recent run logs, timed from when the runner picks up the job:

  • Restoring the 126-134 MB cache takes 27-38 s, mostly tar extraction (the download is 3-4 s).
  • rustup spends 24-29 s downloading 6 components.
  • cargo takes about 48 s and recompiles only the workspace crates, because checkout gives their sources fresh mtimes. Dependencies are always fresh.
  • Every run, pull requests included, saved a new SHA-keyed entry in a 5-10 s post-job step. That save sat before the npm job in CI and in release.

Dropping clippy, rustfmt and llvm-tools skips about 50 of about 150 MB of xz archives on x86_64 Windows. Unpacked on macOS, the toolchain shrinks from 980 MB to 552 MB. Workspace outputs are about a quarter of a release target dir and are rebuilt every run, so caching them only slows restores.

Testing

  • treefmt --ci (actionlint and zizmor with the repo config) is clean. zizmor reports no findings for the new Windows action.
  • Locally with rustup 1.29.1: with RUSTUP_TOOLCHAIN set, cargo installs nothing beyond the minimal profile. Without it, cargo downloads the 3 extra components from the toolchain file.
  • After the prune command, cargo check rebuilt only the ccusage-* crates; dependencies stayed fresh.
  • In this PR's run you can check: 'downloading 3 components' on Windows and no cache save.
  • Expect a cold Windows build here and on the first main push after merge: dropping ~/.cargo/git from the cached paths changes the actions/cache version hash, so no existing cargo-Windows-* entry can be restored, even through the restore-key. That is a one-off cost, not a regression. The main-only save, the pruned entry and exact-key hits only appear after merge, on main pushes and the next release.

rust-toolchain.toml lists clippy, rustfmt, and llvm-tools for local
development, so the first cargo call on the Windows runners downloaded six
components. Install the same channel with the minimal profile instead and pin
it through RUSTUP_TOOLCHAIN, which rustup prefers over the toolchain file. The
compiler, standard library, and target are unchanged; only the ~50 MB of xz
archives (~430 MB unpacked) the build never uses are skipped.
Every Windows run, pull requests included, saved a fresh ~130 MB entry keyed
on the commit, and the 5-10 s post-job save sat on the critical path before
the npm job. Workspace crates recompile on every run regardless, because
checkout gives their sources fresh mtimes, so only the dependency build is
worth caching.

Key the cache on the toolchain, Cargo config, lockfile, and manifests, and
save it explicitly only on main outside pull requests when that key has no
entry yet. Pull requests now restore what main wrote and never save. The
release x64 leg restores what CI saves; arm64, which CI does not build,
remains the writer of its own cache. The single-use setup-native-build-cache
action is folded into the Windows action so restore and save share one key.
Workspace crates rebuild on every run, so their rlibs, fingerprints, build
script outputs, and the final binary in rust/target/release are dead weight
that every restore has to unpack. In a local release target they are about a
quarter of its size. Remove them right before the save; dependency units keep
their fingerprints and stay fresh.
@ryoppippi
ryoppippi added this pull request to stack #1926 October 18, 2026 15:25
@coderabbitai

coderabbitai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fc31d58a-5d9f-41e7-b42e-ae7c75fbaf98
📥 Commits

Reviewing files that changed from the base of the PR and between ad91367 and 69ac221.

📒 Files selected for processing: 3
  • .github/actions/build-windows-native-package/action.yaml
  • .github/actions/setup-native-build-cache/action.yaml
  • .github/workflows/release.yaml
+-----------------------------+
| Don't hate the know-it-all. |
+-----------------------------+
  \
   \     /)/)
        ( '')
        /  |
      o(___)
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ryoppippi ryoppippi changed the title ci/speed up windows build ci: shorten the Windows native build critical path Oct 18, 2026
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ccusage-guide 69ac221 Commit Preview URL

Branch Preview URL
Oct 18 2026, 03:26 PM

@pkg-pr-new

pkg-pr-new Bot commented Oct 18, 2026

Copy link
Copy Markdown

Open in StackBlitz

ccusage

npx https://pkg.pr.new/ccusage@1924

@ccusage/ccusage-darwin-arm64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-arm64@1924

@ccusage/ccusage-darwin-x64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-x64@1924

@ccusage/ccusage-linux-arm64

npx https://pkg.pr.new/@ccusage/ccusage-linux-arm64@1924

@ccusage/ccusage-linux-x64

npx https://pkg.pr.new/@ccusage/ccusage-linux-x64@1924

@ccusage/ccusage-win32-x64

npx https://pkg.pr.new/@ccusage/ccusage-win32-x64@1924

commit: 69ac221

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 69ac2214b158
Base SHA: ad91367a663b

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 371.1ms 2.71 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 341.4ms 2.95 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 145.1ms 6.94 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 117.7ms 8.55 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 64.2ms 20.2ms 3.18x 55.50 MiB 22.97 MiB 0.41x 0.02 MiB/s 0.08 MiB/s
claude session --offline --json 0.00 MiB 57.7ms 12.6ms 4.58x 55.75 MiB 22.97 MiB 0.41x 0.03 MiB/s 0.12 MiB/s
codex daily --offline --json 0.00 MiB 43.1ms 12.4ms 3.48x 55.50 MiB 29.23 MiB 0.53x 0.02 MiB/s 0.07 MiB/s
codex session --offline --json 0.00 MiB 36.1ms 12.3ms 2.93x 55.50 MiB 29.22 MiB 0.53x 0.02 MiB/s 0.07 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 381.3ms 359.1ms 1.06x 939.10 MiB 943.11 MiB 1.00x 2.64 GiB/s 2.80 GiB/s
codex --offline --json 1.01 GiB 149.9ms 119.6ms 1.25x 527.45 MiB 511.44 MiB 0.97x 6.72 GiB/s 8.42 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 24.03 KiB 24.03 KiB +0.00 KiB 1.00x
installed native package binary 4907.09 KiB 4907.09 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 69ac2214b158
Base SHA: ad91367a663b

This compares the PR package against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 381.6ms 2.64 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 336.1ms 3.00 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 137.6ms 7.32 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 116.7ms 8.63 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 41.9ms 38.2ms 1.10x 55.75 MiB 55.25 MiB 0.99x 0.04 MiB/s 0.04 MiB/s
claude session --offline --json 0.00 MiB 36.4ms 35.7ms 1.02x 55.75 MiB 55.75 MiB 1.00x 0.04 MiB/s 0.04 MiB/s
codex daily --offline --json 0.00 MiB 37.6ms 36.6ms 1.03x 55.50 MiB 55.50 MiB 1.00x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 35.7ms 39.9ms 0.90x 55.25 MiB 55.50 MiB 1.00x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 381.7ms 376.2ms 1.01x 947.11 MiB 943.10 MiB 1.00x 2.64 GiB/s 2.68 GiB/s
codex --offline --json 1.01 GiB 147.8ms 142.0ms 1.04x 515.42 MiB 513.44 MiB 1.00x 6.81 GiB/s 7.09 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 24.03 KiB 24.03 KiB +0.00 KiB 1.00x
installed native package binary 4907.09 KiB 4907.09 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@ryoppippi
ryoppippi marked this pull request as ready for review October 18, 2026 17:03
@ryoppippi
ryoppippi merged commit 7a063a0 into main Oct 18, 2026
45 of 50 checks passed
@ryoppippi
ryoppippi deleted the ci/speed-up-windows-build branch October 18, 2026 17:03

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues found — CI-only optimization, verified end to end. Not approving per automation policy, not per concern.

Reviewed changes

  • Minimal Rust toolchain on Windows: installs the rust-toolchain.toml channel with --profile minimal and pins it via RUSTUP_TOOLCHAIN, skipping clippy, rustfmt, and llvm-tools with no compiler change.
  • Dependency-keyed Cargo cache with main-only saves: separate restore/save steps keyed on the toolchain, Cargo config, lockfile, and manifests; pull requests restore only, saves happen on refs/heads/main outside PR events on exact-key miss.
  • Pruned workspace outputs before save and folded the single-use cache action into build-windows-native-package with a save-cargo-cache input; release.yaml restores CI's cache on x64 and lets the arm64 leg own its key.

Correctness checks performed: no git = dependencies (dropping ~/.cargo/git is safe), every workspace crate name starts with ccusage so the prune globs hold, no remaining references to the deleted setup-native-build-cache action, and the x64 restore key matches between CI and release runners (Windows/X64). The one-off cold cache after merge is acknowledged in the PR description. Save gating via the prune step outcome correctly suppresses PR saves.

Pullfrog  | Fix it ➔ | View workflow run | Using muse-spark-1.3-contributor (free via Pullfrog for OSS) | 𝕏

Buster210 added a commit to Buster210/ccusage that referenced this pull request Oct 20, 2026
Upstream tip: c76874e (2026-10-20, chore(pricing): update models.dev snapshot)
Previous fork base: 0c58b88 (the upstream commit this fork was synced to)

Synced 83 upstream commits on top of 0c58b88: 56 automated pricing
snapshot refreshes and the following meaningful ones:

  1cd29f5 chore(deps): update pnpm to v12.13.0 (ccusage#1907)
  e4f63ba chore(deps): update node.js to >=24.22.0 (ccusage#1908)
  130e502 ci: regenerate bun.nix automatically (ccusage#1909)
  259ec33 feat(adapter): report stored cost in JSON model breakdowns (ccusage#1903)
  c1eb448 chore: release v20.0.35 (ccusage#1901)
  2a0875f ci(pricing): update snapshots on push to main and daily instead of hourly (ccusage#1914)
  b090428 chore(deps): update github/codeql-action action to v4.39.0 (ccusage#1912)
  4dcf7f5 fix(nix): pin models-dev-gen remeda to models.dev's version and stop Renovate drift (ccusage#1915)
  46edb28 fix(nix): stop the dev shell from copying the whole repository (ccusage#1917)
  58ec8f1 fix(nix): let commits succeed in worktrees without a loaded dev shell (ccusage#1920)
  a570a6f fix(nix): vendor Cargo deps without import-from-derivation (ccusage#1918)
  f6d28c5 fix(antigravity): map model IDs 1418/1419 to gemini-4-argon (ccusage#1905)
  ce439e5 fix(antigravity): map model ID 1050 to gemini-3.1-flash-lite (ccusage#1902)
  e62a66f chore(deps): update pnpm to v12.14.0 (ccusage#1923)
  117ae68 ci(nix): make the macOS Nix cache save a trimmed store with the Crane layers (ccusage#1921)
  7a063a0 ci: shorten the Windows native build critical path (ccusage#1924)
  85ebca3 perf(rust): move pricing snapshots into a leaf crate (ccusage#1922)
  8fe3f47 perf(nix): build Crane layers with the binary's feature resolution (ccusage#1925)
  ac289aa perf(pricing): give the pricing refresh a verify-only TLS 1.3 rustls provider (ccusage#1927)
  d097f74 chore(deps): update rust crate rustls to 0.23.46 (ccusage#1928)
  f8aaf89 docs: use the compact pkg.pr.new URL for bunx previews [ci skip] (ccusage#1929)
  f56bb3d chore: release v20.0.36 (ccusage#1916)
  aaafb41 chore(deps): update dependency vitepress-plugin-group-icons to ^1.8.1 (ccusage#1931)
  0861e7f fix(claude): count requestless resp_ content blocks once (ccusage#1935)
  867c9f3 chore: release v20.0.37 (ccusage#1933)
  fa40bfd chore(deps): update songmu/tagpr action to v1.21.3 (ccusage#1937)
  de99d15 chore(deps): update rust crate zstd to 0.14.3 (ccusage#1936)

Fork wins kept: the SQLite on-disk cache with --live-only and the per-namespace
ledger, the pi adapter and its per-path namespaces, the OpenCode V2 loader, the
FxHash/Cow hot paths, the table renderer, clear-cache, and the tag-only release
workflow. Upstream's own release and tagpr machinery stays out.

Hand-resolved conflicts:

  rust/adapters/claude/src/lib.rs: upstream's resp_ message-id dedupe (ccusage#1935) lands
    beside the fork's requestless replay and collapse_replays rules.
  rust/adapters/claude/src/README.md: upstream's wording for the msg_/resp_ exception.
  rust/Cargo.toml: zstd 0.14.3, keeping the fork's exact `=` pin style.
  .github/workflows/release.yaml: the fork's tag-only release, without upstream's
    tagpr job or the songmu/tagpr action bump.
  apps/ccusage/package.json, docs/package.json, package.json: keep 20.0.34;
    upstream's 20.0.37 bump is dropped.
  packages/ccusage-*/package.json (6 files): stay deleted; the fork ships no npm
    wrapper packages and the release workflow does not read them.
  rust/adapters/claude/src/daily.rs: stays deleted; the fork's daily path lives in
    lib.rs and shares the dedupe helper the fix changes.

Fixes made while landing this sync, folded in so the merge stands green on its own:

  fix(ci): save the Nix store from the native build composites. The nested
    cache-nix-action read an empty primary-key at post-job and every native
    build failed; upstream's defer-save pattern is the fix.
  fix(rust): keep the agent-recorded cost for deleted sources. The recorded_cost
    ledger column carries it, so storedCost survives a deleted file. The ledger
    schema version stays at 2: the column is not a LedgerEntry field, so the blob
    layout is unchanged and the previously released binary keeps working.
  fix(rust): make the file fingerprint helper private, the one hawk finding. It is
    inherited from the pre-sync master, not introduced here.

Keep 20.0.34 numbering; DO NOT push a v20.0.34 tag from this fork (release.yaml
would publish fork bytes under upstream identity).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant