Skip to content

fix(antigravity): deduplicate unkeyed mirror usage via step index fallback - #2027

Merged
ryoppippi merged 2 commits into
mainfrom
pullfrog/issue-2026-run-42117316179-attempt-1
Nov 9, 2026
Merged

ryoppippi merged 2 commits into
mainfrom
pullfrog/issue-2026-run-42117316179-attempt-1

Conversation

@github-actions

@github-actions github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

References #2026 triple-counting of unkeyed Gemini turns in rust/adapters/antigravity.

  • Parses CortexStepGeneratorMetadata.step_indices (field 2, unpacked and packed) and uses step:{session}:{step_idx} as fallback identity when ModelUsageStats lacks response/provider/message IDs, so gen_metadata rows merge with their steps rows in deduplicate_events.
  • Drops single-entry retry_infos[0] mirror copies in both generator and step metadata when token counts match and identities are a subset of the primary, preventing intra-row 2x.
  • Falls back to gen:{session}:{gen_idx} when step_indices is empty, so unkeyed mirrors still collapse to 1x.

Tests:

  • cargo test --manifest-path rust/Cargo.toml -p ccusage -p ccusage-adapter-antigravity --features ccusage/fetch-litellm-pricing --lib — 39 passed, including new deduplicates_unkeyed_mirror_copies_via_step_index, drops_unkeyed_retry_mirror_without_step_link, parses_step_indices_and_drops_mirror_retry, and keeps_distinct_retry_attempt_for_deduplication.
  • cargo clippy ... --features ccusage/fetch-litellm-pricing -- -D warnings — clean.
  • cargo fmt --all -- --check — clean.

Closes #2026


Summary by cubic

Fixes triple-counting of unkeyed Gemini turns in rust/adapters/antigravity by giving generator rows a step-index fallback identity so mirrored usage rows collapse to 1x instead of 2–3x.

  • gen_metadata rows fall back to step:{session}:{step_idx} (parsed from step_indices), or gen:{session}:{gen_idx} when step indices are absent, so unkeyed usage merges with its step row.
  • Drops the retry_infos entry that is an exact mirror of the primary usage, and leaves remaining retries unlinked from the turn identity so distinct failed attempts are counted as their own usage instead of being folded into the primary by max-merge deduplication.

Tests added for step-index parsing, mirror dropping, and preserving distinct retry attempts; cargo test, clippy, and fmt all pass. Closes #2026.

Written for commit ba4be79. Summary will update on new commits.

View guided diff Turn on auto-fix

@coderabbitai

coderabbitai Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6bd7cba5-bc16-49bb-9b76-557b60aacd72

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ccusage-guide ba4be79 Commit Preview URL

Branch Preview URL
Nov 09 2026, 07:37 PM

…ck identity

The step/generator positional fallback identity identifies the primary
turn, but it was also attached to every unkeyed retry_infos entry. Because
deduplicate_events merges by taking the max of each token field, a real
failed attempt (e.g. same input, fewer output tokens) was folded into the
successful attempt and its usage silently disappeared.

retry_infos lists every attempt, including the one already reported as the
primary usage. So the only retry that duplicates the primary is that one
mirror entry, whatever the attempt count. Drop exactly that entry (not only
when there is a single retry, which still left 2x for multi-attempt turns),
and leave the remaining retries without the turn's positional identity so
they are counted as their own usage.
@pkg-pr-new

pkg-pr-new Bot commented Nov 9, 2026

Copy link
Copy Markdown

Open in StackBlitz

ccusage

npx https://pkg.pr.new/ccusage@2027

@ccusage/ccusage-darwin-arm64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-arm64@2027

@ccusage/ccusage-darwin-x64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-x64@2027

@ccusage/ccusage-linux-arm64

npx https://pkg.pr.new/@ccusage/ccusage-linux-arm64@2027

@ccusage/ccusage-linux-x64

npx https://pkg.pr.new/@ccusage/ccusage-linux-x64@2027

@ccusage/ccusage-win32-x64

npx https://pkg.pr.new/@ccusage/ccusage-win32-x64@2027

commit: ba4be79

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

ccusage performance comparison

PR SHA: ba4be7949013
Base SHA: 546ae51302c3

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 534.9ms 1.88 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 508.3ms 1.98 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 269.7ms 3.73 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 217.9ms 4.62 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 65.1ms 24.0ms 2.72x 56.25 MiB 22.96 MiB 0.41x 0.02 MiB/s 0.06 MiB/s
claude session --offline --json 0.00 MiB 62.1ms 25.4ms 2.44x 55.50 MiB 22.96 MiB 0.41x 0.02 MiB/s 0.06 MiB/s
codex daily --offline --json 0.00 MiB 71.1ms 27.6ms 2.57x 55.75 MiB 26.96 MiB 0.48x 0.01 MiB/s 0.03 MiB/s
codex session --offline --json 0.00 MiB 73.0ms 33.2ms 2.20x 55.50 MiB 26.96 MiB 0.49x 0.01 MiB/s 0.03 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 641.3ms 643.1ms 1.00x 889.32 MiB 917.07 MiB 1.03x 1.57 GiB/s 1.57 GiB/s
codex --offline --json 1.01 GiB 235.2ms 239.9ms 0.98x 467.42 MiB 475.43 MiB 1.02x 4.28 GiB/s 4.20 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 27.64 KiB 27.66 KiB +0.02 KiB 1.00x
installed native package binary 4988.78 KiB 4996.66 KiB +7.88 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

ccusage performance comparison

PR SHA: ba4be7949013
Base SHA: 546ae51302c3

This compares the PR package against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 436.5ms 2.31 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 427.6ms 2.35 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 207.0ms 4.86 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 182.3ms 5.52 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 54.6ms 51.6ms 1.06x 56.00 MiB 56.00 MiB 1.00x 0.03 MiB/s 0.03 MiB/s
claude session --offline --json 0.00 MiB 47.7ms 44.9ms 1.06x 55.75 MiB 56.00 MiB 1.00x 0.03 MiB/s 0.03 MiB/s
codex daily --offline --json 0.00 MiB 49.5ms 48.6ms 1.02x 56.25 MiB 56.25 MiB 1.00x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 46.8ms 53.8ms 0.87x 55.75 MiB 56.25 MiB 1.01x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 448.2ms 473.9ms 0.95x 921.09 MiB 907.09 MiB 0.98x 2.25 GiB/s 2.12 GiB/s
codex --offline --json 1.01 GiB 223.8ms 215.6ms 1.04x 471.43 MiB 481.43 MiB 1.02x 4.50 GiB/s 4.67 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 27.64 KiB 27.66 KiB +0.02 KiB 1.00x
installed native package binary 4988.78 KiB 4996.66 KiB +7.88 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@ryoppippi
ryoppippi merged commit 00fead0 into main Nov 9, 2026
45 checks passed
@ryoppippi
ryoppippi deleted the pullfrog/issue-2026-run-42117316179-attempt-1 branch November 9, 2026 19:46
@github-actions github-actions Bot mentioned this pull request Nov 9, 2026
Buster210 added a commit to Buster210/ccusage that referenced this pull request Nov 13, 2026
Upstream tip: 660e07c (2026-11-13,
chore(pricing): update models.dev snapshot)
Previous fork base: 30b653f (the upstream commit this fork was synced to)
Synced 100 upstream commits on top of 30b653f: 53 automated pricing
snapshot refreshes, 23 dependency bumps, 2 releases, and:
  3541115 feat(cline): add extension and CLI usage adapter (ccusage#1977)
  5e5c475 fix(antigravity): report real model names instead of placeholders (ccusage#2071)
  e39ed83 fix(claude): keep the cache TTL split when deduping equal copies (ccusage#2060)
  c17c591 feat(adapter): report 1h cache writes in JSON model breakdowns (ccusage#2059)
  a06f4bc perf(claude): halve `ccusage claude` report time (ccusage#2058)
  de43952 fix(claude): show model breakdowns for session --id (ccusage#2057)
  5f31039 fix(all): count only in-window Claude usage in unified session reports (ccusage#2052)
  e4f0ea9 fix(codex): count spawned subagent requests stamped at the fork instant (ccusage#2051)
  b85b28f fix(opencode): count usage from completed v2 compaction rows (ccusage#2050)
  00fead0 fix(antigravity): deduplicate unkeyed mirror usage (ccusage#2027)
  2248106 fix(opencode): exclude Claude Agent turns by default (ccusage#2022)
  4fb1c0d fix(codex): guard truncated type fields from panicking the parser (ccusage#2034)
  86707db build(perf): make the PR performance comparison catch real regressions (ccusage#2061)
  686a1a8 ci: support a merge queue on main (ccusage#2062)
Fork wins kept: the Codex optimized scanner with its fork-replay instant,
the Antigravity model resolver, the OpenCode retained-ledger cache with its
per-namespace rows, the SQLite on-disk cache, the table renderer, the fork
CI and tag-only release workflow, and the exact `=` dependency pins.
Hand-resolved conflicts:
  rust/adapters/claude/src/lib.rs: the fork's DedupedEntries copy pairs stay
    and take upstream's carries_more_usage_detail tiebreak (speed rank, then
    a cache TTL split); main_thread_session_days reads the recorded session
    of each copy.
  rust/adapters/opencode/{loader,parser}.rs: the fork's retain_via_ledger
    stays beside upstream's v2 compaction scan; SessionMessageKind and
    MessageRowColumns carry the row type through the five-value scan.
  rust/adapters/codex/{replay,parser}.rs: open_rollout_reader_with_capacity
    and CodexForkReplay both land in the shared parser state.
  rust/crates/ccusage-core/src/{cache,pricing}.rs: the opencode-db-compaction
    namespace joins the ledger namespaces; the fork's embedded pricing loader
    and OnceLock path stay.
  rust/crates/ccusage/src/cli/timezone.rs: Cline joins the shared-args arm,
    the ClearCache arm stays unreachable from the parser.
  rust/Cargo.toml: the fork's exact `=` pins stay; jiff 0.2.41,
    schemars 1.2.2, serde 1.0.229, smallvec 1.16.5, toml 1.2.1.
  .github/workflows/{ci,release}.yaml: the fork's design is kept, including
    its own release flow. Upstream's merge-queue, tagpr and pkg.pr.new jobs
    need scripts this fork deleted, so they are not taken; the two
    save-nix-cache inputs upstream passes to the native-package actions are
    dropped because the fork's composite actions declare no such input.
  packages/ccusage-*/package.json (6 files), the contribution-gate scripts,
    sponsors.md, blacksmith.png and rust/adapters/claude/src/daily.rs stay
    deleted; the two Blacksmith SVG logos upstream added are not taken.
Docs: the agent lists take upstream's Cline addition but keep this fork's
`pi` name, because the CLI subcommand is still `pi`; upstream renamed the
docs to `pi-agent` without renaming the command.
Snapshots regenerated for the fork table renderer: the Cline adapter, CLI
and unified reports, codex issue_2046, opencode compaction, claude session
--id and the config schema (new Cline section).
Validation: cargo test --workspace --offline 1297 passed, 0 failed; cargo
clippy --workspace --all-targets clean; rustfmt --check clean.
Version: take upstream's 20.0.50 in package.json, apps/ccusage/package.json
and docs/package.json; the Rust workspace keeps version 0.0.0.
DO NOT push a v20.0.50 tag from this fork: release.yaml would publish fork
bytes under upstream identity.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(antigravity): 3x token/cost triple-counting when ModelUsageStats lacks message/response IDs

1 participant