Skip to content

fix(claude): scope requestless dedupe by timestamp so daily matches weekly/monthly - #1799

Merged
ryoppippi merged 6 commits into
mainfrom
pullfrog/issue-1797-run-36116493263-attempt-1
Sep 25, 2026
Merged

ryoppippi merged 6 commits into
mainfrom
pullfrog/issue-1797-run-36116493263-attempt-1

Conversation

@github-actions

@github-actions github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1797: ccusage claude daily reported ~475M tokens while weekly/monthly reported ~270M over the same window.

Root cause

The two Claude loaders disagreed on requestless (requestId-absent) dedupe. The entry loader behind weekly/monthly/session/blocks collapsed the same message ID within a session across distinct timestamps. The daily loader keys requestless entries by message ID, session, and timestamp. Gateways that answer every response with one message ID and no request ID (#1635) therefore lost usage in weekly/monthly, while daily counted each response.

Changes

  • Timestamp-scoped requestless dedupe in both loaders. Both now share one exact-match key (usage_dedupe_hash): matching message and request IDs identify one response across sessions; without a request ID the key is scoped to session and timestamp. Repeated writes at the same timestamp still collapse to the most complete record.
  • Requestless /btw replays stay deduplicated. A sidechain replay where neither entry has a requestId matches its parent across timestamps, as the daily loader already did. Replays carrying a new request ID keep the regular loader's existing timestamp rule.
  • Linear loading for gateway logs. Parent candidates look up a replay route that only indexes sidechain entries, new entries are appended without a duplicate scan, and replacements only register keys that actually changed. Applied to both loaders.

Validation

  • New CLI tests run claude daily|weekly|monthly|session and require identical totals for a requestless sidechain replay and for gateway responses reusing one message ID. The replay test fails on the previous PR head and the gateway test fails on main.
  • cargo test --workspace (943 passed), cargo clippy --workspace --all-targets -- -D warnings, and just hawk are clean.
  • Real Claude history (all entries carry request IDs): claude daily|weekly|monthly|session --since 20260901 --until 20260924 --json output is byte-identical to main.
  • Release-build fixtures with one reused gateway message ID:
Fixture Command main this PR
8k responses weekly 0.01s (102 tokens, undercounted) 0.01s (816,000 tokens)
8k responses daily 0.23s 0.15s
128k responses + same-timestamp rewrites monthly 0.18s (undercounted) 0.21s
128k responses + same-timestamp rewrites daily 62.8s 0.29s

@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 25b8ad22-e5ef-418c-8c68-673e9e1d5b47

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ccusage-guide d93a58c Commit Preview URL

Branch Preview URL
Sep 25 2026, 04:06 PM

@pkg-pr-new

pkg-pr-new Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

ccusage

npx https://pkg.pr.new/ccusage@1799

@ccusage/ccusage-darwin-arm64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-arm64@1799

@ccusage/ccusage-darwin-x64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-x64@1799

@ccusage/ccusage-linux-arm64

npx https://pkg.pr.new/@ccusage/ccusage-linux-arm64@1799

@ccusage/ccusage-linux-x64

npx https://pkg.pr.new/@ccusage/ccusage-linux-x64@1799

@ccusage/ccusage-win32-x64

npx https://pkg.pr.new/@ccusage/ccusage-win32-x64@1799

commit: d93a58c

Scoping requestless exact matches by timestamp let /btw sidechain
replays without a request ID through, because the regular loader's
replay fallback also required equal timestamps. Match requestless
replays across timestamps, as the daily loader does.

Share the timestamp-aware exact key between both loaders so gateway
logs that reuse one message ID no longer land in a single bucket, and
give parent candidates a replay route that only holds sidechain entries.
New entries are appended to their buckets without a duplicate scan, so
loading stays linear.
Run daily, weekly, monthly and session over a requestless /btw replay
and over gateway responses that share one message ID, and require the
same totals from every report.
@github-actions

Copy link
Copy Markdown
Contributor Author

ccusage performance comparison

PR SHA: aefb44531489
Base SHA: 7e3a180c0751

Performance comparison skipped.

Base package URL was not ready before 300.000s. Fixture performance comparison requires a base package when --base-dir is not provided.

Base package: 7e3a180c0751

@github-actions

Copy link
Copy Markdown
Contributor Author

ccusage performance comparison

PR SHA: aefb44531489
Base SHA: 7e3a180c0751

Performance comparison skipped.

Base package URL was not ready before 300.000s. Fixture performance comparison requires a base package when --base-dir is not provided.

Base package: 7e3a180c0751

@github-actions

Copy link
Copy Markdown
Contributor Author

ccusage performance comparison

PR SHA: 7668a04bb20a
Base SHA: 15830fb1636f

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 511.2ms 1.97 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 510.3ms 1.97 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 183.3ms 5.49 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 147.1ms 6.84 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 43.7ms 20.7ms 2.11x 55.25 MiB 22.96 MiB 0.42x 0.04 MiB/s 0.07 MiB/s
claude session --offline --json 0.00 MiB 47.1ms 12.1ms 3.90x 55.00 MiB 22.96 MiB 0.42x 0.03 MiB/s 0.13 MiB/s
codex daily --offline --json 0.00 MiB 54.5ms 17.7ms 3.09x 55.25 MiB 21.21 MiB 0.38x 0.02 MiB/s 0.05 MiB/s
codex session --offline --json 0.00 MiB 53.9ms 16.1ms 3.34x 55.25 MiB 21.21 MiB 0.38x 0.02 MiB/s 0.05 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 550.8ms 489.1ms 1.13x 959.07 MiB 951.09 MiB 0.99x 1.83 GiB/s 2.06 GiB/s
codex --offline --json 1.01 GiB 179.8ms 143.2ms 1.26x 431.43 MiB 429.41 MiB 1.00x 5.60 GiB/s 7.03 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 23.69 KiB 23.69 KiB +0.00 KiB 1.00x
installed native package binary 4638.97 KiB 4639.53 KiB +0.56 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor Author

ccusage performance comparison

PR SHA: 7668a04bb20a
Base SHA: 15830fb1636f

This compares the PR package against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 396.0ms 2.54 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 359.4ms 2.80 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 132.8ms 7.58 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 108.3ms 9.29 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 40.7ms 40.2ms 1.01x 55.00 MiB 55.00 MiB 1.00x 0.04 MiB/s 0.04 MiB/s
claude session --offline --json 0.00 MiB 36.2ms 35.5ms 1.02x 55.00 MiB 55.00 MiB 1.00x 0.04 MiB/s 0.04 MiB/s
codex daily --offline --json 0.00 MiB 35.4ms 34.6ms 1.02x 55.25 MiB 55.00 MiB 1.00x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 34.5ms 35.2ms 0.98x 55.00 MiB 55.00 MiB 1.00x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 404.4ms 388.5ms 1.04x 907.09 MiB 915.34 MiB 1.01x 2.49 GiB/s 2.59 GiB/s
codex --offline --json 1.01 GiB 133.0ms 136.1ms 0.98x 417.41 MiB 425.41 MiB 1.02x 7.57 GiB/s 7.40 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 23.69 KiB 23.69 KiB +0.00 KiB 1.00x
installed native package binary 4638.97 KiB 4639.53 KiB +0.56 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

A requestless gateway rewrite at the same timestamp replaces the
survivor without changing its exact key or replay route, yet both were
registered again with a linear membership scan over buckets that now
hold every retained timestamp. Only register keys that changed.
@github-actions

Copy link
Copy Markdown
Contributor Author

ccusage performance comparison

PR SHA: d93a58cff87c
Base SHA: 15830fb1636f

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 384.7ms 2.62 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 380.6ms 2.65 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 141.5ms 7.12 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 111.7ms 9.02 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 43.1ms 17.4ms 2.48x 55.00 MiB 22.96 MiB 0.42x 0.04 MiB/s 0.09 MiB/s
claude session --offline --json 0.00 MiB 34.4ms 10.9ms 3.15x 54.75 MiB 22.96 MiB 0.42x 0.04 MiB/s 0.14 MiB/s
codex daily --offline --json 0.00 MiB 36.3ms 10.3ms 3.54x 55.00 MiB 21.20 MiB 0.39x 0.02 MiB/s 0.08 MiB/s
codex session --offline --json 0.00 MiB 31.9ms 10.4ms 3.06x 55.00 MiB 21.20 MiB 0.39x 0.03 MiB/s 0.08 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 363.4ms 359.8ms 1.01x 937.09 MiB 931.09 MiB 0.99x 2.77 GiB/s 2.80 GiB/s
codex --offline --json 1.01 GiB 137.5ms 108.8ms 1.26x 419.42 MiB 441.42 MiB 1.05x 7.32 GiB/s 9.25 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 23.69 KiB 23.69 KiB +0.00 KiB 1.00x
installed native package binary 4638.97 KiB 4640.28 KiB +1.31 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor Author

ccusage performance comparison

PR SHA: d93a58cff87c
Base SHA: 15830fb1636f

This compares the PR package against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 383.1ms 2.63 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 365.5ms 2.75 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 128.9ms 7.81 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 107.4ms 9.37 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 38.1ms 31.2ms 1.22x 55.00 MiB 55.00 MiB 1.00x 0.04 MiB/s 0.05 MiB/s
claude session --offline --json 0.00 MiB 34.0ms 38.8ms 0.88x 55.00 MiB 55.00 MiB 1.00x 0.05 MiB/s 0.04 MiB/s
codex daily --offline --json 0.00 MiB 31.8ms 31.5ms 1.01x 55.00 MiB 55.00 MiB 1.00x 0.03 MiB/s 0.03 MiB/s
codex session --offline --json 0.00 MiB 30.6ms 32.4ms 0.95x 55.00 MiB 55.00 MiB 1.00x 0.03 MiB/s 0.03 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 383.1ms 370.3ms 1.03x 923.09 MiB 933.09 MiB 1.01x 2.63 GiB/s 2.72 GiB/s
codex --offline --json 1.01 GiB 129.1ms 133.8ms 0.96x 433.43 MiB 445.41 MiB 1.03x 7.80 GiB/s 7.52 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 23.69 KiB 23.69 KiB +0.00 KiB 1.00x
installed native package binary 4638.97 KiB 4640.28 KiB +1.31 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

~Double total tokens for the running month reported on daily vs monthly and weekly

1 participant