Skip to content

Codex totals are inflated when duplicate token_count events repeat unchanged total_token_usage #876

Closed

Description

@JuncongHe

Summary

I can reproduce inflated Codex totals with the current released CLI (@ccusage/codex 18.0.8).

On my local logs, the overcount comes from summing info.last_token_usage even when consecutive token_count events carry the same cumulative info.total_token_usage.

I checked this against my own session JSONL files and local Codex SQLite state, and PR #824 appears to be the correct fix for this specific duplicate-total pattern.

Version

  • @ccusage/codex: 18.0.8

Current released output

Running:

npx -y @ccusage/[email protected] monthly -s 2026-03-01 -u 2026-03-05 -j

reported:

  • totalTokens = 8,321,045,582

Local audit of the same closed window

I audited the matching JSONL files under:

  • ~/.codex/sessions/2026/03/01
  • ~/.codex/sessions/2026/03/02
  • ~/.codex/sessions/2026/03/03
  • ~/.codex/sessions/2026/03/04
  • ~/.codex/sessions/2026/03/05

and compared them with the corresponding thread totals in ~/.codex/state_5.sqlite.

Across that closed window on my machine:

  • Sessions: 88
  • token_count events: 66,386
  • Duplicate events with unchanged total_token_usage: 42,869
  • Current released logic (sum(last_token_usage.total_tokens)): 8,299,097,962
  • PR fix(codex): ignore duplicate token_count totals #824 logic (ignore last_token_usage when total_token_usage is unchanged): 2,899,127,807
  • SQLite threads.tokens_used: 2,899,127,807

So for this dataset, the PR #824 logic matches the local SQLite totals exactly, while the current released logic overcounts by about 2.86x.

Single-session reproduction

For this session file:

  • rollout-2026-03-04T00-00-40-019cb46e-1a1c-79f2-bc67-6551633c9e03.jsonl

I measured:

  • token_count events: 8,545
  • Duplicate events with unchanged totals: 5,703
  • Current released logic: 1,231,236,178
  • PR fix(codex): ignore duplicate token_count totals #824 logic: 411,914,227
  • sum(diff(total_token_usage.total_tokens)): 411,914,227
  • SQLite threads.tokens_used: 411,914,227

So the single-session case also lines up exactly with PR #824.

Why I think PR #824 is the right fix

This is the exact pattern in my logs:

  1. A token_count event is emitted.
  2. The next token_count event repeats the same last_token_usage.
  3. total_token_usage does not advance.
  4. The current loader still adds last_token_usage again.

PR #824 adds the missing guard for that case and makes my local totals line up with both:

  • the cumulative total_token_usage deltas in the JSONL logs
  • the persisted threads.tokens_used values in state_5.sqlite

Related PR

Activity

  1. ryoppippi commented on May 19, 2026

    @ryoppippi
    Member

    Thank you for the report/suggestion and for the related TypeScript-era work.

    I am sorry that the Rust migration made the original TypeScript implementation no longer mergeable as-is. The production CLI has moved from the TypeScript implementation to the Rust implementation under rust/crates/ccusage. The related TypeScript PR has been closed as obsolete, so I am closing this issue as well to keep the tracker focused.

    If this behavior is still needed, please open a fresh issue or PR against the current Rust CLI with current-version output and a minimal reproduction or concrete use case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions