[Bug] Large Codex session JSONL files are silently skipped, causing historical days to disappear from reports #952

Summary

@ccusage/codex appears to silently skip very large Codex session JSONL files.

In my case, I have a long-lived Codex session file that has grown to 1,136,784,076 bytes (~1.1 GB). Usage from dates contained in that file used to appear in ccusage-codex, but later disappeared even though the underlying file still contains valid token_count events for those dates.

This makes historical daily usage appear to "disappear retroactively".

What I expected

If a session JSONL file is valid and contains token_count events for a date, those dates should continue to appear in the daily report.

At minimum, if a file cannot be read, the CLI should emit a visible warning/error instead of silently dropping all usage from that session.

What actually happened

Running:

npx @ccusage/[email protected] daily --since 20260414 --until 20260421 --json

returns daily rows for:

• Apr 14, 2026
• Apr 15, 2026
• Apr 16, 2026
• Apr 17, 2026
• Apr 20, 2026
• Apr 21, 2026

But it omits:

• Apr 18, 2026
• Apr 19, 2026

Those omitted days are still present inside a valid session JSONL file on disk.

Environment

• macOS
• Codex Desktop session files under ~/.codex/sessions
• Reproduced with:
  • @ccusage/[email protected]
  • @ccusage/[email protected]
  • @ccusage/[email protected]

Minimal evidence

1. The large session file exists and is still valid JSONL

File:

~/.codex/sessions/2026/04/14/rollout-2026-04-14T23-00-47-019d8c82-5d54-7463-b5e3-0895b9f71664.jsonl

Size:

1136784076 bytes

A streaming line-by-line parse succeeds and finds:

• 45889 parsed JSONL records
• 13884 token_count events

2. That same file contains token_count events for the missing dates

From a streaming parse of that one file:

• 2026-04-18

  • 1396 token_count events
  • input tokens: 204165791
  • output tokens: 732536

• 2026-04-19

  • 408 token_count events
  • input tokens: 71243044
  • output tokens: 166956

So the data is present in the JSONL file even though @ccusage/codex no longer reports those days.

3. Direct readFile(..., "utf8") on a copied version of the file fails

When I copy that file to /tmp and do a plain Node read:

await fs.promises.readFile(file, "utf8")

Node throws:

RangeError: Invalid string length

4. Isolating only that one file still produces an empty report

I copied only that one JSONL file into a temporary CODEX_HOME layout:

/tmp/repro/sessions/2026/04/14/rollout-2026-04-14T23-00-47-019d8c82-5d54-7463-b5e3-0895b9f71664.jsonl

Then ran:

CODEX_HOME=/tmp/repro npx @ccusage/[email protected] daily --since 20260414 --until 20260421 --json

Actual output:

{"daily":[],"totals":null}

So the file is valid and contains token_count events, but the parser effectively drops it.

Why a ~1GB Codex JSONL file is plausible and should be handled

This does not appear to be local file corruption. In long-lived Codex Desktop threads, very large rollout JSONL files seem plausible and expected because the session can accumulate:

• many days of continued work in the same thread
• many tool calls and tool outputs
• command output from exec_command
• patch / diff output
• assistant messages
• repeated token_count events
• repeated compacted events that serialize replacement history

For this specific file, a streaming inspection shows:

• total file size: ~1084 MB
• compacted: ~859 MB
• response_item:function_call_output: ~70 MB
• event_msg:mcp_tool_call_end: ~41 MB
• response_item:message: ~33 MB
• event_msg:user_message: ~31 MB
• event_msg:exec_command_end: ~17 MB
• response_item:reasoning: ~11 MB
• event_msg:token_count: ~10 MB

There are 58 compacted records, and the largest single compacted line is about 39.5 MB.

So the large size is not inherently suspicious by itself; it can happen naturally for a long-running, tool-heavy Codex thread. The parser should not assume session files remain small enough for whole-file UTF-8 reads.

Likely root cause

From the current bundled implementation, it looks like the parser does something equivalent to:

const fileContent = await readFile(file, "utf8");
const lines = fileContent.split(/\r?\n/);

and then skips the file on read failure.

That is fragile for long-lived Codex rollout files that grow very large over time.

This also explains the retroactive disappearance:
once the session file grows beyond what readFile(..., "utf8") can handle, all historical days represented by that file disappear from the report.

Suggested fix

• Parse session JSONL files as a stream / line-by-line reader instead of loading the entire file into a single UTF-8 string
• Surface a visible warning/error when a session file cannot be read or parsed
• Add a regression test for large Codex session files or for the readFile(..., "utf8") failure path

Why I think this is not local data corruption

• The JSONL file is still stream-parseable
• The missing dates are still present in token_count events inside that file
• The problem reproduces across multiple @ccusage/codex versions
• Smaller/newer session files are still counted normally