Skip to content

Claude entries are dropped when usage.iterations[].model is null (Fable 5.1 since 2026-09-12) #1710

Closed

Description

@n0soc

What happened?

Summary

Since 2026-09-12, Claude Code writes Fable 5.1 assistant entries whose main-model iteration
inside message.usage.iterations carries an explicit "model": null. ccusage 20.0.20 discards
every such entry before deserialising it, so daily, session, blocks, and claude daily --instances report no Fable usage for the day at all. There is no warning; the model is absent
from the output as if it had not been used.

Environment

  • ccusage 20.0.20 (npm, npx ccusage@latest), Linux x64
  • Claude Code 2.1.268 and 2.1.269; the null appears on both, so it is not tied to a client
    release
  • Only Fable 5.1 entries carry the null. Opus 4.7 entries written by the same client on the same
    day have no model key in their iteration and are counted

Steps to reproduce

Put this single line into <config>/projects/p/s.jsonl and run
CLAUDE_CONFIG_DIR=<config> ccusage daily --json --offline:

{"type":"assistant","timestamp":"2026-09-12T04:38:42.296Z","version":"2.1.268","sessionId":"s","requestId":"req_1","message":{"id":"msg_1","model":"claude-fable-5-1","role":"assistant","usage":{"input_tokens":2,"cache_creation_input_tokens":55866,"cache_read_input_tokens":0,"output_tokens":289,"iterations":[{"type":"message","model":null,"input_tokens":2,"output_tokens":289,"cache_creation_input_tokens":55866,"cache_read_input_tokens":0}]}}}

Expected: one row for 2026-09-12 with 56,157 tokens under claude-fable-5-1.

Actual: "daily": [].

Deleting the "model":null pair from the iteration, or replacing null with
"claude-fable-5-1", makes the same line count as expected. A real 200-line session file with
44 such entries yields nothing; the same file with the null removed yields 1.7M tokens.

Cause

rust/adapters/claude/src/lib.rs, in the per-line loop of the file loader:

if has_unsupported_null_field(line) {
    continue;
}
let Ok(data) = serde_json::from_slice::<UsageEntry>(line) else {

has_unsupported_null_field scans the raw bytes for every :null and rejects the line when the
field name in front of it is in is_unsupported_nullable_field, which lists model. The scan
has no notion of nesting, so it cannot tell message.model (which the TypeScript loader's
schema rejected when null, and which the list exists to mirror) from
message.usage.iterations[].model. The typed structs would accept the nested null:
UsageIteration.model is Option<String>, and advisor_usages_from_line already filters
iterations without a model. The pre-check rejects the line before either gets a chance.

The existing test rejects_null_schema_fields_like_typescript_loader covers
message.model: null; no test covers a null inside iterations.

Proposed fix

Keep the top-level rejection and stop it from reaching into iterations. Two shapes, either of
which fixes the reported case:

  1. In has_unsupported_null_field, skip the span of the "iterations":[ ... ] array before
    scanning for :null, or scan only up to the first "iterations" marker. The fields the
    list rejects are all top-level or directly under message and message.usage, so nothing
    the check is meant to catch lives inside the array.
  2. Remove model from the byte list and reject a null message.model after typed
    deserialisation, where UsageEntry.message.model is already Option<String> and the loader
    can continue on None in the same place it handles <synthetic>.

A regression test in the same module:

#[test]
fn allows_null_model_inside_iterations() {
    assert!(!has_unsupported_null_field(
        br#"{"message":{"model":"m","usage":{"input_tokens":0,"iterations":[{"type":"message","model":null,"input_tokens":0,"output_tokens":0}]}}}"#
    ));
}

Workaround

Feed ccusage a copy of the transcripts with the null filled in from the entry's own
message.model (or with the key deleted), for example per file:

jq -c 'if (try .message.usage.iterations catch null) then (.message.model // null) as $m
       | .message.usage.iterations |= map(if has("model") and .model == null
                                          then (if $m then .model = $m else del(.model) end)
                                          else . end)
       else . end'

Expected behavior

No response

Version

20.0.20

Activity

  1. pullfrog commented on Sep 12, 2026

    @pullfrog
    Contributor
  2. added
    priority:highHigh user impact or an important regression.
    triage:maintainableFits the repository's maintainable scope and needs review.
    on Sep 12, 2026
  3. github-actions commented on Sep 12, 2026

    @github-actions
    Contributor

    Pullfrog triage: priority:critical

    A maintainer explicitly requested an implementation attempt. Valid bug with reproduction: null model in iterations drops Fable 5.1 entries silently. Confirmed by other users and maintainer requested a fix. No merged fix yet.

    Decision: kept open; Pullfrog will attempt a focused implementation PR.

  4. iamnolanhu commented on Sep 12, 2026

    @iamnolanhu

    Hit the same thing today: every Fable 5.1 line dropped, Opus 5 mostly gone.

    I have a fix on a branch: exempt iterations[].model nulls from has_unsupported_null_field in both envelopes so the line is retained (iteration model stays optional, other null checks unchanged). Five regression tests: null-model line, missing-model line, empty iterations, agent-progress envelope, unrelated null still rejected. fmt, clippy -D warnings and the adapter tests are green.

    On my logs for 2026-09-12 the report went from Fable absent / Opus 5 617 output tokens to Fable 136k / Opus 5 267k, Haiku unchanged.

    Branch: https://github.com/iamnolanhu/ccusage/tree/fix/iterations-null-model

    Happy to open the PR if you lgtm me, or take the patch directly.

  5. rremo commented on Sep 12, 2026

    @rremo

    same for me; Fable 5.1 and Opus 5 is gone - i only see sonnet 5

  6. ryoppippi commented on Sep 12, 2026

    @ryoppippi
    Member

    thanks. i'll check it out and ship it in a few days

  7. ryoppippi commented on Sep 12, 2026

    @ryoppippi
    Member

    @pullfrog could you work on it

  8. pullfrog commented on Sep 12, 2026

    @pullfrog
    Contributor

    Fixed #1710 locally on branch pullfrog/1710-iterations-null-model (commit 4b22a3c).

    Change in rust/adapters/claude/src/lib.rs: has_unsupported_null_field now computes the byte ranges of every "iterations": [...] array (string-aware bracket matching) and skips any :null inside them, so message.usage.iterations[].model: null (Fable 5.1) no longer drops the entry. Top-level message.model: null and other listed nulls are still rejected. Shared by both load_entries and daily loaders.

    Tests: added allows_null_model_inside_iterations, keeps_entry_with_null_model_inside_iterations (uses the exact line from the issue, asserts 1 entry under claude-fable-5-1), and still_rejects_top_level_null_model_next_to_iterations. cargo test -p ccusage-adapter-claude --features ccusage-core/fetch-litellm-pricing: 25 passed, 0 failed. Clippy -D warnings clean, cargo fmt --check clean.

    Could not open a PR: pullfrog_push_branch failed with the actual error git push failed (exit 128): remote: Permission to ccusage/ccusage.git denied to github-actions[bot]. fatal: unable to access 'https://github.com/ccusage/ccusage.git/': The requested URL returned error: 403. The commit is on the local branch; a maintainer with push rights can push it or apply the same diff.

    Task list (2/2 completed)
    • Fix has_unsupported_null_field to exempt iterations[].model nulls
    • Add regression tests and verify

    Pullfrog  | View workflow run | via Pullfrog | Using openrouter/meta/muse-spark-1.3-contributor (free via Pullfrog for OSS) | 𝕏

  9. added
    priority:criticalCritical impact or an urgent security or release issue.
    and removed
    priority:highHigh user impact or an important regression.
    on Sep 12, 2026
  10. added a commit that references this issue on Sep 13, 2026
    d9e20f2
  11. n0soc commented on Sep 13, 2026

    @n0soc
    Author

    Three fixes now exist for this, and I read all of them against main:

    All three patch has_unsupported_null_field only and produce the same result for the line in this issue: the entry is kept and counted under message.model, in both loaders and both envelopes. They differ at the edges.

    What they exempt. #1715 skips every null inside an "iterations":[...] span. #1719 and the iamnolanhu branch exempt only model there, so a null token count inside an iteration still drops the whole line. The broader exemption is safe: TokenUsageRaw has no iterations field, so nothing in the array reaches the typed parse of the main entry, and speed is already Option<Speed>. A null inside the array can at most make advisor_usages_from_line return nothing. Every field in is_unsupported_nullable_field lives outside the array, so the pre-check was never meant to police its contents. This is also why the CodeRabbit finding on #1715 does not need a change.

    Cost per line. #1715 and #1719 add one byte pass on lines that carry "iterations". The iamnolanhu branch parses the whole line into a serde_json::Value on every "model":null hit, which is now every Fable 5.1 assistant line, inside a check whose purpose is to avoid deserialization.

    Scanner details. #1715 has a string-state bug on its fallthrough path; I left an inline comment with a one-line fix and a test. #1719 first used a plain marker search for the array; after review it now walks each usage object member by member and takes only a direct iterations array, with whitespace tolerated and decoy arrays elsewhere on the line ignored. I read that version and found nothing to flag.

    Tests. The iamnolanhu branch has the widest coverage: both loaders end to end, the agent-progress envelope, a missing model key, an empty array. #1715 and #1719 test the general loader end to end and the pre-check in isolation. #1719 also reports before-and-after numbers on real data.

    My suggestion: either #1715 with the fallthrough fix or #1719 as it standsnow. #1719 has the cleaner scanner; #1715 has the exemption that survives a future null on another field inside the array, and #1719 could adopt that by dropping its field == b"model" condition. Whichever lands, please add the daily loader and agent-progress tests from the iamnolanhu branch so
    both code paths stay covered.

  12. added a commit that references this issue on Sep 15, 2026
    d341949
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:criticalCritical impact or an urgent security or release issue.triage:maintainableFits the repository's maintainable scope and needs review.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions