Skip to content

Claude adapter reads the whole history for date bounded reports #1791

Closed

Description

@alissonlinneker

What do you want to change?

Make the Claude adapter skip session files whose mtime predates the requested window, so a date bounded report stops reading the entire history.

Right now load_entries_inner in rust/adapters/claude/src/lib.rs discovers every file under projects/ and hands all of them to read_files_parallel, regardless of --since. The date filtering happens after the entries are materialized.

Why?

On a large history the cost is significant. Mine is 12.7 GB across 8,546 JSONL files, and a report covering a single day still reads all 8,546.

Measured on an M2 Max, release build, same query alternated between the two binaries across three rounds:

median wall time peak RSS
today's report, current behavior 10.3 s 3.88 GB
today's report, with the prune 6.7 s 3.21 GB

That single day touches 1,709 files. The other 6,837 are opened and parsed to produce nothing.

It matters most from the statusline hook, where the command runs on every render and several sessions render concurrently, so the machine pays that cost many times over.

The Codex adapter already carries this idea as has_date_bounds in rust/adapters/codex/src/loader.rs, added in #1598 for the same reason. The Claude path has no equivalent.

How? (optional)

Session JSONL files are append only, so a file last written before the window opened holds no entry inside it, and its mtime settles that without opening the file.

prune_files_outside_window sits between usage_files and read_files_parallel in both lib.rs and daily.rs. It narrows the file list only, never the entries parsed from a file that survives, and it returns the list untouched unless since is set, so an unbounded report is byte for byte what it is today.

Two cases deliberately keep every file rather than guess. Metadata that cannot be read, and a window that opens ahead of the current clock, where entry timestamps can outrun any mtime the filesystem reports. A 24 hour margin on the lower bound absorbs the gap between the timezone resolved bound and wall clock mtime, plus sessions flushed well after the entries they hold.

I have this implemented and tested against the suite, and I would like to open the PR if you are happy with the approach.

Activity

  1. pullfrog commented on Sep 23, 2026

    @pullfrog
    Contributor
  2. added
    enhancementNew feature or request
    priority:mediumA normal bug or meaningful improvement.
    triage:maintainableFits the repository's maintainable scope and needs review.
    on Sep 23, 2026
  3. added
    priority:lowMinor polish, documentation, or a low-impact edge case.
    triage:needs-reviewRequires a fresh technical or product decision.
    and removed
    priority:mediumA normal bug or meaningful improvement.
    triage:maintainableFits the repository's maintainable scope and needs review.
    on Sep 23, 2026
  4. github-actions commented on Sep 23, 2026

    @github-actions
    Contributor

    Pullfrog triage: priority:low
    Kind: maintenance
    Maintenance fit: needs review
    Confidence: medium

    Pullfrog was not highly confident; maintainer review is required. No incorrect reporting is shown. The request is an internal performance optimization to skip old files by mtime for date-bounded reports, matching the existing Codex pattern. The mtime safety, time margin, and long-term support cost need maintainer review.

    Decision: left open for maintainer review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority:lowMinor polish, documentation, or a low-impact edge case.triage:needs-reviewRequires a fresh technical or product decision.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions