Skip to content

feat(dsh): add DeepSeek Harness usage adapter - #1976

Closed
ryoppippi wants to merge 1 commit into
mainfrom
feat/dsh-adapter
Closed

ryoppippi wants to merge 1 commit into
mainfrom
feat/dsh-adapter

Conversation

@ryoppippi

@ryoppippi ryoppippi commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Adds DeepSeek Harness (ccusage dsh daily, weekly, monthly, and session) and includes DSH in unified reports. Reads legacy, v3, and v4 plain/zstd session logs, ignores streaming chunks, deduplicates final messages, and preserves cache token accounting. Provider model aliases resolve through bundled pricing, while direct DeepSeek models reuse the existing time-based rates.

Includes CLI/help, config/schema, Nix build-layer wiring, documentation, and JSON/table snapshots. Regression fixtures cover concatenated and truncated zstd frames, repeated and cross-session message IDs, unversioned logs, unknown models, and weekly boundaries.

Validation:

  • direnv exec . just fmt
  • direnv exec . just typecheck
  • direnv exec . just test
  • direnv exec . cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings
  • direnv exec . cargo hawk check --manifest-path rust/Cargo.toml -D warnings
  • direnv exec . just check
  • Synthetic compiled-CLI checks for all four focused DSH reports

Builds on the prior DSH adapter work by @oksure; the commit includes the requested co-author attribution.

Refs #1647

Summary by CodeRabbit

  • New Features
    • Added DSH as a supported data source, including daily, weekly, monthly, and session reports.
    • DSH usage is included in unified reports and supports standard report options, such as date filters and output formats.
    • Added support for discovering DSH data in ~/.dsh or a custom location set with DSH_HOME.
  • Documentation
    • Added DSH setup, configuration, CLI, and troubleshooting guidance.

Read final assistant messages from plain and zstd session histories,
including legacy and versioned logs with concatenated compressed frames.
Deduplicate messages by session, preserve cache token categories, and
resolve provider model names through existing pricing.

Wire focused and unified reports, configuration, help, docs, and the
Nix build layer. Cover parsing and report output with regression fixtures
and snapshots.

Co-authored-by: oksure <[email protected]>
@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Walkthrough

This pull request adds DSH as a data source. It reads DSH session logs and provides daily, weekly, monthly, and session reports through the individual and unified CLI commands.

Changes: DSH source support

Layer / File(s) Summary
Discover and load DSH sessions
rust/adapters/dsh/Cargo.toml, rust/adapters/dsh/README.md, rust/adapters/dsh/src/paths.rs, rust/adapters/dsh/src/parser.rs, rust/adapters/dsh/src/loader.rs
Discovers supported session files under DSH_HOME or ~/.dsh. Parses plain and .zstd JSONL, handles usage and pricing data, and loads entries with deduplication and timestamp ordering.
Build DSH reports and output
rust/adapters/dsh/src/report.rs, rust/adapters/dsh/src/lib.rs
Aggregates entries by day, week, month, or session. Generates JSON reports and table output through the adapter command.
Register DSH across CLI and reports
rust/Cargo.toml, nix/cargo-artifacts.nix, rust/crates/ccusage-adapter-all/*, rust/crates/ccusage-cli-parser/*, rust/crates/ccusage-cli/src/*, rust/crates/ccusage-config/src/*, rust/crates/ccusage-core/src/lib.rs, rust/crates/ccusage/Cargo.toml, rust/crates/ccusage/src/*
Adds DSH to command parsing and dispatch, configuration, unified-report loading, built-in source names, and workspace dependencies. Tests cover CLI parsing, configuration, report loading, time windows, and timezone validation.
Document DSH support
apps/ccusage/README.md, apps/ccusage/config-schema.json, docs/.vitepress/config.ts, docs/guide/*
Adds the DSH source guide and updates source lists, configuration examples, environment-variable guidance, and CLI examples.

Priority: Not assessed

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant DSHAdapter
  participant SessionLoader
  participant ReportBuilder
  participant Output
  CLI->>DSHAdapter: dispatch DSH report arguments
  DSHAdapter->>SessionLoader: load entries using shared options and pricing
  DSHAdapter->>ReportBuilder: filter and summarize entries
  ReportBuilder->>Output: provide report rows
  Output->>CLI: return JSON/JQ output or print table
Loading

Possibly related PRs

  • ccusage/ccusage#1959: Adds source-specific options to unified loading, which the DSH source registration uses.
  • ccusage/ccusage#1951: Tests provider-qualified DeepSeek pricing precedence, also implemented for DSH provider and model candidates.
Merge Risk: Low · up to d4d97

Some DSH users may see missing usage or incorrectly grouped weekly totals. These bounded issues warrant fixes or explicit acceptance before merging.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

🔧 Clippy (1.99.0)

Clippy execution failed

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ryoppippi

Copy link
Copy Markdown
Member Author

@coderabbitai @cubic-dev-ai Please review the DSH adapter on d4d9780, especially legacy/versioned plain and concatenated zstd logs, final-message deduplication, provider pricing, and unified report integration. Local formatting, type/lint checks, full Rust/Node tests, Clippy, Hawk, and host flake checks pass.

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

@coderabbitai @cubic-dev-ai Please review the DSH adapter on d4d9780, especially legacy/versioned plain and concatenated zstd logs, final-message deduplication, provider pricing, and unified report integration. Local formatting, type/lint checks, full Rust/Node tests, Clippy, Hawk, and host flake checks pass.

@ryoppippi I have started the AI code review. It will take a few minutes to complete.

@ryoppippi ryoppippi closed this Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: d4d9780930ab
Base SHA: 154e13bb6658

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 410.9ms 2.45 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 371.2ms 2.71 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 147.8ms 6.81 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 124.2ms 8.11 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 45.1ms 14.0ms 3.22x 56.00 MiB 25.23 MiB 0.45x 0.03 MiB/s 0.11 MiB/s
claude session --offline --json 0.00 MiB 49.5ms 13.9ms 3.57x 56.00 MiB 25.23 MiB 0.45x 0.03 MiB/s 0.11 MiB/s
codex daily --offline --json 0.00 MiB 40.4ms 14.1ms 2.86x 56.00 MiB 24.97 MiB 0.45x 0.02 MiB/s 0.06 MiB/s
codex session --offline --json 0.00 MiB 36.6ms 13.8ms 2.65x 56.00 MiB 24.97 MiB 0.45x 0.02 MiB/s 0.06 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 391.0ms 378.6ms 1.03x 935.35 MiB 921.35 MiB 0.99x 2.57 GiB/s 2.66 GiB/s
codex --offline --json 1.01 GiB 149.3ms 122.4ms 1.22x 519.43 MiB 511.19 MiB 0.98x 6.74 GiB/s 8.23 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 24.02 KiB 24.75 KiB +0.72 KiB 0.97x
installed native package binary 4886.47 KiB 4927.16 KiB +40.69 KiB 0.99x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: d4d9780930ab
Base SHA: 154e13bb6658

This compares the PR package against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 597.3ms 1.69 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 546.6ms 1.84 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 302.1ms 3.33 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 223.8ms 4.50 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 77.6ms 84.4ms 0.92x 55.25 MiB 55.50 MiB 1.00x 0.02 MiB/s 0.02 MiB/s
claude session --offline --json 0.00 MiB 77.9ms 70.0ms 1.11x 56.00 MiB 55.50 MiB 0.99x 0.02 MiB/s 0.02 MiB/s
codex daily --offline --json 0.00 MiB 74.0ms 72.9ms 1.02x 56.00 MiB 56.25 MiB 1.00x 0.01 MiB/s 0.01 MiB/s
codex session --offline --json 0.00 MiB 76.0ms 75.3ms 1.01x 55.50 MiB 56.00 MiB 1.01x 0.01 MiB/s 0.01 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 673.6ms 641.6ms 1.05x 911.09 MiB 919.10 MiB 1.01x 1.49 GiB/s 1.57 GiB/s
codex --offline --json 1.01 GiB 240.6ms 284.9ms 0.84x 455.43 MiB 479.44 MiB 1.05x 4.18 GiB/s 3.53 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 24.02 KiB 24.75 KiB +0.72 KiB 0.97x
installed native package binary 4886.47 KiB 4927.16 KiB +40.69 KiB 0.99x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

@ryoppippi I’ll review commit d4d9780930abbcc153fab4f6ab5fda887ed18d5d, with particular attention to:

  • Legacy and versioned plain logs.
  • Concatenated and truncated zstd logs.
  • Final-message deduplication and session boundaries.
  • Provider pricing and cache token accounting.
  • DSH integration into unified reports.

I’ll treat the reported local checks as supporting context, not as independent verification.


🧠 Learnings used
Learnt from: CR
Repo: ccusage/ccusage PR: 0
File: rust/adapters/README.md:0-0
Timestamp: 2026-10-14T20:33:20.330Z
Learning: Applies to rust/adapters/README.md : Applicability reference:
rust/adapters/AGENTS.md: # AGENTS.md - Agent Source Architecture
`README.md` covers adapter architecture, the shared-vs-source boundary, and the crate module shape;
`rust/adapters/opencode/src/` is a compact example of it. This file adds the workflow rules for
changing or adding an adapter.

Source excerpt:
# Agent Adapters

When several adapters expose the same raw-log shape, prefer a small shared Rust
helper over duplicating period/session aggregation. Keep highly specialized
loaders such as Codex parsing separate when their file format or pricing
semantics require it.
Learnt from: CR
Repo: ccusage/ccusage PR: 0
File: rust/adapters/README.md:0-0
Timestamp: 2026-10-14T20:33:20.330Z
Learning: Applies to rust/adapters/README.md : Applicability reference:
rust/adapters/AGENTS.md: # AGENTS.md - Agent Source Architecture
`README.md` covers adapter architecture, the shared-vs-source boundary, and the crate module shape;
`rust/adapters/opencode/src/` is a compact example of it. This file adds the workflow rules for
changing or adding an adapter.

Source excerpt:
# Agent Adapters

Each agent adapter owns source-specific log discovery, parsing, token mapping,
model mapping, source metadata, and source-specific pricing behavior. Shared
report rendering, date handling, progress, pricing fetcher lifecycle, file
walking, dedupe, and aggregation should stay in `ccusage-core` or
`ccusage-adapter-common` when the source data allows it.
Learnt from: ryoppippi
Repo: ccusage/ccusage PR: 0
File: :0-0
Timestamp: 2026-08-10T19:47:29.939Z
Learning: In `rust/crates/ccusage-core/src/pricing.rs`, `PricingMap::find` and `PricingMap::context_limit` use a fuzzy fallback that selects the longest matching key. Models.dev pricing snapshots must retain published pricing keys so known IDs resolve exactly. Separately priced tier IDs and unversioned IDs must be excluded from this fuzzy fallback through `PricingMap::exact_only`; otherwise they can incorrectly price a related model identifier.
⚠️ Action not completed

Pull request is closed.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues found — mergeable as-is.

Reviewed changes

  • New ccusage-adapter-dsh crate loading ${DSH_HOME:-~/.dsh}/sessions/**/session{,.vN}.jsonl{.zstd} histories into usage entries, skipping assistant/chunk streaming events and deduplicating final assistant/message records by (session_id, message_id) with newest filename generation winning.
  • Pricing keeps provider-qualified identities (together_ai/..., ollama/...) with exact-plus-fallback lookup, while direct deepseek models resolve through the existing timestamped DeepSeek schedule; unknown models retain usage with zero cost and a missing-pricing marker.
  • Weekly buckets start Monday (matching OpenCode/Antigravity and the unified default), with daily/monthly/session shapes following the shared summarizers plus JSON and terminal-table snapshots.
  • Full wiring: dsh CLI subcommand with daily/weekly/monthly/session reports, config schema and dsh namespace, unified adapter-all aggregation, BUILT_IN_AGENT_NAMES, last-window and timezone dispatch, Nix layer, docs, and npm schema artifact.

Pullfrog  | Fix it ➔ | View workflow run | Using muse-spark-1.3-contributor (free via Pullfrog for OSS) | 𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant