Skip to content

Commit 8841f92

Browse files
ryoppippijay-tau
andauthored
feat(copilot): read session-state usage events (ccusage#1676)
* feat(copilot): read session-state usage events Read session.shutdown model metrics from Copilot session-state files alongside OTel, including cache bucket normalization and canonical model identities for pricing and source deduplication. Support COPILOT_HOME, session-aware monthly and session output with snapshots, and documented source discovery while preserving malformed-line handling and OTel fallback behavior. * fix(copilot): reconcile cumulative session usage Keep only the latest session-state shutdown usage for each canonical session/model pair before suppressing matching OTel records. Normalize both Copilot 1m suffix spellings so pricing and source selection share identity. Replace manually built report snapshots with sanitized fixture-backed focused and unified CLI stdout snapshots, remove the stale terminal test seam, and correct the reasoning-token documentation. Co-authored-by: Joel Tony <[email protected]> * fix(copilot): retain post-shutdown OTel usage Keep matching OTel records emitted after the latest cumulative session-state shutdown so resumed sessions are not undercounted. Suppress records at or before the shutdown timestamp and cover both boundaries in the focused loader regression test. Co-authored-by: Joel Tony <[email protected]> * docs(copilot): clarify post-shutdown OTel filtering Document the timestamp boundary for matching OpenTelemetry rows. Rows at or before the latest canonical shutdown are suppressed, while rows emitted after shutdown by a resumed session remain included. Co-authored-by: Joel Tony <[email protected]> * fix(copilot): reconcile ranged session usage Select cumulative shutdown snapshots as of --until and subtract the preceding snapshot for --since, while keeping resumed OTEL rows and duplicate-event dedupe intact. Include OTEL totals that prove separate reasoning tokens in billed and report totals and map session-state request counts above one to messageCount. Add range, request-count, reasoning, and resumed-session regressions with refreshed Copilot fixtures and snapshots. Co-authored-by: Joel Tony <[email protected]> * docs(copilot): document recursive OTel sources Document recursive discovery under the Copilot OTel directory while retaining the session-state glob and single-file exporter wording. Describe the range-aware cumulative snapshot behavior and the distinction between folded session-state reasoning and separately evidenced OTEL reasoning. Co-authored-by: Joel Tony <[email protected]> * docs(copilot): clarify session-state usage fields Document that session-state reports read requests.count for messageCount while ignoring requests.cost. Add focused daily JSON and table snapshots to cover the parser output alongside the existing monthly and session coverage. Co-authored-by: Joel Tony <[email protected]> * fix(copilot): count retained OTel usage rows Count each emitted OpenTelemetry usage row as one message while retaining the existing session-state requests.count greater-than-one gate. Keep cumulative range subtraction and post-shutdown filtering unchanged so resumed OTel rows add only usage not covered by the canonical shutdown snapshot. Co-authored-by: Joel Tony <[email protected]> * test(copilot): refresh monthly and session table snapshots Refresh the focused and unified Copilot monthly/session table snapshots for the current terminal table formatting. Co-authored-by: Joel Tony <[email protected]> * test(zcode): refresh session table snapshot Refresh the ZCode session table snapshot for the current terminal table formatting. Co-authored-by: Joel Tony <[email protected]> --------- Co-authored-by: Joel Tony <[email protected]>
1 parent d39a09d commit 8841f92

21 files changed

Lines changed: 1880 additions & 93 deletions

‎docs/guide/configuration.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,8 @@ export OPENCODE_DATA_DIR="$HOME/.local/share/opencode"
5555
export AMP_DATA_DIR="$HOME/.local/share/amp"
5656
export PI_AGENT_DIR="$HOME/.pi/agent/sessions"
5757
export KILO_DATA_DIR="$HOME/.local/share/kilo"
58-
export COPILOT_OTEL_FILE_EXPORTER_PATH="$HOME/.copilot/otel/copilot-otel.jsonl"
58+
export COPILOT_HOME="$HOME/.copilot"
59+
export COPILOT_OTEL_FILE_EXPORTER_PATH="$COPILOT_HOME/otel/copilot-otel.jsonl"
5960
export ZCODE_HOME="$HOME/.zcode"
6061
```
6162

‎docs/guide/copilot/index.md‎

Lines changed: 24 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# GitHub Copilot CLI Data Source (Beta)
22

3-
> GitHub Copilot CLI support is experimental. The adapter reads local OpenTelemetry JSONL files only.
3+
> GitHub Copilot CLI support is experimental. The adapter reads local session-state and OpenTelemetry JSONL files.
44
5-
ccusage can read GitHub Copilot CLI OpenTelemetry file exports as one of its supported local data sources. It uses the same reporting experience as the rest of ccusage: responsive tables, JSON output, LiteLLM-based pricing, cache token accounting, and all-source aggregation.
5+
ccusage can read GitHub Copilot CLI session-state and OpenTelemetry files as supported local data sources. It uses the same reporting experience as the rest of ccusage: responsive tables, JSON output, LiteLLM-based pricing, cache token accounting, and all-source aggregation.
66

77
## Focused Views
88

@@ -24,19 +24,29 @@ pnpm dlx ccusage copilot --help
2424

2525
## Data Source
2626

27-
The CLI reads Copilot OpenTelemetry JSONL files from `~/.copilot/otel/*.jsonl` and also includes the explicit file pointed to by `COPILOT_OTEL_FILE_EXPORTER_PATH`.
27+
The CLI reads Copilot session-state shutdown events from `${COPILOT_HOME:-~/.copilot}/session-state/*/events.jsonl` by default. It also reads OpenTelemetry JSONL files recursively from `${COPILOT_HOME:-~/.copilot}/otel/**/*.jsonl` and includes the single explicit file pointed to by `COPILOT_OTEL_FILE_EXPORTER_PATH`. Set `COPILOT_HOME` to the Copilot data root when the default `~/.copilot` directory has been relocated.
2828

29-
Enable these variables before starting or resuming a Copilot CLI session. Sessions that ran without OpenTelemetry file export enabled do not produce local JSONL usage data for ccusage to read.
29+
Session-state files do not require OpenTelemetry configuration. Shutdown usage is cumulative per canonical session/model pair, so only the latest shutdown is retained. For date-bounded reports, ccusage selects the latest snapshot visible through `--until` and subtracts the latest earlier snapshot before `--since` when one exists. When both sources contain the same session/model pair, the session-state usage is used and matching OpenTelemetry rows are suppressed only when their timestamps are at or before the latest canonical shutdown timestamp for that pair; rows emitted after that timestamp by a resumed session are retained. OpenTelemetry rows for other session/model pairs remain available.
30+
31+
For session-state, only `session.shutdown` events are used. For each `data.modelMetrics.<model>` entry, ccusage reads its `usage` fields and, when greater than one, `requests.count` for `messageCount`; each retained OpenTelemetry usage row contributes one message. `requests.cost` is ignored and costs continue to use ccusage's normal token pricing. In session-state data, `inputTokens` includes both cache buckets, so ccusage derives uncached input as `max(inputTokens - cacheReadTokens - cacheWriteTokens, 0)` before populating `inputTokens`; cache reads and cache writes are then reported separately. `reasoningTokens` is a subset of `outputTokens`, so it is not added again to output, total tokens, or cost. Copilot model IDs with `-1m` or `-1m-internal` suffixes, such as `claude-opus-4.6-1m`, are normalized to their priced model name for pricing and source deduplication.
32+
33+
For example, a `claude-opus-4.7` shutdown with `inputTokens=100`, `outputTokens=50`, `cacheReadTokens=10`, and `cacheWriteTokens=20` is reported as 70 input tokens, 50 output tokens, 20 cache-creation tokens, and 10 cache-read tokens. With the embedded Opus pricing, its calculated cost is exactly `$0.00173`.
34+
35+
Enable these variables before starting or resuming a Copilot CLI session when you want OTel data. Sessions that ran without OpenTelemetry file export remain readable from their session-state files.
3036

3137
```bash
38+
export COPILOT_HOME="$HOME/.copilot"
3239
export COPILOT_OTEL_ENABLED=true
3340
export COPILOT_OTEL_EXPORTER_TYPE=file
34-
mkdir -p "$HOME/.copilot/otel"
35-
export COPILOT_OTEL_FILE_EXPORTER_PATH="$HOME/.copilot/otel/copilot-otel-$(date +%Y%m%d-%H%M%S).jsonl"
41+
mkdir -p "$COPILOT_HOME/otel"
42+
export COPILOT_OTEL_FILE_EXPORTER_PATH="$COPILOT_HOME/otel/copilot-otel-$(date +%Y%m%d-%H%M%S).jsonl"
3643
```
3744

3845
```text
39-
~/.copilot/
46+
${COPILOT_HOME:-~/.copilot}/
47+
├── session-state/
48+
│ └── <session-id>/
49+
│ └── events.jsonl
4050
└── otel/
4151
└── *.jsonl
4252
```
@@ -53,24 +63,26 @@ These views support `--json` for structured output, `--compact` for narrow termi
5363

5464
## What Gets Calculated
5565

56-
- **Token usage** - chat spans are preferred, with inference logs and agent-turn logs used as fallbacks.
66+
- **Token usage** - the latest cumulative session shutdown usage is read from session-state files; OTel chat spans are preferred within the OTel source, with inference logs and agent-turn logs used as fallbacks.
5767
- **Cache tokens** - cache read and cache creation token attributes are counted when present.
58-
- **Reasoning tokens** - reasoning output tokens are included in total tokens and cost calculation.
59-
- **Pricing** - costs are calculated from LiteLLM pricing data using the reported model name.
68+
- **Input tokens** - session-state `inputTokens` is normalized to uncached input after subtracting cache reads and cache writes.
69+
- **Reasoning tokens** - session-state reasoning tokens are already included in output tokens; OpenTelemetry reasoning is included when total usage metadata shows it is separate.
70+
- **Pricing** - costs are calculated from LiteLLM pricing data using the normalized model name; both `-1m` and `-1m-internal` suffixes are removed.
6071

6172
## Environment Variables
6273

6374
| Variable | Description |
6475
| --------------------------------- | ---------------------------------------------------- |
76+
| `COPILOT_HOME` | Copilot data root; defaults to `~/.copilot` |
6577
| `COPILOT_OTEL_FILE_EXPORTER_PATH` | Explicit Copilot OpenTelemetry JSONL file to include |
6678
| `LOG_LEVEL` | Adjust verbosity (0 silent ... 5 trace) |
6779

6880
## Troubleshooting
6981

7082
::: details No Copilot usage data found
71-
Ensure OpenTelemetry file export is enabled and the exporter path points to an existing `.jsonl` file, or place exported `.jsonl` files under `~/.copilot/otel/`.
83+
Ensure Copilot session-state files exist under `${COPILOT_HOME:-~/.copilot}/session-state/<session-id>/events.jsonl`, or enable OpenTelemetry file export and place exported `.jsonl` files under `${COPILOT_HOME:-~/.copilot}/otel/`.
7284

73-
If you are using `copilot --resume`, set the OpenTelemetry environment variables before running the resume command. Earlier activity from sessions started without file export cannot be recovered by ccusage.
85+
If you are using `copilot --resume`, session-state events remain available without OpenTelemetry. OTel-only activity from sessions started without file export cannot be recovered by ccusage.
7486
:::
7587

7688
::: details Costs showing as $0.00

‎docs/guide/environment-variables.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ ccusage supports several environment variables for configuration and customizati
44

55
## Agent Data Directories
66

7-
ccusage detects supported data source files from conventional locations by default. Set these variables when your data lives somewhere else. Directory variables can be one directory or a comma-separated list of directories; the Copilot variable points at one explicit JSONL export file, and `GROK_HOME` accepts a single root only:
7+
ccusage detects supported data source files from conventional locations by default. Set these variables when your data lives somewhere else. Directory variables can be one directory or a comma-separated list of directories; `COPILOT_HOME` and `GROK_HOME` accept a single root, while `COPILOT_OTEL_FILE_EXPORTER_PATH` points at one explicit JSONL export file:
88

99
| Variable | Agent | Default |
1010
| --------------------------------- | -------------- | ---------------------------------------------------- |
@@ -21,6 +21,7 @@ ccusage detects supported data source files from conventional locations by defau
2121
| `KILO_DATA_DIR` | Kilo | `~/.local/share/kilo` |
2222
| `KIMI_DATA_DIR` | Kimi | `~/.kimi`, `~/.kimi-code` |
2323
| `QWEN_DATA_DIR` | Qwen | `~/.qwen` |
24+
| `COPILOT_HOME` | Copilot CLI | `~/.copilot` |
2425
| `COPILOT_OTEL_FILE_EXPORTER_PATH` | Copilot CLI | Explicit `.jsonl` file |
2526
| `GEMINI_DATA_DIR` | Gemini CLI | `~/.gemini/tmp` |
2627
| `ANTIGRAVITY_DATA_DIR` | Antigravity | `~/.gemini/antigravity*` and `~/.config/antigravity` |
@@ -42,6 +43,7 @@ export OPENCLAW_DIR="/path/to/openclaw,/archive/openclaw"
4243
export KILO_DATA_DIR="/path/to/kilo,/archive/kilo"
4344
export KIMI_DATA_DIR="/path/to/kimi,/archive/kimi"
4445
export QWEN_DATA_DIR="/path/to/qwen,/archive/qwen"
46+
export COPILOT_HOME="/path/to/copilot"
4547
export COPILOT_OTEL_FILE_EXPORTER_PATH="/path/to/copilot-otel.jsonl"
4648
export GEMINI_DATA_DIR="/path/to/gemini/tmp,/archive/gemini/tmp"
4749
export ANTIGRAVITY_DATA_DIR="/path/to/antigravity,/archive/antigravity"

‎docs/guide/getting-started.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -176,7 +176,7 @@ If ccusage shows no data, check:
176176
- Kimi: `${KIMI_DATA_DIR:-~/.kimi}` (also scans `~/.kimi-code`)
177177
- OpenClaw: `${OPENCLAW_DIR:-~/.openclaw}` (also scans `~/.clawdbot`, `~/.moltbot`, `~/.moldbot`)
178178
- Qwen: `${QWEN_DATA_DIR:-~/.qwen}`
179-
- GitHub Copilot CLI: `~/.copilot/otel/*.jsonl` or `COPILOT_OTEL_FILE_EXPORTER_PATH`
179+
- GitHub Copilot CLI: `${COPILOT_HOME:-~/.copilot}/session-state/*/events.jsonl`, `${COPILOT_HOME:-~/.copilot}/otel/**/*.jsonl`, or the single file specified by `COPILOT_OTEL_FILE_EXPORTER_PATH`
180180
- Antigravity: `${ANTIGRAVITY_DATA_DIR:-~/.gemini/antigravity*}` or `~/.config/antigravity`
181181
- Grok Build CLI: `${GROK_HOME:-~/.grok}`
182182
- ZCode: `${ZCODE_HOME:-~/.zcode}/cli/db/db.sqlite`
@@ -199,13 +199,14 @@ export OPENCLAW_DIR="/path/to/openclaw"
199199
export KILO_DATA_DIR="/path/to/kilo"
200200
export KIMI_DATA_DIR="/path/to/kimi"
201201
export QWEN_DATA_DIR="/path/to/qwen"
202+
export COPILOT_HOME="/path/to/copilot"
202203
export ANTIGRAVITY_DATA_DIR="/path/to/antigravity"
203204
export COPILOT_OTEL_FILE_EXPORTER_PATH="/path/to/copilot-otel.jsonl"
204205
export GROK_HOME="/path/to/grok-home"
205206
export ZCODE_HOME="/path/to/zcode-home"
206207
```
207208

208-
Directory variables can contain comma-separated directories. `COPILOT_OTEL_FILE_EXPORTER_PATH` points to one JSONL file, `GROK_HOME` accepts one root, and `ZCODE_HOME` supports multiple roots and deduplicates them:
209+
Directory variables can contain comma-separated directories, except `COPILOT_HOME` and `GROK_HOME`, which take a single root. `COPILOT_OTEL_FILE_EXPORTER_PATH` points to one JSONL file, and `ZCODE_HOME` supports multiple roots and deduplicates them:
209210

210211
```bash
211212
export CODEX_HOME="/path/to/codex,/archive/codex,/path/to/codex-exec-jsonl"

‎docs/guide/index.md‎

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -72,26 +72,26 @@ Each data source page covers the details that only apply to that source, includi
7272

7373
ccusage reads from local coding CLI data directories:
7474

75-
| Agent | ID | Default data location |
76-
| -------------- | ------------- | ---------------------------------------------------------------------------- |
77-
| Claude Code | `claude` | `~/.config/claude/projects/`, `~/.claude/` |
78-
| Codex | `codex` | `${CODEX_HOME:-~/.codex}` |
79-
| OpenCode | `opencode` | `${OPENCODE_DATA_DIR-${XDG_DATA_HOME:-$HOME/.local/share}/opencode}` |
80-
| Amp | `amp` | `${AMP_DATA_DIR:-~/.local/share/amp}` |
81-
| Droid | `droid` | `${DROID_SESSIONS_DIR:-~/.factory/sessions}` |
82-
| Codebuff | `codebuff` | `${CODEBUFF_DATA_DIR:-~/.config/manicode}` |
83-
| Hermes Agent | `hermes` | `${HERMES_HOME:-~/.hermes}/state.db` |
84-
| pi-agent | `pi` | `${PI_AGENT_DIR:-~/.pi/agent/sessions}` |
85-
| Goose | `goose` | Standard Goose data roots or `GOOSE_PATH_ROOT` |
86-
| OpenClaw | `openclaw` | `${OPENCLAW_DIR:-~/.openclaw}` |
87-
| Kilo | `kilo` | `${KILO_DATA_DIR:-~/.local/share/kilo}` |
88-
| Kimi | `kimi` | `${KIMI_DATA_DIR:-~/.kimi}` (also `~/.kimi-code`) |
89-
| Qwen | `qwen` | `${QWEN_DATA_DIR:-~/.qwen}` |
90-
| Copilot CLI | `copilot` | `~/.copilot/otel/*.jsonl` |
91-
| Gemini CLI | `gemini` | `${GEMINI_DATA_DIR:-~/.gemini/tmp}` |
92-
| Antigravity | `antigravity` | `${ANTIGRAVITY_DATA_DIR:-~/.gemini/antigravity*}` or `~/.config/antigravity` |
93-
| Grok Build CLI | `grok` | `${GROK_HOME:-~/.grok}` |
94-
| ZCode | `zcode` | `${ZCODE_HOME:-~/.zcode}` |
75+
| Agent | ID | Default data location |
76+
| -------------- | ------------- | --------------------------------------------------------------------------------------------------------- |
77+
| Claude Code | `claude` | `~/.config/claude/projects/`, `~/.claude/` |
78+
| Codex | `codex` | `${CODEX_HOME:-~/.codex}` |
79+
| OpenCode | `opencode` | `${OPENCODE_DATA_DIR-${XDG_DATA_HOME:-$HOME/.local/share}/opencode}` |
80+
| Amp | `amp` | `${AMP_DATA_DIR:-~/.local/share/amp}` |
81+
| Droid | `droid` | `${DROID_SESSIONS_DIR:-~/.factory/sessions}` |
82+
| Codebuff | `codebuff` | `${CODEBUFF_DATA_DIR:-~/.config/manicode}` |
83+
| Hermes Agent | `hermes` | `${HERMES_HOME:-~/.hermes}/state.db` |
84+
| pi-agent | `pi` | `${PI_AGENT_DIR:-~/.pi/agent/sessions}` |
85+
| Goose | `goose` | Standard Goose data roots or `GOOSE_PATH_ROOT` |
86+
| OpenClaw | `openclaw` | `${OPENCLAW_DIR:-~/.openclaw}` |
87+
| Kilo | `kilo` | `${KILO_DATA_DIR:-~/.local/share/kilo}` |
88+
| Kimi | `kimi` | `${KIMI_DATA_DIR:-~/.kimi}` (also `~/.kimi-code`) |
89+
| Qwen | `qwen` | `${QWEN_DATA_DIR:-~/.qwen}` |
90+
| Copilot CLI | `copilot` | `${COPILOT_HOME:-~/.copilot}/session-state/*/events.jsonl`, `${COPILOT_HOME:-~/.copilot}/otel/**/*.jsonl` |
91+
| Gemini CLI | `gemini` | `${GEMINI_DATA_DIR:-~/.gemini/tmp}` |
92+
| Antigravity | `antigravity` | `${ANTIGRAVITY_DATA_DIR:-~/.gemini/antigravity*}` or `~/.config/antigravity` |
93+
| Grok Build CLI | `grok` | `${GROK_HOME:-~/.grok}` |
94+
| ZCode | `zcode` | `${ZCODE_HOME:-~/.zcode}` |
9595

9696
For OpenCode, `${XDG_DATA_HOME:-$HOME/.local/share}/opencode` is the default-path fallback. When set, `OPENCODE_DATA_DIR` overrides that path; an explicitly empty value disables fallback discovery.
9797

‎rust/adapters/copilot/README.md‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ccusage-adapter-copilot
22

3-
The GitHub Copilot CLI adapter: it turns OpenTelemetry JSONL that the CLI exports to a file
3+
The GitHub Copilot CLI adapter: it turns Copilot session-state and OpenTelemetry JSONL files
44
into the usage entries the reports render.
55

66
## Owns
@@ -15,7 +15,20 @@ Anything that is not specific to this source belongs in `ccusage-core` or
1515

1616
## Data source
1717

18-
- `${COPILOT_OTEL_FILE_EXPORTER_PATH:-~/.copilot/otel}/**/*.jsonl`
18+
- `${COPILOT_HOME:-~/.copilot}/session-state/*/events.jsonl`
19+
- `${COPILOT_HOME:-~/.copilot}/otel/**/*.jsonl`
20+
- `COPILOT_HOME` (single relocated Copilot data root)
21+
- `COPILOT_OTEL_FILE_EXPORTER_PATH` (one explicit JSONL file)
22+
23+
Session-state shutdown records are cumulative per canonical `(session, model)` pair, so only the
24+
latest shutdown is retained. They are preferred for a matching pair when both sources contain it.
25+
Matching OpenTelemetry rows are suppressed only when their timestamps are at or before the latest
26+
canonical shutdown timestamp for that pair; rows emitted after that timestamp by a resumed session
27+
are retained. Other OpenTelemetry records remain available. Session-state `inputTokens` includes cache reads
28+
and writes, so the adapter reports the uncached remainder as input and keeps the cache buckets
29+
separate. Session-state reasoning tokens are already included in output tokens; OpenTelemetry
30+
reasoning is included when total usage metadata shows it is separate. Internal model suffixes such
31+
as `-1m` and `-1m-internal` are removed before pricing and source deduplication.
1932

2033
Reads plain files through `ccusage-adapter-common`, which handles walking, size-balanced
2134
chunking, and ordered parallel reads.

‎rust/adapters/copilot/src/lib.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ pub fn run(args: AgentCommandArgs) -> Result<()> {
5151
}
5252

5353
fn empty_usage_message() -> &'static str {
54-
"No GitHub Copilot CLI usage data found.\nEnable Copilot OpenTelemetry file export before starting or resuming Copilot sessions.\nSee https://ccusage.com/guide/copilot/#data-source"
54+
"No GitHub Copilot CLI usage data found.\nSession-state events are read from ${COPILOT_HOME:-~/.copilot}/session-state/<session-id>/events.jsonl; OpenTelemetry file export remains supported.\nSee https://ccusage.com/guide/copilot/#data-source"
5555
}
5656

5757
#[cfg(test)]

0 commit comments

Comments
 (0)