You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4c2f433
Browse filesBrowse the repository at this point in the historyBrowse files
feat(adapter/copilot): read session-state events.jsonl as default data source
Extend the `copilot` adapter to read `~/.copilot/session-state/<uuid>/events.jsonl`
in addition to the existing OpenTelemetry JSONL exports, fixing the long-standing
'No usage data found' issue (ccusage#1174) for users who have not manually enabled OTel.
The Copilot CLI writes per-session events with full per-model token totals on every
install, so on a fresh system `ccusage copilot daily` now produces real reports
without any environment setup.
Behavior
* New default source: `${COPILOT_CONFIG_DIR:-~/.copilot}/session-state/*/events.jsonl`.
* Existing OTel source unchanged. Both are read in parallel.
* Coexistence: when the same session appears in both sources, shutdown-derived
entries win. Configurable via `COPILOT_OTEL_DEDUP=loose|strict|off` and the
`COPILOT_PREFER_OTEL=1` inversion escape hatch.
* Loader now actually deduplicates by `dedup_key` (HashMap pattern from
`adapter/codebuff/loader.rs:26-33`), required for content-based keys.
* Model normalization applied at pricing call sites only (both
`calculate_cost_for_usage` and `missing_pricing_model_for_usage`); the raw
Copilot model id is preserved on `LoadedEntry.model` for display fidelity.
Empirical correctness fixes
These derive from 18 rounds of multi-model code review (Opus 4.7, Opus 4.8,
GPT-5.5) against the real 200-file local session-state corpus:
* `usage.inputTokens` is inclusive of BOTH `cacheReadTokens` and
`cacheWriteTokens` in this schema (different from OTel) - subtract both to
recover the fresh-input bucket.
* `data.tokenDetails` is NOT used as a source - `usage.*` and
`tokenDetails.*` are independent snapshots that diverge in ~31% of resumed
sessions; using `usage.*` consistently avoids cross-source blending.
* Multiple `session.shutdown` events in a resumed session are per-process
snapshots (verified non-monotonic), so each becomes a distinct entry keyed by
`shutdown:{event.id}:{model}`.
* Provider-aware reasoning rule: `gpt-*` and `claude-*` already include
reasoning inside `outputTokens` (subset convention, 107/107 GPT and 48/48
Claude rows verified); `gemini-*` reports them separately (1/1 verified).
Default for unknown providers is subset (cannot over-count).
* Byte-oriented JSONL reading - real local data contains at least one
invalid-UTF-8 file (a crashed/in-progress write); `fs::read_to_string` would
abort the whole adapter.
* Substring prefilter + top-level `type` re-check - other event types embed
the literal string "session.shutdown" inside their arguments/content.
Tests
* 19 fixture-backed session-state parser tests covering token math, provider
reasoning rules, model preservation, malformed/invalid-UTF-8 handling, skip
rule, and resumed-session semantics.
* 8 loader tests covering all dedup modes, the `COPILOT_PREFER_OTEL`
inversion, pricing-via-normalization for `-1m-internal` models (raw model
preserved + cost > 0 + missing_pricing_model None), and the back-compat
pin that `--mode display` still yields 0 for existing OTel users.
* 3 paths tests for env-var discovery and missing-directory handling.
Docs
In-PR per AGENTS.md cross-cutting flow: rewrites
`docs/guide/copilot/index.md` (data sources, coexistence rule,
troubleshooting) and adds the new env vars to
`docs/guide/environment-variables.md`, `docs/guide/configuration.md`,
`docs/guide/getting-started.md`, and `docs/guide/index.md`.
This commit covers Piece A from the plan; Pieces B (`--mode credits`) and C
(`--mode api-equiv`) follow as separate commits.
Refs ccusage#1174
Co-authored-by: Copilot <[email protected]>
> GitHub Copilot CLI support is experimental. The adapter reads local OpenTelemetry JSONL files only.
3
+
> GitHub Copilot CLI support is experimental.
4
4
5
-
ccusage can read GitHub Copilot CLI OpenTelemetry file exports as one of its supported local data sources. It uses the same reporting experience as the rest of ccusage: responsive tables, JSON output, LiteLLM-based pricing, cache token accounting, and all-source aggregation.
5
+
ccusage reads two complementary local data sources from the Copilot CLI directory: the per-session event stream the CLI writes by default (`session-state/<sessionId>/events.jsonl`), and optional OpenTelemetry JSONL exports (`otel/*.jsonl`). Both are consumed automatically when present, with deduplication between them. Reporting uses the same experience as the rest of ccusage: responsive tables, JSON output, LiteLLM-based pricing, cache token accounting, and all-source aggregation.
The CLI reads Copilot OpenTelemetry JSONL files from `~/.copilot/otel/*.jsonl`and also includes the explicit file pointed to by `COPILOT_OTEL_FILE_EXPORTER_PATH`.
27
+
The Copilot CLI writes per-session events to `~/.copilot/session-state/<sessionId>/events.jsonl`on every install — no configuration is required. This is the primary source ccusage reads. The OpenTelemetry directory and explicit exporter file are only populated when the user has explicitly enabled OTel file export.
28
28
29
-
Enable these variables before starting or resuming a Copilot CLI session. Sessions that ran without OpenTelemetry file export enabled do not produce local JSONL usage data for ccusage to read.
29
+
```text
30
+
~/.copilot/
31
+
├── session-state/ # primary source — written by default
32
+
│ └── <sessionId>/
33
+
│ └── events.jsonl
34
+
└── otel/ # optional fallback / alternative
35
+
└── *.jsonl
36
+
```
37
+
38
+
To override the base directory, set `COPILOT_CONFIG_DIR` to point elsewhere; ccusage will look for `session-state/` and `otel/` underneath it.
39
+
40
+
If you previously relied on the OpenTelemetry-only path described in older versions of this guide, those exports continue to work — both sources are read in parallel and deduplicated. See [Source coexistence](#source-coexistence-otel-session-state) below.
41
+
42
+
### session-state event schema
43
+
44
+
Each `events.jsonl` file contains a stream of JSON records. The adapter reads `session.shutdown` events only; their `data.modelMetrics` map carries per-model token totals (`usage.{inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens, reasoningTokens}`) and request counts. A resumed Copilot session produces multiple shutdown events in the same file — each is a per-process snapshot, summed at report time.
45
+
46
+
### Optional OpenTelemetry export
47
+
48
+
OTel export still works for users who want it. To enable, set the env vars before starting or resuming a Copilot CLI session:
Without these variables, ccusage relies on the session-state source — which is the default for every Copilot CLI install.
43
58
44
59
## Report Views
45
60
@@ -53,26 +68,39 @@ These views support `--json` for structured output, `--compact` for narrow termi
53
68
54
69
## What Gets Calculated
55
70
56
-
-**Token usage** - chat spans are preferred, with inference logs and agent-turn logs used as fallbacks.
57
-
-**Cache tokens** - cache read and cache creation token attributes are counted when present.
58
-
-**Reasoning tokens** - reasoning output tokens are included in total tokens and cost calculation.
59
-
-**Pricing** - costs are calculated from LiteLLM pricing data using the reported model name.
71
+
-**Token usage** — for session-state, `inputTokens` in the source schema already includes both cache reads and cache writes, so ccusage subtracts both to recover the "fresh" input bucket. For OTel, chat spans are preferred with inference logs and agent-turn logs as fallbacks.
72
+
-**Cache tokens** — cache read and cache creation token attributes are counted from `cacheReadTokens` / `cacheWriteTokens` (session-state) or the corresponding `gen_ai.usage.cache_*` attributes (OTel).
73
+
-**Reasoning tokens** — provider-dependent. For OpenAI (`gpt-*`) and Anthropic (`claude-*`) models reasoning is already inside `outputTokens` and is not double-counted. For Google (`gemini-*`) models reasoning is reported separately and is summed into the output cost bucket.
74
+
-**Pricing** — costs are calculated from LiteLLM pricing data using a normalized form of the model name. The raw name (e.g. `claude-opus-4.7-1m-internal`) is preserved in JSON, the table's `Models` column, and `modelsUsed`.
75
+
76
+
## Source coexistence (OTel + session-state)
77
+
78
+
When both `~/.copilot/otel/*.jsonl` and `~/.copilot/session-state/<uuid>/events.jsonl` exist, both are loaded. To avoid double-counting, ccusage drops OTel-derived entries whose resolved `session_id` matches a session-state session. The OTel `session_id` is resolved from the existing parser priority chain (`gen_ai.conversation.id`, `copilot_chat.session_id`, `session.id`, `github.copilot.interaction_id`, `gen_ai.response.id`).
79
+
80
+
> Caveat: whether OTel's `session_id` matches the session-state directory UUID is not yet empirically verified across all Copilot CLI versions. If your IDs do not match, the loose default leaves both sources in place and totals can be inflated. The `COPILOT_OTEL_DEDUP=strict` setting drops OTel wholesale whenever any session-state shutdown is loaded — a conservative anti-double-count if you observe inflated totals.
|`COPILOT_CONFIG_DIR`| Override the base Copilot directory (default: `~/.copilot`). Both `session-state/` and `otel/` are discovered underneath it. |
87
+
|`COPILOT_OTEL_FILE_EXPORTER_PATH`| Add a single explicit OpenTelemetry `.jsonl` file as an additional source (mirrors what the Copilot CLI itself reads when set). |
88
+
|`COPILOT_OTEL_DEDUP`| Coexistence policy when both sources contain entries. `loose` (default) — match by session id. `strict` — drop OTel whenever any session-state shutdown is loaded. `off` — disable the dedup step entirely. |
89
+
|`COPILOT_PREFER_OTEL`| When set (non-empty / non-zero / not `false`), invert the coexistence rule so OTel wins for any session that appears in both sources. |
Ensure OpenTelemetry file export is enabled and the exporter path points to an existing `.jsonl` file, or place exported `.jsonl` files under `~/.copilot/otel/`.
95
+
The Copilot CLI writes per-session events to `~/.copilot/session-state/<sessionId>/events.jsonl` automatically. If `ccusage copilot daily` still reports no data, verify that the directory exists and that at least one of the per-session subdirectories contains an `events.jsonl` with a `session.shutdown` event (resumed sessions only produce shutdown rows on graceful exit).
96
+
97
+
If you have configured `COPILOT_CONFIG_DIR`, make sure it points at the directory that contains `session-state/`. As a fallback, OpenTelemetry file export still works — see the env-var block above.
98
+
:::
72
99
73
-
If you are using `copilot --resume`, set the OpenTelemetry environment variables before running the resume command. Earlier activity from sessions started without file export cannot be recovered by ccusage.
100
+
::: details Inflated totals when OTel export is also enabled
101
+
If you have both `~/.copilot/otel/*.jsonl` files and `~/.copilot/session-state/<uuid>/events.jsonl` files for the same Copilot sessions, the same usage may be counted twice. Set `COPILOT_OTEL_DEDUP=strict` to drop the OTel entries whenever a session-state shutdown is present, or unset `COPILOT_OTEL_FILE_EXPORTER_PATH` and remove the `otel/` directory to skip the OTel path entirely.
74
102
:::
75
103
76
104
::: details Costs showing as $0.00
77
-
If a model is not in LiteLLM's database, the cost will be $0.00. [Open an issue](https://github.com/ryoppippi/ccusage/issues/new) to request alias support.
105
+
If a model is not in LiteLLM's database — including newer Copilot model variants — the cost will be $0.00. [Open an issue](https://github.com/ryoppippi/ccusage/issues/new) to request alias support.
- GitHub Copilot CLI: `${COPILOT_CONFIG_DIR:-~/.copilot}/session-state/<sessionId>/events.jsonl` (default; written by the CLI on every install) or `${COPILOT_CONFIG_DIR:-~/.copilot}/otel/*.jsonl`/`COPILOT_OTEL_FILE_EXPORTER_PATH` for OpenTelemetry exports
0 commit comments