|
| 1 | +# GitHub Copilot CLI Source |
| 2 | + |
| 3 | +Data source: |
| 4 | + |
| 5 | +```text |
| 6 | +${COPILOT_CONFIG_DIR:-~/.copilot}/session-state/<sessionId>/events.jsonl |
| 7 | +``` |
| 8 | + |
| 9 | +The Copilot CLI writes `session-state/<uuid>/events.jsonl` automatically during each CLI session, appending a `session.shutdown` event on graceful exit. |
| 10 | + |
| 11 | +## session-state schema |
| 12 | + |
| 13 | +Relevant JSONL event: |
| 14 | + |
| 15 | +- `type === "session.shutdown"` (other event types — `tool.execution_start`, |
| 16 | + `assistant.message`, etc. — may contain the literal string |
| 17 | + `"session.shutdown"` inside `arguments.command` or `content`; the parser |
| 18 | + requires the top-level `type` field after JSON parsing, never trusts the |
| 19 | + substring alone). |
| 20 | +- `id` — per-event UUID, used as part of the dedup key |
| 21 | + (`shutdown:{session_id}:{event.id}:{model}` for token-bearing rows; |
| 22 | + `session_id` is prefixed as defense-in-depth against a future Copilot |
| 23 | + CLI that might scope event ids per-session). |
| 24 | +- `timestamp` — RFC3339 with ms (`"2026-05-02T15:27:09.013Z"`). |
| 25 | +- `data.modelMetrics` — map keyed by Copilot model id; each value carries |
| 26 | + `usage.{inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens, |
| 27 | + reasoningTokens}` and `requests.{count, cost}`. |
| 28 | + |
| 29 | +### Token mapping (session-state) |
| 30 | + |
| 31 | +`usage.inputTokens` in this schema is inclusive of **both** `cacheReadTokens` |
| 32 | +and `cacheWriteTokens`. |
| 33 | + |
| 34 | +| Field on `LoadedEntry` | Source | |
| 35 | +| -------------------------------- | --------------------------------------------------------------------------- | |
| 36 | +| `input_tokens` | `usage.inputTokens − usage.cacheReadTokens − usage.cacheWriteTokens` (saturating) | |
| 37 | +| `output_tokens` | `usage.outputTokens` | |
| 38 | +| `cache_read_tokens` | `usage.cacheReadTokens` | |
| 39 | +| `cache_creation_input_tokens` | `usage.cacheWriteTokens` | |
| 40 | +| `reasoning_output_tokens` | provider-dependent — see below | |
| 41 | + |
| 42 | +`data.tokenDetails` exists in the schema but is **intentionally not used as a |
| 43 | +source**: empirical verification showed `usage.*` and `tokenDetails.*` are |
| 44 | +independent snapshots that diverge in ~31% of resumed-session rows. We use |
| 45 | +`usage.*` consistently across all four token fields. |
| 46 | + |
| 47 | +### Reasoning tokens |
| 48 | + |
| 49 | +Provider conventions differ. Verified locally across the full session-state |
| 50 | +corpus: |
| 51 | + |
| 52 | +| Provider | Convention | Rule | |
| 53 | +| ------------------------- | ------------------------------------------------------- | --------------------------------- | |
| 54 | +| `claude-*` (Anthropic) | Extended-thinking tokens included in `outputTokens` | `reasoning_output_tokens = 0` | |
| 55 | +| `gpt-*` (OpenAI) | `reasoning_tokens` is a subset of `output_tokens` | `reasoning_output_tokens = 0` | |
| 56 | +| `gemini-*` (Google) | `thoughtsTokenCount` is a separate field from output | `reasoning_output_tokens = usage.reasoningTokens` | |
| 57 | +| Anything else (default) | Treated like Claude/GPT — the subset rule cannot over-count | `reasoning_output_tokens = 0` | |
| 58 | + |
| 59 | +### Skip rule |
| 60 | + |
| 61 | +A `modelMetrics` row is skipped when **all** of the following are true: |
| 62 | + |
| 63 | +1. All non-reasoning token fields are zero (`input`, `output`, `cache_read`, |
| 64 | + `cache_write`), AND for Gemini models `reasoning_tokens` is also zero. |
| 65 | +2. `requests.count` is zero. |
| 66 | +3. `metrics.totalNanoAiu` is absent or zero. |
| 67 | + |
| 68 | +Matches PR #957's `data-loader.ts` inline skip-row predicate, extended with the AIU |
| 69 | +carve-out so that real-data credit-only rows (zero tokens, zero requests, |
| 70 | +non-zero `totalNanoAiu`) survive. The reasoning-token clause is |
| 71 | +provider-gated to stay symmetric with `build_session_state_entry`: for |
| 72 | +Gemini the builder preserves `reasoning_tokens` verbatim (separate field |
| 73 | +from output), so the skip rule counts it as usage; for OpenAI/Anthropic |
| 74 | +the builder discards it (already subsumed into `output_tokens`), so the |
| 75 | +skip rule ignores it to avoid surfacing all-zero phantom rows. See |
| 76 | +`keeps_zero_token_row_when_per_model_total_nano_aiu_is_present`, |
| 77 | +`keeps_gemini_row_with_only_reasoning_tokens`, and |
| 78 | +`skips_non_gemini_row_with_only_reasoning_tokens` for the matching |
| 79 | +regressions. |
| 80 | + |
| 81 | +### Resumed sessions and dedup-key design |
| 82 | + |
| 83 | +A single `events.jsonl` may contain multiple `session.shutdown` events (the user |
| 84 | +re-opened the same session UUID multiple times). Each shutdown is a |
| 85 | +**per-process snapshot** — token counts are NOT cumulative across resumes. The |
| 86 | +parser emits one entry per `(session_id, event.id, model)` triple so the |
| 87 | +resumed-session totals are summed correctly at report time. |
| 88 | + |
| 89 | +The dedup-key selector chooses between two key shapes for each surviving row, |
| 90 | +and the two shapes deliberately encode **opposite beliefs about what a duplicate |
| 91 | +shutdown means** — because real-data shapes differ between the two paths: |
| 92 | + |
| 93 | +- **Token-bearing / priced rows** key on `shutdown:{session_id}:{event_id}:{model}`. |
| 94 | + Two shutdowns with distinct event ids are treated as distinct charges (sum at |
| 95 | + report time), even if their token counts happen to coincide. This is correct |
| 96 | + because per-process snapshots commonly produce near-identical counts when a |
| 97 | + user resumes a similar workload, and treating them as duplicates would |
| 98 | + silently under-count usage. The `session_id` prefix is defense-in-depth: |
| 99 | + today every observed `event.id` is a globally-unique UUID, but prefixing |
| 100 | + with `session_id` guards against a hypothetical future Copilot CLI that |
| 101 | + scopes event ids per-session, which would otherwise silently collapse |
| 102 | + distinct sessions' rows in the loader's final `HashMap`-keyed dedup. Pinned |
| 103 | + by `token_bearing_rows_keep_event_id_dedup_key_not_content_based` and |
| 104 | + `treats_resumed_session_shutdowns_as_distinct_entries`. |
| 105 | +- **Credit-only zero-token rows** key on |
| 106 | + `credit-shutdown:{session_id}:{model}:{naiu}` — **deliberately omitting |
| 107 | + `event_id`**. Real session-state data ships credit-only rows in |
| 108 | + duplicate pairs with distinct event ids but identical `totalNanoAiu` |
| 109 | + values (observed pattern in `1135875b-…` and `76dc438a-…`; the local |
| 110 | + 200-file corpus shows roughly $2.65 of credits would be double-counted |
| 111 | + by event-id-based keying). Two such rows are treated as duplicates of |
| 112 | + the same snapshot, not distinct charges. Pinned by |
| 113 | + `auto_mode_dedupes_duplicate_credit_only_rows_within_same_session` and |
| 114 | + `credit_only_rows_with_identical_naiu_but_distinct_event_ids_are_treated_as_duplicates`. |
| 115 | + |
| 116 | +**The asymmetry is intentional.** Token rows trust event ids because their |
| 117 | +data shape (per-process snapshots) makes coincidence-collision a far more |
| 118 | +likely failure mode than missed-duplicate. Credit-only rows distrust event |
| 119 | +ids because their data shape (paired snapshots) makes the opposite true. |
| 120 | +Both choices accept a small known under/over-count risk in the unlikely |
| 121 | +forward-compat case (token counts coincidentally identical across resumes; |
| 122 | +or distinct genuine credit charges that happen to share the same naiu) in |
| 123 | +exchange for correctness on the observed real-data shape. |
| 124 | + |
| 125 | +The aggregate-credit synthetic entry (emitted only when no per-model row |
| 126 | +carries a priced billing signal — see `parse_session_state_file`'s |
| 127 | +suppression guard) uses the same content-based key family |
| 128 | +(`credit-aggregate:{session_id}:{nano_aiu}`) to mirror the credit-only |
| 129 | +"paired snapshot" semantics. |
| 130 | + |
| 131 | +If a future Copilot CLI ships genuine non-coincidental duplicate event ids |
| 132 | +on credit-only rows, the over-suppression would surface as a visible |
| 133 | +under-bill in `credits` JSON output (which is always populated regardless |
| 134 | +of mode) — a soft signal that surfaces the schema change rather than |
| 135 | +silently doubling the bill. |
| 136 | + |
| 137 | +### Date filter must run before dedup |
| 138 | + |
| 139 | +The credit-only key shape above (which deliberately omits `event_id`) makes |
| 140 | +the loader's `HashMap`-based dedup collapse **last-wins by parse order**: |
| 141 | +two credit-only rows in the same session with the same `(model, naiu)` but |
| 142 | +distinct timestamps collapse into whichever was inserted last. That's the |
| 143 | +intended behaviour for paired snapshots inside an active date range — but |
| 144 | +if the two rows straddle a `--since` / `--until` boundary, applying the |
| 145 | +date filter **after** the dedup HashMap could keep the out-of-range row, |
| 146 | +drop it on the filter, and silently lose the in-range row's credits |
| 147 | +entirely. |
| 148 | + |
| 149 | +The loader therefore applies the `--since` / `--until` filter to the |
| 150 | +`CopilotUsageEntry` vector **before** the dedup `HashMap::insert` collapse. |
| 151 | +Each surviving entry already satisfies the date range, so the collapse can |
| 152 | +never pick an out-of-range winner. Token-bearing rows (event-id keyed) are |
| 153 | +immune to this ordering bug, but the filter applies uniformly to keep the |
| 154 | +loader-layer surface symmetric across both key families. The downstream |
| 155 | +`filter_loaded_entries_by_date` calls in `copilot::run` and the cross-source |
| 156 | +aggregator (`adapter::all::loader`) remain in place as defense-in-depth |
| 157 | +no-ops once the loader has filtered, and to keep the user-facing |
| 158 | +`LoadedEntry` surface uniform with the other adapters. Pinned by |
| 159 | +`date_filter_runs_before_credit_only_dedup_collapse_to_preserve_in_range_row`. |
| 160 | + |
| 161 | +## Model normalization |
| 162 | + |
| 163 | +The adapter ships `normalize_copilot_model()` for pricing-lookup callers. The |
| 164 | +function is applied **only inside `loader.rs::usage_entry_to_loaded`** for both |
| 165 | +`calculate_cost_for_usage` and `missing_pricing_model_for_usage` call sites — |
| 166 | +the entry itself keeps the raw model name so reports, JSON and `modelsUsed` |
| 167 | +preserve source fidelity. |
| 168 | + |
| 169 | +Rules: |
| 170 | + |
| 171 | +1. Strip Copilot routing suffixes in this order: `-1m-internal`, `-internal`, |
| 172 | + `-xhigh`, `-high`, `-1m`. Required because |
| 173 | + `pricing::suffix_starts_with_numeric_model_version` rejects suffixes that |
| 174 | + begin with another digit, so without stripping the `-1m` family of variants |
| 175 | + no pricing entry resolves. |
| 176 | +2. For `claude-*` names, convert version-number dots to dashes (e.g. |
| 177 | + `claude-opus-4.7` → `claude-opus-4-7`). Matches the canonical LiteLLM keys. |
| 178 | +3. GPT names pass through unchanged — LiteLLM keeps them dotted. |
| 179 | +4. Unknown prefixes pass through unchanged. |
| 180 | + |
| 181 | +## Cost modes |
| 182 | + |
| 183 | +| `--mode` | Behavior for Copilot | |
| 184 | +| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| 185 | +| `auto` | True bill, billing-field-aware: AIU when present (typically post-cutover) → premium-requests × $0.04 when present (typically pre-cutover) → token-priced via LiteLLM (neither billing field). The default. Free-tier rows (`requests.cost == 0`) bill as $0. | |
| 186 | +| `display`| Source-precomputed only: AIU → premium-requests → $0. No token-priced fallback. | |
| 187 | +| `calculate` / `api` | Always token-priced via LiteLLM (`normalize_copilot_model()` applied at both `calculate_cost_for_usage` and `missing_pricing_model_for_usage` call sites). | |
| 188 | + |
| 189 | +The billing-field-aware default reflects that GitHub Copilot switched billing |
| 190 | +models on June 1, 2026: pre-cutover sessions were billed in "premium requests" |
| 191 | +with per-model multipliers at $0.04/overage-request, and post-cutover sessions |
| 192 | +bill in AI Units (1 AIU = 1 credit = $0.01). The Copilot CLI records each |
| 193 | +session's own billing data, so `auto` surfaces what GitHub actually charged |
| 194 | +you for that session. |
| 195 | + |
| 196 | +**The dispatch keys off field presence, not the cutover date.** The |
| 197 | +loader never compares `entry.timestamp` to June 1, 2026 — it just asks |
| 198 | +"did this row ship `totalNanoAiu`?" and picks AIU billing if so. The cutover |
| 199 | +date is the *why* behind the two billing columns existing in the wire |
| 200 | +schema; it is not a branching condition in the code. This shape is more |
| 201 | +robust than a date check: it correctly handles a pre-cutover session that |
| 202 | +happens to ship AIU (rare but possible if GitHub ever backfills) and |
| 203 | +generalises to a hypothetical future third billing channel without a |
| 204 | +schema-version migration. |
| 205 | + |
| 206 | +`PREMIUM_REQUEST_COST_USD = $0.04` and `AI_CREDIT_COST_USD = $0.01` are |
| 207 | +GitHub's published external pricing rates, not contracts encoded in the |
| 208 | +CLI's wire format. If GitHub adjusts either rate, `auto`/`display` will |
| 209 | +mis-bill until the constants are updated. The mitigation users have today |
| 210 | +is `--mode api` (alias of `calculate`): that path bills from token counts |
| 211 | +via LiteLLM and does not depend on either constant. |
| 212 | + |
| 213 | +`LoadedEntry.credits` is populated independently of `--mode` whenever the |
| 214 | +source carried `totalNanoAiu`, so JSON consumers see the raw credit count |
| 215 | +regardless of mode selection. |
| 216 | + |
| 217 | +The `missing_pricing_model` warning never fires for the AIU and premium-request |
| 218 | +branches of `auto`/`display` because they don't consult LiteLLM — those |
| 219 | +branches return `None` directly. The shared |
| 220 | +`cost.rs::missing_pricing_model_for_usage` early-return remains intact so other |
| 221 | +adapters still see legitimate missing-pricing warnings when their token-pricing |
| 222 | +fallback can't resolve a model. |
| 223 | + |
| 224 | +## Environment variables |
| 225 | + |
| 226 | +| Variable | Effect | |
| 227 | +| -------------------- | --------------------------------------------------------------- | |
| 228 | +| `COPILOT_CONFIG_DIR` | Override the base Copilot directory (defaults to `~/.copilot`). | |
| 229 | + |
| 230 | +## Legacy OpenTelemetry environment variables |
| 231 | + |
| 232 | +ccusage no longer reads OpenTelemetry exports — the production loader uses |
| 233 | +`~/.copilot/session-state/<uuid>/events.jsonl` directly. The legacy OTel |
| 234 | +environment variables (`COPILOT_OTEL_FILE_EXPORTER_PATH`, `COPILOT_OTEL_DEDUP`, |
| 235 | +`COPILOT_PREFER_OTEL`) are inert; if any of them are set when `ccusage copilot` |
| 236 | +runs, a one-shot stderr warning explains they are ignored. Unset them, or set |
| 237 | +`LOG_LEVEL=0`, to silence the warning. |
0 commit comments