Skip to content

Commit f9e6386

Browse files
jay-tauCopilot
andcommitted
refactor(adapter/copilot): remove legacy OTel parser and its tests
The production loader has been reading session-state events.jsonl exclusively since the start of this PR series. The OTel parser had been carried under #[allow(dead_code)] pending ccusage#1208, but ccusage#1208 was auto-closed by the contribution gate, so folding the removal in here avoids leaving an orphaned dead-code parser with no tracked owner. Removed ------- * parse_otel_file and ~500 lines of OTel-specific helpers (CopilotUsageSource, CopilotUsageCandidate, trace context collection, span/log classification, OTel attribute readers, span-id/trace-id extraction, OTel timestamp parsers). * 4 OTel parser tests in loader.rs::tests (parses_copilot_chat_spans, suppresses_lower_priority_records_*, includes_reasoning_tokens_in_total_tokens, falls_back_to_total_tokens_when_copilot_parts_are_missing). * Now-unused imports: HashSet, serde_json::Map, apply_total_token_fallback, TokenUsageRaw (parser.rs); fs_fixture, parse_otel_file, AgentReportKind, report_from_rows, summarize_entries (loader.rs). * OTel-mentioning doc fragments in nano_aiu, premium_request_cost, the loader's mode-dispatch comment, the OTel-coexistence-tests placeholder, and the per-file-isolation test header. * The "Legacy OpenTelemetry parser" README section. Retained (intentionally) ------------------------ * file_modified_timestamp helper (moved into the session-state section; used by parse_session_state_file's fallback timestamp). * The OTel env-var deprecation warning surface (warn_about_inert_legacy_env_vars_once, LEGACY_OTEL_ENV_VARS, legacy_otel_env_vars_in_use) and its tests. This is the user-facing notice on first run after upgrade; users who had COPILOT_OTEL_FILE_EXPORTER_PATH / COPILOT_OTEL_DEDUP / COPILOT_PREFER_OTEL set still need to learn those are inert. * The otel/ directory ignore tests (paths.rs) and the README section documenting the env-var deprecation. Validation ---------- * cargo fmt --check clean. * cargo clippy --workspace --all-targets -- -D warnings clean. * cargo test --workspace: 364 passed, 0 failed (was 368: -4 OTel parser tests). All session-state tests, all aggregation tests, and the env-var deprecation tests continue to pass. Diff: 4 files changed, +30 / -711 lines. Co-authored-by: Copilot <[email protected]>
1 parent 5a62fbc commit f9e6386

4 files changed

Lines changed: 30 additions & 712 deletions

File tree

‎rust/crates/ccusage/src/adapter/copilot/README.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -237,11 +237,11 @@ the collapse possible.
237237
| -------------------- | --------------------------------------------------------------- |
238238
| `COPILOT_CONFIG_DIR` | Override the base Copilot directory (defaults to `~/.copilot`). |
239239

240-
## Legacy OpenTelemetry parser
241-
242-
`parser.rs` retains `parse_otel_file` and its helpers under
243-
`#[allow(dead_code)]`, kept solely so existing parser unit tests continue to
244-
verify the OTel chat-span / inference-log / agent-turn dispatch. The
245-
production loader no longer reads OTel data — the full OTel surface (parser
246-
code, env vars, coexistence handling, dedup policy) is scheduled for removal
247-
in [#1208](https://github.com/ryoppippi/ccusage/issues/1208).
240+
## Legacy OpenTelemetry environment variables
241+
242+
ccusage no longer reads OpenTelemetry exports — the production loader uses
243+
`~/.copilot/session-state/<uuid>/events.jsonl` directly. The legacy OTel
244+
environment variables (`COPILOT_OTEL_FILE_EXPORTER_PATH`, `COPILOT_OTEL_DEDUP`,
245+
`COPILOT_PREFER_OTEL`) are inert; if any of them are set when `ccusage copilot`
246+
runs, a one-shot stderr warning explains they are ignored. Unset them, or set
247+
`LOG_LEVEL=0`, to silence the warning.

‎rust/crates/ccusage/src/adapter/copilot/loader.rs‎

Lines changed: 15 additions & 217 deletions
Original file line numberDiff line numberDiff line change
@@ -62,20 +62,14 @@ fn load_entries_inner(
6262
) -> Result<Vec<LoadedEntry>> {
6363
let tz = parse_tz(shared.timezone.as_deref());
6464

65-
// session-state is the only Copilot data source the production loader
66-
// reads. The OTel parser (`parse_otel_file`) still exists in
67-
// `parser.rs` with its own tests, but nothing in production calls it —
68-
// pending full removal in #1208.
69-
//
7065
// Per-file error isolation: one unreadable/corrupted session-state
71-
// file must NOT abort the whole report. The legacy OTel reader and
72-
// every other ccusage adapter treat parse errors as per-file skips
73-
// (with a warning to stderr when logging is enabled); now that
74-
// session-state is the always-on default path, we adopt the same
75-
// semantics here. A locked file (Windows, antivirus scanning), a
76-
// partial write from a still-running CLI, or one file with malformed
77-
// JSON shouldn't make `ccusage copilot daily` print zero — it
78-
// should report whatever the other sessions show.
66+
// file must NOT abort the whole report. Other ccusage adapters treat
67+
// parse errors as per-file skips (with a warning to stderr when
68+
// logging is enabled); we adopt the same semantics here. A locked
69+
// file (Windows, antivirus scanning), a partial write from a
70+
// still-running CLI, or one file with malformed JSON shouldn't make
71+
// `ccusage copilot daily` print zero — it should report whatever the
72+
// other sessions show.
7973
let mut entries: Vec<CopilotUsageEntry> = Vec::new();
8074
for path in session_state_paths()? {
8175
match parse_session_state_file(&path) {
@@ -243,8 +237,8 @@ fn usage_entry_to_loaded(
243237
// sonnet/haiku rows whose `requests.cost == 0` — Auto/Display bill
244238
// those as a genuine `$0` against the premium-request plan, not as
245239
// missing data. The rightmost Auto cell's token-priced fallback only
246-
// fires when the source ships NEITHER billing field (historical OTel
247-
// or a future Copilot CLI variant).
240+
// fires when the source ships NEITHER billing field (a future Copilot
241+
// CLI variant that drops both fields).
248242
//
249243
// Synthetic aggregate-credit rows (empty `entry.model`, emitted by
250244
// `parser.rs::parse_session_state_file` when `data.totalNanoAiu > 0`
@@ -340,198 +334,11 @@ fn usage_entry_to_loaded(
340334
}
341335
}
342336

343-
#[cfg(test)]
344-
use super::report::{report_from_rows, summarize_entries};
345-
346337
#[cfg(test)]
347338
mod tests {
348-
use ccusage_test_support::fs_fixture;
349339
use serde_json::json;
350340

351-
use super::super::parser::parse_otel_file;
352341
use super::*;
353-
use crate::cli::AgentReportKind;
354-
355-
#[test]
356-
fn parses_copilot_chat_spans() {
357-
let fixture = fs_fixture!({
358-
"copilot.jsonl": [
359-
json!({ "type": "metric", "name": "gen_ai.client.token.usage" }).to_string(),
360-
json!({
361-
"type": "span",
362-
"traceId": "trace-1",
363-
"spanId": "span-1",
364-
"name": "chat claude-sonnet-4",
365-
"endTime": [1_775_934_264_u64, 967_317_833_u64],
366-
"attributes": {
367-
"gen_ai.operation.name": "chat",
368-
"gen_ai.request.model": "claude-sonnet-4",
369-
"gen_ai.response.model": "claude-sonnet-4",
370-
"gen_ai.conversation.id": "conv-1",
371-
"gen_ai.usage.input_tokens": 19_452,
372-
"gen_ai.usage.output_tokens": 281,
373-
"gen_ai.usage.cache_read.input_tokens": 123,
374-
"gen_ai.usage.cache_creation.input_tokens": 25,
375-
"gen_ai.usage.reasoning.output_tokens": 128,
376-
},
377-
})
378-
.to_string(),
379-
]
380-
.join("\n"),
381-
});
382-
let file = fixture.path("copilot.jsonl");
383-
384-
let entries = parse_otel_file(&file).unwrap();
385-
386-
assert_eq!(entries.len(), 1);
387-
assert_eq!(entries[0].timestamp_text, "2026-04-11T19:04:24.967Z");
388-
assert_eq!(entries[0].session_id, "conv-1");
389-
assert_eq!(entries[0].model, "claude-sonnet-4");
390-
assert_eq!(entries[0].input_tokens, 19_329);
391-
assert_eq!(entries[0].output_tokens, 281);
392-
assert_eq!(entries[0].cache_creation_tokens, 25);
393-
assert_eq!(entries[0].cache_read_tokens, 123);
394-
assert_eq!(entries[0].reasoning_output_tokens, 128);
395-
assert_eq!(entries[0].dedup_key, "trace-1:span-1");
396-
}
397-
398-
#[test]
399-
fn suppresses_lower_priority_records_for_same_response() {
400-
let fixture = fs_fixture!({
401-
"copilot.jsonl": [
402-
json!({
403-
"type": "span",
404-
"traceId": "trace-dupe",
405-
"spanId": "agent-1",
406-
"name": "invoke_agent GitHub Copilot Chat",
407-
"attributes": {
408-
"gen_ai.operation.name": "invoke_agent",
409-
"gen_ai.response.model": "gpt-5.4-mini",
410-
"gen_ai.conversation.id": "conv-dupe",
411-
"gen_ai.response.id": "resp-dupe",
412-
"gen_ai.usage.input_tokens": 100,
413-
"gen_ai.usage.output_tokens": 30,
414-
},
415-
})
416-
.to_string(),
417-
json!({
418-
"hrTime": [1_775_934_263_u64, 0_u64],
419-
"attributes": {
420-
"event.name": "gen_ai.client.inference.operation.details",
421-
"gen_ai.response.model": "gpt-5.4-mini",
422-
"gen_ai.response.id": "resp-dupe",
423-
"gen_ai.usage.input_tokens": 80,
424-
"gen_ai.usage.output_tokens": 20,
425-
},
426-
"_body": "GenAI inference: gpt-5.4-mini",
427-
})
428-
.to_string(),
429-
json!({
430-
"type": "span",
431-
"traceId": "trace-dupe",
432-
"spanId": "chat-1",
433-
"name": "chat gpt-5.4-mini",
434-
"attributes": {
435-
"gen_ai.operation.name": "chat",
436-
"gen_ai.response.model": "gpt-5.4-mini",
437-
"gen_ai.conversation.id": "conv-dupe",
438-
"gen_ai.response.id": "resp-dupe",
439-
"gen_ai.usage.input_tokens": 60,
440-
"gen_ai.usage.output_tokens": 10,
441-
},
442-
})
443-
.to_string(),
444-
]
445-
.join("\n"),
446-
});
447-
let file = fixture.path("copilot.jsonl");
448-
449-
let entries = parse_otel_file(&file).unwrap();
450-
451-
assert_eq!(entries.len(), 1);
452-
assert_eq!(entries[0].dedup_key, "trace-dupe:chat-1");
453-
assert_eq!(entries[0].input_tokens, 60);
454-
assert_eq!(entries[0].output_tokens, 10);
455-
}
456-
457-
#[test]
458-
fn includes_reasoning_tokens_in_total_tokens() {
459-
let fixture = fs_fixture!({
460-
"copilot.jsonl":
461-
format!(
462-
"{}\n",
463-
json!({
464-
"type": "span",
465-
"traceId": "trace-1",
466-
"spanId": "span-1",
467-
"name": "chat test-model",
468-
"endTime": [1_775_934_264_u64, 0_u64],
469-
"attributes": {
470-
"gen_ai.operation.name": "chat",
471-
"gen_ai.response.model": "test-model",
472-
"gen_ai.conversation.id": "conv-1",
473-
"gen_ai.usage.input_tokens": 100,
474-
"gen_ai.usage.output_tokens": 50,
475-
"gen_ai.usage.cache_read.input_tokens": 10,
476-
"gen_ai.usage.cache_creation.input_tokens": 20,
477-
"gen_ai.usage.reasoning.output_tokens": 5,
478-
},
479-
})
480-
),
481-
});
482-
let file = fixture.path("copilot.jsonl");
483-
let mut pricing = crate::PricingMap::default();
484-
pricing.load_json(
485-
r#"{"test-model":{"input_cost_per_token":1,"output_cost_per_token":2,"cache_creation_input_token_cost":3,"cache_read_input_token_cost":4}}"#,
486-
);
487-
488-
let loaded: Vec<LoadedEntry> = parse_otel_file(&file)
489-
.unwrap()
490-
.into_iter()
491-
.map(|entry| usage_entry_to_loaded(entry, None, CostMode::Auto, &pricing))
492-
.collect();
493-
let rows = summarize_entries(&loaded, AgentReportKind::Daily).unwrap();
494-
let report = report_from_rows(&rows, AgentReportKind::Daily);
495-
496-
assert_eq!(report["daily"][0]["inputTokens"], 90);
497-
assert_eq!(report["daily"][0]["outputTokens"], 50);
498-
assert_eq!(report["daily"][0]["totalTokens"], 175);
499-
assert_eq!(report["daily"][0]["totalCost"], 300.0);
500-
}
501-
502-
#[test]
503-
fn falls_back_to_total_tokens_when_copilot_parts_are_missing() {
504-
let fixture = fs_fixture!({
505-
"copilot.jsonl":
506-
format!(
507-
"{}\n",
508-
json!({
509-
"type": "span",
510-
"traceId": "trace-1",
511-
"spanId": "span-1",
512-
"name": "chat test-model",
513-
"endTime": [1_775_934_264_u64, 0_u64],
514-
"attributes": {
515-
"gen_ai.operation.name": "chat",
516-
"gen_ai.response.model": "test-model",
517-
"gen_ai.conversation.id": "conv-1",
518-
"gen_ai.usage.total_tokens": 567,
519-
},
520-
})
521-
),
522-
});
523-
let file = fixture.path("copilot.jsonl");
524-
525-
let entries = parse_otel_file(&file).unwrap();
526-
527-
assert_eq!(entries.len(), 1);
528-
assert_eq!(entries[0].output_tokens, 567);
529-
assert_eq!(entries[0].reasoning_output_tokens, 0);
530-
}
531-
532-
// ---------------------------------------------------------------
533-
// Session-state loader integration tests
534-
// ---------------------------------------------------------------
535342

536343
use ccusage_test_support::EnvScope;
537344

@@ -592,13 +399,6 @@ mod tests {
592399
fixture
593400
}
594401

595-
// OTel↔session-state coexistence tests previously lived here. The OTel
596-
// data source is no longer read by the production loader (see
597-
// `session_state_paths()` in `paths.rs`), so the dedup/prefer-OTel/etc.
598-
// tests have been removed alongside the env vars they exercised. The
599-
// OTel parser code is kept in `parser.rs` (with `#[allow(dead_code)]`)
600-
// pending full removal in #1208.
601-
602402
#[test]
603403
fn loader_dedup_keeps_distinct_sessions_with_colliding_event_ids() {
604404
// Defense-in-depth: the parser's token-bearing dedup key is
@@ -1643,14 +1443,12 @@ mod tests {
16431443
#[cfg(unix)]
16441444
fn one_unreadable_session_file_does_not_abort_whole_report() {
16451445
// Per-file error isolation: one bad/locked/corrupted session-state
1646-
// file MUST NOT abort the whole report. The legacy OTel reader
1647-
// and every other ccusage adapter treat parse errors as per-file
1648-
// skips; now that session-state is the always-on default, the
1649-
// same semantic must hold here. A locked file (Windows,
1650-
// antivirus scanning), a partial write from a still-running CLI,
1651-
// or one file with permission errors shouldn't make
1652-
// `ccusage copilot daily` print zero — it should report whatever
1653-
// the OTHER sessions show.
1446+
// file MUST NOT abort the whole report. Other ccusage adapters
1447+
// treat parse errors as per-file skips; the same semantic must
1448+
// hold here. A locked file (Windows, antivirus scanning), a
1449+
// partial write from a still-running CLI, or one file with
1450+
// permission errors shouldn't make `ccusage copilot daily` print
1451+
// zero — it should report whatever the OTHER sessions show.
16541452
//
16551453
// Hermetic: build a fixture with TWO session directories. The
16561454
// first is a real, readable shutdown event ($0.04 under Auto).

0 commit comments

Comments
 (0)