Skip to content

Commit 1232b97

Browse files
jay-tauCopilot
andcommitted
refactor(adapter/copilot): remove legacy OTel parser and its tests
The production loader has been reading session-state events.jsonl exclusively since the start of this PR series. The OTel parser had been carried under #[allow(dead_code)] pending ccusage#1208, but ccusage#1208 was auto-closed by the contribution gate, so folding the removal in here avoids leaving an orphaned dead-code parser with no tracked owner. Removed ------- * parse_otel_file and ~500 lines of OTel-specific helpers (CopilotUsageSource, CopilotUsageCandidate, trace context collection, span/log classification, OTel attribute readers, span-id/trace-id extraction, OTel timestamp parsers). * 4 OTel parser tests in loader.rs::tests (parses_copilot_chat_spans, suppresses_lower_priority_records_*, includes_reasoning_tokens_in_total_tokens, falls_back_to_total_tokens_when_copilot_parts_are_missing). * Now-unused imports: HashSet, serde_json::Map, apply_total_token_fallback, TokenUsageRaw (parser.rs); fs_fixture, parse_otel_file, AgentReportKind, report_from_rows, summarize_entries (loader.rs). * OTel-mentioning doc fragments in nano_aiu, premium_request_cost, the loader's mode-dispatch comment, the OTel-coexistence-tests placeholder, and the per-file-isolation test header. * The "Legacy OpenTelemetry parser" README section. Retained (intentionally) ------------------------ * file_modified_timestamp helper (moved into the session-state section; used by parse_session_state_file's fallback timestamp). * The OTel env-var deprecation warning surface (warn_about_inert_legacy_env_vars_once, LEGACY_OTEL_ENV_VARS, legacy_otel_env_vars_in_use) and its tests. This is the user-facing notice on first run after upgrade; users who had COPILOT_OTEL_FILE_EXPORTER_PATH / COPILOT_OTEL_DEDUP / COPILOT_PREFER_OTEL set still need to learn those are inert. * The otel/ directory ignore tests (paths.rs) and the README section documenting the env-var deprecation. Validation ---------- * cargo fmt --check clean. * cargo clippy --workspace --all-targets -- -D warnings clean. * cargo test --workspace: 364 passed, 0 failed (was 368: -4 OTel parser tests). All session-state tests, all aggregation tests, and the env-var deprecation tests continue to pass. Diff: 4 files changed, +30 / -711 lines. Co-authored-by: Copilot <[email protected]>
1 parent 0cbfeac commit 1232b97

4 files changed

Lines changed: 30 additions & 711 deletions

File tree

‎rust/crates/ccusage/src/adapter/copilot/README.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -237,11 +237,11 @@ the collapse possible.
237237
| -------------------- | --------------------------------------------------------------- |
238238
| `COPILOT_CONFIG_DIR` | Override the base Copilot directory (defaults to `~/.copilot`). |
239239

240-
## Legacy OpenTelemetry parser
241-
242-
`parser.rs` retains `parse_otel_file` and its helpers under
243-
`#[allow(dead_code)]`, kept solely so existing parser unit tests continue to
244-
verify the OTel chat-span / inference-log / agent-turn dispatch. The
245-
production loader no longer reads OTel data — the full OTel surface (parser
246-
code, env vars, coexistence handling, dedup policy) is scheduled for removal
247-
in [#1208](https://github.com/ryoppippi/ccusage/issues/1208).
240+
## Legacy OpenTelemetry environment variables
241+
242+
ccusage no longer reads OpenTelemetry exports — the production loader uses
243+
`~/.copilot/session-state/<uuid>/events.jsonl` directly. The legacy OTel
244+
environment variables (`COPILOT_OTEL_FILE_EXPORTER_PATH`, `COPILOT_OTEL_DEDUP`,
245+
`COPILOT_PREFER_OTEL`) are inert; if any of them are set when `ccusage copilot`
246+
runs, a one-shot stderr warning explains they are ignored. Unset them, or set
247+
`LOG_LEVEL=0`, to silence the warning.

‎rust/crates/ccusage/src/adapter/copilot/loader.rs‎

Lines changed: 15 additions & 217 deletions
Original file line numberDiff line numberDiff line change
@@ -62,20 +62,14 @@ fn load_entries_inner(
6262
) -> Result<Vec<LoadedEntry>> {
6363
let tz = parse_tz(shared.timezone.as_deref());
6464

65-
// session-state is the only Copilot data source the production loader
66-
// reads. The OTel parser (`parse_otel_file`) still exists in
67-
// `parser.rs` with its own tests, but nothing in production calls it —
68-
// pending full removal in #1208.
69-
//
7065
// Per-file error isolation: one unreadable/corrupted session-state
71-
// file must NOT abort the whole report. The legacy OTel reader and
72-
// every other ccusage adapter treat parse errors as per-file skips
73-
// (with a warning to stderr when logging is enabled); now that
74-
// session-state is the always-on default path, we adopt the same
75-
// semantics here. A locked file (Windows, antivirus scanning), a
76-
// partial write from a still-running CLI, or one file with malformed
77-
// JSON shouldn't make `ccusage copilot daily` print zero — it
78-
// should report whatever the other sessions show.
66+
// file must NOT abort the whole report. Other ccusage adapters treat
67+
// parse errors as per-file skips (with a warning to stderr when
68+
// logging is enabled); we adopt the same semantics here. A locked
69+
// file (Windows, antivirus scanning), a partial write from a
70+
// still-running CLI, or one file with malformed JSON shouldn't make
71+
// `ccusage copilot daily` print zero — it should report whatever the
72+
// other sessions show.
7973
let mut entries: Vec<CopilotUsageEntry> = Vec::new();
8074
for path in session_state_paths()? {
8175
match parse_session_state_file(&path) {
@@ -241,8 +235,8 @@ fn usage_entry_to_loaded(
241235
// sonnet/haiku rows whose `requests.cost == 0` — Auto/Display bill
242236
// those as a genuine `$0` against the premium-request plan, not as
243237
// missing data. The rightmost Auto cell's token-priced fallback only
244-
// fires when the source ships NEITHER billing field (historical OTel
245-
// or a future Copilot CLI variant).
238+
// fires when the source ships NEITHER billing field (a future Copilot
239+
// CLI variant that drops both fields).
246240
//
247241
// Synthetic aggregate-credit rows (empty `entry.model`, emitted by
248242
// `parser.rs::parse_session_state_file` when `data.totalNanoAiu > 0`
@@ -338,198 +332,11 @@ fn usage_entry_to_loaded(
338332
}
339333
}
340334

341-
#[cfg(test)]
342-
use super::report::{report_from_rows, summarize_entries};
343-
344335
#[cfg(test)]
345336
mod tests {
346-
use ccusage_test_support::fs_fixture;
347337
use serde_json::json;
348338

349-
use super::super::parser::parse_otel_file;
350339
use super::*;
351-
use crate::cli::AgentReportKind;
352-
353-
#[test]
354-
fn parses_copilot_chat_spans() {
355-
let fixture = fs_fixture!({
356-
"copilot.jsonl": [
357-
json!({ "type": "metric", "name": "gen_ai.client.token.usage" }).to_string(),
358-
json!({
359-
"type": "span",
360-
"traceId": "trace-1",
361-
"spanId": "span-1",
362-
"name": "chat claude-sonnet-4",
363-
"endTime": [1_775_934_264_u64, 967_317_833_u64],
364-
"attributes": {
365-
"gen_ai.operation.name": "chat",
366-
"gen_ai.request.model": "claude-sonnet-4",
367-
"gen_ai.response.model": "claude-sonnet-4",
368-
"gen_ai.conversation.id": "conv-1",
369-
"gen_ai.usage.input_tokens": 19_452,
370-
"gen_ai.usage.output_tokens": 281,
371-
"gen_ai.usage.cache_read.input_tokens": 123,
372-
"gen_ai.usage.cache_creation.input_tokens": 25,
373-
"gen_ai.usage.reasoning.output_tokens": 128,
374-
},
375-
})
376-
.to_string(),
377-
]
378-
.join("\n"),
379-
});
380-
let file = fixture.path("copilot.jsonl");
381-
382-
let entries = parse_otel_file(&file).unwrap();
383-
384-
assert_eq!(entries.len(), 1);
385-
assert_eq!(entries[0].timestamp_text, "2026-04-11T19:04:24.967Z");
386-
assert_eq!(entries[0].session_id, "conv-1");
387-
assert_eq!(entries[0].model, "claude-sonnet-4");
388-
assert_eq!(entries[0].input_tokens, 19_329);
389-
assert_eq!(entries[0].output_tokens, 281);
390-
assert_eq!(entries[0].cache_creation_tokens, 25);
391-
assert_eq!(entries[0].cache_read_tokens, 123);
392-
assert_eq!(entries[0].reasoning_output_tokens, 128);
393-
assert_eq!(entries[0].dedup_key, "trace-1:span-1");
394-
}
395-
396-
#[test]
397-
fn suppresses_lower_priority_records_for_same_response() {
398-
let fixture = fs_fixture!({
399-
"copilot.jsonl": [
400-
json!({
401-
"type": "span",
402-
"traceId": "trace-dupe",
403-
"spanId": "agent-1",
404-
"name": "invoke_agent GitHub Copilot Chat",
405-
"attributes": {
406-
"gen_ai.operation.name": "invoke_agent",
407-
"gen_ai.response.model": "gpt-5.4-mini",
408-
"gen_ai.conversation.id": "conv-dupe",
409-
"gen_ai.response.id": "resp-dupe",
410-
"gen_ai.usage.input_tokens": 100,
411-
"gen_ai.usage.output_tokens": 30,
412-
},
413-
})
414-
.to_string(),
415-
json!({
416-
"hrTime": [1_775_934_263_u64, 0_u64],
417-
"attributes": {
418-
"event.name": "gen_ai.client.inference.operation.details",
419-
"gen_ai.response.model": "gpt-5.4-mini",
420-
"gen_ai.response.id": "resp-dupe",
421-
"gen_ai.usage.input_tokens": 80,
422-
"gen_ai.usage.output_tokens": 20,
423-
},
424-
"_body": "GenAI inference: gpt-5.4-mini",
425-
})
426-
.to_string(),
427-
json!({
428-
"type": "span",
429-
"traceId": "trace-dupe",
430-
"spanId": "chat-1",
431-
"name": "chat gpt-5.4-mini",
432-
"attributes": {
433-
"gen_ai.operation.name": "chat",
434-
"gen_ai.response.model": "gpt-5.4-mini",
435-
"gen_ai.conversation.id": "conv-dupe",
436-
"gen_ai.response.id": "resp-dupe",
437-
"gen_ai.usage.input_tokens": 60,
438-
"gen_ai.usage.output_tokens": 10,
439-
},
440-
})
441-
.to_string(),
442-
]
443-
.join("\n"),
444-
});
445-
let file = fixture.path("copilot.jsonl");
446-
447-
let entries = parse_otel_file(&file).unwrap();
448-
449-
assert_eq!(entries.len(), 1);
450-
assert_eq!(entries[0].dedup_key, "trace-dupe:chat-1");
451-
assert_eq!(entries[0].input_tokens, 60);
452-
assert_eq!(entries[0].output_tokens, 10);
453-
}
454-
455-
#[test]
456-
fn includes_reasoning_tokens_in_total_tokens() {
457-
let fixture = fs_fixture!({
458-
"copilot.jsonl":
459-
format!(
460-
"{}\n",
461-
json!({
462-
"type": "span",
463-
"traceId": "trace-1",
464-
"spanId": "span-1",
465-
"name": "chat test-model",
466-
"endTime": [1_775_934_264_u64, 0_u64],
467-
"attributes": {
468-
"gen_ai.operation.name": "chat",
469-
"gen_ai.response.model": "test-model",
470-
"gen_ai.conversation.id": "conv-1",
471-
"gen_ai.usage.input_tokens": 100,
472-
"gen_ai.usage.output_tokens": 50,
473-
"gen_ai.usage.cache_read.input_tokens": 10,
474-
"gen_ai.usage.cache_creation.input_tokens": 20,
475-
"gen_ai.usage.reasoning.output_tokens": 5,
476-
},
477-
})
478-
),
479-
});
480-
let file = fixture.path("copilot.jsonl");
481-
let mut pricing = crate::PricingMap::default();
482-
pricing.load_json(
483-
r#"{"test-model":{"input_cost_per_token":1,"output_cost_per_token":2,"cache_creation_input_token_cost":3,"cache_read_input_token_cost":4}}"#,
484-
);
485-
486-
let loaded: Vec<LoadedEntry> = parse_otel_file(&file)
487-
.unwrap()
488-
.into_iter()
489-
.map(|entry| usage_entry_to_loaded(entry, None, CostMode::Auto, &pricing))
490-
.collect();
491-
let rows = summarize_entries(&loaded, AgentReportKind::Daily).unwrap();
492-
let report = report_from_rows(&rows, AgentReportKind::Daily);
493-
494-
assert_eq!(report["daily"][0]["inputTokens"], 90);
495-
assert_eq!(report["daily"][0]["outputTokens"], 50);
496-
assert_eq!(report["daily"][0]["totalTokens"], 175);
497-
assert_eq!(report["daily"][0]["totalCost"], 300.0);
498-
}
499-
500-
#[test]
501-
fn falls_back_to_total_tokens_when_copilot_parts_are_missing() {
502-
let fixture = fs_fixture!({
503-
"copilot.jsonl":
504-
format!(
505-
"{}\n",
506-
json!({
507-
"type": "span",
508-
"traceId": "trace-1",
509-
"spanId": "span-1",
510-
"name": "chat test-model",
511-
"endTime": [1_775_934_264_u64, 0_u64],
512-
"attributes": {
513-
"gen_ai.operation.name": "chat",
514-
"gen_ai.response.model": "test-model",
515-
"gen_ai.conversation.id": "conv-1",
516-
"gen_ai.usage.total_tokens": 567,
517-
},
518-
})
519-
),
520-
});
521-
let file = fixture.path("copilot.jsonl");
522-
523-
let entries = parse_otel_file(&file).unwrap();
524-
525-
assert_eq!(entries.len(), 1);
526-
assert_eq!(entries[0].output_tokens, 567);
527-
assert_eq!(entries[0].reasoning_output_tokens, 0);
528-
}
529-
530-
// ---------------------------------------------------------------
531-
// Session-state loader integration tests
532-
// ---------------------------------------------------------------
533340

534341
use ccusage_test_support::EnvScope;
535342

@@ -590,13 +397,6 @@ mod tests {
590397
fixture
591398
}
592399

593-
// OTel↔session-state coexistence tests previously lived here. The OTel
594-
// data source is no longer read by the production loader (see
595-
// `session_state_paths()` in `paths.rs`), so the dedup/prefer-OTel/etc.
596-
// tests have been removed alongside the env vars they exercised. The
597-
// OTel parser code is kept in `parser.rs` (with `#[allow(dead_code)]`)
598-
// pending full removal in #1208.
599-
600400
#[test]
601401
fn loader_dedup_keeps_distinct_sessions_with_colliding_event_ids() {
602402
// Defense-in-depth: the parser's token-bearing dedup key is
@@ -1641,14 +1441,12 @@ mod tests {
16411441
#[cfg(unix)]
16421442
fn one_unreadable_session_file_does_not_abort_whole_report() {
16431443
// Per-file error isolation: one bad/locked/corrupted session-state
1644-
// file MUST NOT abort the whole report. The legacy OTel reader
1645-
// and every other ccusage adapter treat parse errors as per-file
1646-
// skips; now that session-state is the always-on default, the
1647-
// same semantic must hold here. A locked file (Windows,
1648-
// antivirus scanning), a partial write from a still-running CLI,
1649-
// or one file with permission errors shouldn't make
1650-
// `ccusage copilot daily` print zero — it should report whatever
1651-
// the OTHER sessions show.
1444+
// file MUST NOT abort the whole report. Other ccusage adapters
1445+
// treat parse errors as per-file skips; the same semantic must
1446+
// hold here. A locked file (Windows, antivirus scanning), a
1447+
// partial write from a still-running CLI, or one file with
1448+
// permission errors shouldn't make `ccusage copilot daily` print
1449+
// zero — it should report whatever the OTHER sessions show.
16521450
//
16531451
// Hermetic: build a fixture with TWO session directories. The
16541452
// first is a real, readable shutdown event ($0.04 under Auto).

0 commit comments

Comments
 (0)