Skip to content

test(llm-drivers): pin claude_code resolved-model parsing (#6318)#6331

Merged
houko merged 2 commits into
mainfrom
fix-agent-actual-model-canonical
Jun 26, 2026
Merged

test(llm-drivers): pin claude_code resolved-model parsing (#6318)#6331
houko merged 2 commits into
mainfrom
fix-agent-actual-model-canonical

Conversation

@houko

@houko houko commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #6318 (the "auto-detect the model an agent CLI actually ran" half). The detection itself already exists across the agent-CLI drivers; this PR closes the one remaining gap — it was untested for claude_code.

State of the auto-detect mechanism (verified):

  • codex_cliparse_model_from_banner reads the model: line from codex's stderr startup banner and stamps it into CompletionResponse.actual_model (landed in codex cli 需要获取实际使用的模型 #6134). Already covered by test_parse_model_from_banner_*.
  • claude_codestream() captures the resolved model from the stream-json system/init event; complete() reads it from the result JSON's model field (forward-compat — claude -p --output-format json does not emit it yet, so that path stays None and the kernel falls back to the requested id). Both paths were wired but had no test.
  • Kernel metering honours it uniformly: model: result.actual_model.clone().unwrap_or_else(|| model.clone()) (messaging.rs, agent_execution.rs, tools_and_skills.rs), so a driver that resolves its own model attributes the real one.

Change

Test-only. Two regression tests in crates/librefang-llm-drivers/src/drivers/claude_code.rs:

  • stream_json_init_event_carries_resolved_model — deserializes a representative stream-json system/init line and asserts ClaudeStreamEvent.model is parsed (the streaming agent path's detection source).
  • json_output_surfaces_model_when_present_else_none — pins both shapes of the result JSON: absent modelNone (today's --output-format json behaviour, kernel falls back to requested id), present model → surfaced (forward-compat for a future CLI version).

These lock the deserialization contract so a refactor that drops or renames the serde model field fails loudly instead of silently turning resolved-model detection back into a no-op.

Not changed (and why)

  • gemini_cli stamps actual_model: None. Gemini CLI emits no model banner/JSON field and runs exactly the --model it is given (no divergence), so there is nothing to detect; None correctly falls back to the requested id. Its CLI calls report zero tokens (billing is on the user's Google account), so the recorded model string is display-only and not a cost-attribution input — stamping the prefix-stripped id would only drop the gemini-cli/ provenance for no benefit.

Verification

In the worktree:

  • cargo test -p librefang-llm-drivers --lib model — 24 passed, 0 failed (includes the two new tests)
  • cargo clippy -p librefang-llm-drivers --all-targets -- -D warnings — clean
  • cargo fmt -p librefang-llm-drivers -- --check — clean

Refs #6318. The manual-management half landed in #6327.

The #6318 auto-detect mechanism for the claude_code driver was wired but untested: stream() captures the model from the stream-json system/init event, and complete() reads it from the result JSON's model field (forward-compat, since `claude -p --output-format json` does not emit it yet) — both stamping CompletionResponse.actual_model so kernel metering attributes the real model rather than the requested id.

Add two regression tests pinning the deserialization contract for both shapes so a refactor that drops or renames the serde `model` field fails loudly instead of silently turning resolved-model detection back into a no-op.
@houko
houko merged commit 71cac88 into main Jun 26, 2026
31 checks passed
@houko
houko deleted the fix-agent-actual-model-canonical branch June 26, 2026 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S 10-49 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants