Skip to content

Commit 167bc08

Browse files
atishpatelclaude
andauthored
feat(agent): make Gemini and MLflow-route models usable through databricks_v2 (#3569)
## Summary Makes Gemini — and every other non-Claude, non-GPT-5 model on the Databricks MLflow route (`databricks_v2`) — usable in an agent loop. These are the non-`benchmarks/` changes from `benchmark/harness-accounting-and-solo`, lifted onto a clean base off `main` so they can land independently while the harness work continues. Two defects made these models unusable, one fatal and one silent. Both live only in `openai_body` / `parse_openai`, which is the least-exercised of the three `databricks_v2` sub-routes — the `luna`/`sol` conditions run the Responses route and the `opus` conditions run the Anthropic route, so **this change is inert for every model already in use** and only lights up the MLflow path. ## Why a third route at all `databricks_v2_route_for_model` buckets by model family: `claude*` → Anthropic Messages, `gpt-5`/code-names → OpenAI Responses, **everything else → MLflow chat-completions**. Gemini, Qwen, gpt-oss, and friends all fall through to that third pair — and both bugs below live only there. ## D1 — dropped thought signatures (fatal) Gemini returns a `thoughtSignature` on every tool call and **requires it echoed back**. `openai_body` reserialized each call as `{id, type, function}` only, dropping the field, so the next request 400'd: ``` HTTP 400 Function call is missing a thought_signature in functionCall parts. ``` For a coding agent this fires on the **first** tool call, so the model never completes a single turn. **Position is load-bearing.** A four-shape replay probe against the live gateway established that the signature must sit as a *sibling* of `function` — nesting it inside `function{}` fails with the *same* 400 as omitting it. A fix that "preserves the field" without preserving its position passes a unit test and still 400s. The fix: `ToolCall` gains `provider_extra: Map<String, Value>`. `parse_openai` captures every top-level wire key except the three we model (`id`, `type`, `function`); `openai_body` re-emits them beside `function`. Keeping *whatever we did not model*, rather than naming `thoughtSignature`, means the next provider with an opaque per-call token needs no change here. The Responses and Anthropic replay shapes are fully modelled, so they pass `Default::default()` and stay **byte-identical** to before. ### D1b — duplicate tool-call ids (same root cause) Gemini returns the **function name** as the id, so two parallel calls to one function arrive sharing an id — and that id is what pairs a `role:"tool"` result back to its call, making two results indistinguishable. `dedupe_provider_ids` suffixes collisions (`get_weather`, `get_weather-2`). Safe because both halves of the pairing (the assistant `tool_calls[].id` and the result's `tool_call_id`) are re-emitted from this same value; the provider never sees its original id again. ## D2 — block-array content discarded (silent, worse than a crash) `parse_openai` read `content` with `as_str()`, which returns `""` for anything that isn't a JSON string. Gemini (and Qwen35, gpt-oss) send an array of typed blocks: ```json "content": [ {"type": "reasoning", "summary": [{"type": "summary_text", "text": "…"}]}, {"type": "text", "text": "391"} ] ``` So the model answered and the answer was thrown away — no error, no warning, just a turn that looked like the model had said nothing. On a benchmark this reads as "Gemini is bad at the task" rather than "buzz dropped the reply." `openai_content_parts` now accepts either shape — string as before, or a block array where `text` blocks concatenate into text and `reasoning` blocks into reasoning (Gemini nests the prose one level down under `summary`). Message-level `reasoning_content` / `reasoning` still win when present, so DeepSeek and vLLM-style hosts are unchanged; block reasoning is the last fallback. ## Also: a turn-start log line (`buzz-acp` `pool.rs`) Small, independent observability change that also rides in the non-benchmark delta: `run_prompt_task` now emits a `pool::prompt` "turn starting" line, labelled by the same `prompt_label` helper as `log_stop_reason`, so a log reads as start/stop pairs. An unpaired start is the only durable evidence that a turn was entered and never returned — without it, a stalled agent and an agent nobody woke leave identical (zero-completion) logs. ## Interaction with #3538 #3538 (already merged) rewrote `databricks_v2_route_for_model` to route by boundary-aware model-family segments. That change and this one touch **different functions** in `llm.rs` — routing vs. body/parse — and compose cleanly; the family routing decides *which* pair runs, and this fixes the MLflow pair it can now select. ## Testing - `cargo fmt --all -- --check`, `cargo clippy -p buzz-agent -p buzz-acp --all-targets -- -D warnings` — clean. - `cargo test -p buzz-agent -p buzz-acp` — all green (304 + 632 lib tests plus integration suites, 0 failures). Five new tests cover: block-array text extraction, plain-string regression, passthrough capture (and non-duplication of the modelled keys), replay position (`thoughtSignature` beside `function`, not inside it), and id de-duplication. - Wire evidence: the four-shape replay table and the reasoning-effort probe were run against `block-lakehouse-staging` (recorded in the design doc). ## Relationship to the benchmark branch The full design write-up (four-shape replay table, position-matters analysis, effort verification, and open pricing item) lives in `docs/08-gemini-provider-fixes.md` on `benchmark/harness-accounting-and-solo`. The benchmark manifests and endpoint-config entries that exercise these models are separable and stay on that branch. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Atish Patel <[email protected]> Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
1 parent 0692fa4 commit 167bc08

4 files changed

Lines changed: 294 additions & 16 deletions

File tree

‎crates/buzz-acp/src/pool.rs‎

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1897,6 +1897,18 @@ pub async fn run_prompt_task(
18971897
None => prompt_sections.iter().map(String::as_str).collect(),
18981898
};
18991899

1900+
// Turn start, labelled exactly as `log_stop_reason` labels the end, so a
1901+
// log reads as start/stop pairs. Purely observational: an unpaired start is
1902+
// the only durable evidence that a turn was entered and never returned, and
1903+
// without it a stalled agent and an agent nobody woke leave identical logs —
1904+
// zero completions either way, so anything reading them afterwards has to
1905+
// guess which happened.
1906+
tracing::info!(
1907+
target: "pool::prompt",
1908+
"turn starting for {}",
1909+
prompt_label(&source)
1910+
);
1911+
19001912
// When control_rx is Some (channel tasks), wrap the prompt in select! so
19011913
// the main loop can cancel, interrupt, or rotate it. Heartbeats
19021914
// (control_rx=None) take the simple await path — they are not controllable.
@@ -3131,12 +3143,19 @@ fn classify_control_cancel_failure(
31313143
}
31323144
}
31333145

3134-
/// Log a stop reason at the appropriate tracing level.
3135-
fn log_stop_reason(source: &PromptSource, stop_reason: &StopReason) {
3136-
let label = match source {
3146+
/// How a turn's source is named in the `pool::prompt` log lines.
3147+
///
3148+
/// Shared by the turn-start and turn-stop lines so a log can be read as pairs.
3149+
fn prompt_label(source: &PromptSource) -> String {
3150+
match source {
31373151
PromptSource::Channel(cid) => format!("channel {cid}"),
31383152
PromptSource::Heartbeat => "heartbeat".to_string(),
3139-
};
3153+
}
3154+
}
3155+
3156+
/// Log a stop reason at the appropriate tracing level.
3157+
fn log_stop_reason(source: &PromptSource, stop_reason: &StopReason) {
3158+
let label = prompt_label(source);
31403159
match stop_reason {
31413160
StopReason::EndTurn => {
31423161
tracing::info!(target: "pool::prompt", "turn complete for {label}: end_turn");

‎crates/buzz-agent/src/agent.rs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -696,6 +696,9 @@ pub(crate) fn push_hook_outputs_as_tool_results(
696696
provider_id: provider_id.clone(),
697697
name: tool_name,
698698
arguments: serde_json::json!({}),
699+
// Synthesised locally, so there is no provider wire form to
700+
// preserve.
701+
provider_extra: Default::default(),
699702
}],
700703
});
701704
history.push(HistoryItem::ToolResult(ToolResult {

0 commit comments

Comments
 (0)