Repository navigation
Commit 6438ded
feat(agent): route Claude/GPT model families to their native gateway wire (#3538)
## Summary
Databricks v2 chooses the gateway wire format — OpenAI Responses,
Anthropic Messages, or MLflow chat — purely from substrings in the
endpoint name. There is no family field on the endpoint to key off, so
the substring set *is* the routing contract. The matcher only recognised
`gpt-5`/`gpt5` and `claude`, which makes correct billing depend on every
Claude endpoint happening to be named with the literal string "claude".
## Why this matters
Getting a Claude model onto the Anthropic Messages route is exactly what
lets buzz attach the `cache_control` breakpoint (the fix in #3463). If a
Claude endpoint's catalog name omits "claude" — an alias, a bare
`opus-5`, a `goose-opus-5` — it silently falls through to the MLflow
(OpenAI-wire) path, where Anthropic prompt caching is **structurally
impossible**. The result is the same failure #3463 fixed: 0% cache
reads, the full ~10x read discount lost, and no error — a naming
convention quietly holding up a billing-correctness invariant.
## What changed
`databricks_v2_route_for_model` (`crates/buzz-agent/src/llm.rs`) now
matches broader, case-insensitive marker sets:
- **Claude → Anthropic Messages:** `claude`, `opus`, `sonnet`, `haiku`,
`mythos`, `fable` — the Claude family names and release code names, so a
Claude endpoint reaches the cache-capable route regardless of how it's
named.
- **GPT → OpenAI Responses:** the `gpt` family (now `gpt` on its own,
not just `gpt-5`) plus the GPT-5 launch code names `sol`, `luna`,
`terra`.
OpenAI markers are evaluated first, preserving the prior `gpt-5`-first
precedence for any name that could carry both. Names matching neither
set still fall through to the MLflow chat route.
## Testing
- `cargo fmt`, `cargo clippy -p buzz-agent --all-targets -- -D warnings`
— clean.
- `cargo test -p buzz-agent` — all green (299 lib + integration suites,
0 failures). The `databricks_v2_routes_by_model_family` test was
expanded to cover each new marker, the GPT-5 code names,
case-insensitivity, and the unchanged MLflow fallback (including
`gemini`).
## Relationship to #3463
#3463 taught the Anthropic path to request caching; this makes sure
Claude models actually land on that path. Follow-up still open:
surfacing `cache_creation_input_tokens` end-to-end so a persistent
`reads == 0 && writes == 0` reveals a disabled cache regardless of which
wire a model takes — happy to do that next.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: Atish Patel <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>1 parent 24d90d1 commit 6438ded
1 file changed
Lines changed: 133 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1019 | 1019 | | |
1020 | 1020 | | |
1021 | 1021 | | |
| 1022 | + | |
| 1023 | + | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
| 1030 | + | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
1022 | 1049 | | |
1023 | | - | |
1024 | | - | |
1025 | | - | |
1026 | | - | |
| 1050 | + | |
| 1051 | + | |
| 1052 | + | |
| 1053 | + | |
| 1054 | + | |
| 1055 | + | |
| 1056 | + | |
| 1057 | + | |
| 1058 | + | |
| 1059 | + | |
| 1060 | + | |
| 1061 | + | |
| 1062 | + | |
| 1063 | + | |
| 1064 | + | |
| 1065 | + | |
| 1066 | + | |
1027 | 1067 | | |
1028 | | - | |
| 1068 | + | |
1029 | 1069 | | |
1030 | 1070 | | |
1031 | 1071 | | |
| |||
2579 | 2619 | | |
2580 | 2620 | | |
2581 | 2621 | | |
| 2622 | + | |
2582 | 2623 | | |
| 2624 | + | |
2583 | 2625 | | |
2584 | 2626 | | |
2585 | | - | |
| 2627 | + | |
| 2628 | + | |
| 2629 | + | |
| 2630 | + | |
| 2631 | + | |
| 2632 | + | |
| 2633 | + | |
| 2634 | + | |
| 2635 | + | |
| 2636 | + | |
| 2637 | + | |
| 2638 | + | |
| 2639 | + | |
| 2640 | + | |
| 2641 | + | |
| 2642 | + | |
| 2643 | + | |
| 2644 | + | |
| 2645 | + | |
2586 | 2646 | | |
2587 | 2647 | | |
| 2648 | + | |
| 2649 | + | |
| 2650 | + | |
2588 | 2651 | | |
2589 | 2652 | | |
2590 | | - | |
| 2653 | + | |
2591 | 2654 | | |
2592 | 2655 | | |
| 2656 | + | |
| 2657 | + | |
| 2658 | + | |
| 2659 | + | |
| 2660 | + | |
| 2661 | + | |
| 2662 | + | |
| 2663 | + | |
| 2664 | + | |
| 2665 | + | |
| 2666 | + | |
| 2667 | + | |
| 2668 | + | |
| 2669 | + | |
| 2670 | + | |
| 2671 | + | |
| 2672 | + | |
| 2673 | + | |
| 2674 | + | |
| 2675 | + | |
| 2676 | + | |
| 2677 | + | |
| 2678 | + | |
| 2679 | + | |
| 2680 | + | |
| 2681 | + | |
| 2682 | + | |
| 2683 | + | |
| 2684 | + | |
| 2685 | + | |
| 2686 | + | |
| 2687 | + | |
2593 | 2688 | | |
2594 | 2689 | | |
2595 | | - | |
| 2690 | + | |
| 2691 | + | |
| 2692 | + | |
| 2693 | + | |
| 2694 | + | |
| 2695 | + | |
| 2696 | + | |
| 2697 | + | |
| 2698 | + | |
| 2699 | + | |
| 2700 | + | |
| 2701 | + | |
| 2702 | + | |
| 2703 | + | |
| 2704 | + | |
| 2705 | + | |
| 2706 | + | |
| 2707 | + | |
| 2708 | + | |
| 2709 | + | |
| 2710 | + | |
| 2711 | + | |
| 2712 | + | |
| 2713 | + | |
| 2714 | + | |
| 2715 | + | |
| 2716 | + | |
| 2717 | + | |
| 2718 | + | |
| 2719 | + | |
| 2720 | + | |
2596 | 2721 | | |
2597 | 2722 | | |
2598 | 2723 | | |
| |||
0 commit comments