fix(gateway): apply Vertex Anthropic request fields in `adapter_class` so judges work by Yash-Chindam · Pull Request #25556 · mlflow/mlflow

Yash-Chindam · 2026-09-03T04:24:11Z

Related Issues/PRs

What changes are proposed in this pull request?

Vertex rejects Anthropic partner-model requests that omit anthropic_version. That field was injected only by _VertexAIClaudeProvider._prepare_payload(), a provider hook reached exclusively from AnthropicProvider._chat() and ._chat_stream().

The judge path does not go through either. GatewayAdapter._invoke_and_handle_tools() builds its request from adapter_class.chat_to_model() and posts it with send_chat_request() (gateway_adapter.py:581-590), so _prepare_payload() never runs and every vertex_ai:/claude-* judge invocation fails with anthropic_version: Field required.

Bedrock does not have this problem because AmazonBedrockAnthropicAdapter applies the equivalent field in chat_to_model() (bedrock.py:28-35), one layer up. Vertex placed the same required transformation on the provider instead — the layer the judge path skips.

This adds _VertexAIClaudeAdapter(AnthropicAdapter) and overrides adapter_class on _VertexAIClaudeProvider, as agreed on the issue.

On the double-execution concern raised in review: _prepare_payload() deliberately does not call _VertexAIClaudeAdapter.chat_to_model. It calls a small _apply_vertex_fields() classmethod that only pops model and sets anthropic_version, so message conversion and max_tokens defaulting still run exactly once on the gateway path. Verified by running both sequences on a payload that exercises system-message conversion and max_tokens defaulting:

upstream: AnthropicAdapter.chat_to_model -> old _prepare_payload
branch  : AnthropicAdapter.chat_to_model -> new _prepare_payload

gateway path identical to upstream : True
  max_tokens applied once          : True  -> 8192
  system message converted once    : True  -> 'be terse'
  messages identical               : True

Behaviour of the judge path, which is the bug:

BEFORE  adapter_class: AnthropicAdapter         anthropic_version: None                model in body: True
        -> Vertex: REJECTED, "anthropic_version: Field required"
AFTER   adapter_class: _VertexAIClaudeAdapter   anthropic_version: vertex-2023-10-16   model in body: False
        -> Vertex: accepted

One existing test is updated, which is a review decision rather than a mechanical change: test_adapter_class_matches_the_active_delegate asserted _make_claude_provider().adapter_class is AnthropicAdapter. That assertion encodes the buggy behaviour — the plain adapter, with no anthropic_version — so it now asserts _VertexAIClaudeAdapter, with a comment explaining why.

How is this PR tested?

Existing unit/integration tests
New unit/integration tests
Manual tests

New: test_claude_adapter_applies_vertex_fields_without_provider_hooks formats through adapter_class.chat_to_model and .chat_streaming_to_model by themselves — the judge path — and asserts anthropic_version is set and model is absent. Nothing covered that route before.

A note on my own verification, in the interest of transparency: I could not run the suite locally. Every test in tests/gateway/providers/test_vertex_ai.py errors at setup with RuntimeError: Failed to start server — the session fixture at tests/conftest.py:812-853 spawns an mlflow server subprocess via uv run and times out after 60s in my environment. It affects all tests in the file, including ones this PR does not touch (test_with_credentials, test_project_required, the test_maas_* family), so it is environmental. I verified the assertions by executing the test bodies directly against both the patched and unpatched source, which is where the before/after output above comes from. CI is the real check.

ruff check and ruff format --check both pass under the pinned ruff==0.16.4 from lint-requirements.txt. Commits are DCO signed off.

Does this PR require documentation update?

No.

Does this PR require updating the MLflow Skills repository?

No.

Release Notes

Is this a user-facing change?

Yes. Give a description of this change to be included in the release notes for MLflow users.

Fixed mlflow.genai judges failing with anthropic_version: Field required when using Anthropic Claude models hosted on Vertex AI (vertex_ai:/claude-*). The failure was silent — it surfaced as a SCORER_ERROR with value=None, so mlflow.genai.evaluate() completed and rendered a full table with an empty judge column.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

area/evaluation: MLflow model evaluation features, evaluation metrics, and evaluation workflows
area/gateway: MLflow AI Gateway client APIs, server, and third-party integrations

How should the PR be classified in the release notes? Choose one:

rn/bug-fix - A user-facing bug fix worth mentioning in the release notes

Is this PR a critical bugfix or security fix that should go into the next patch release?

This PR is critical and needs to be in the next patch release
This PR can wait for the next minor release

Disclosure: I used an AI assistant (Claude) to help trace the call paths, implement the fix, and draft this description. Every file/line reference and every result above was produced by running the code, and I reviewed the change before submitting.

Vertex rejects Anthropic partner-model requests that omit anthropic_version. That field was injected only by _VertexAIClaudeProvider._prepare_payload(), a provider hook reached exclusively from AnthropicProvider._chat() and ._chat_stream(). The judge path does not go through either. GatewayAdapter._invoke_and_handle_tools() builds its request from adapter_class.chat_to_model() and posts it with send_chat_request(), so _prepare_payload() never runs and every vertex_ai:/claude-* judge invocation fails with "anthropic_version: Field required". The failure is silent to the caller: it is stored as a SCORER_ERROR with value=None, so mlflow.genai.evaluate() completes and renders a full table with an empty judge column. Bedrock does not have this problem because AmazonBedrockAnthropicAdapter applies the equivalent field in chat_to_model(), one layer up. Vertex placed the same required transformation on the provider instead, which is the layer the judge path skips. Move it to a _VertexAIClaudeAdapter and point _VertexAIClaudeProvider.adapter_class at it, so both paths are covered from one definition. _chat()/._chat_stream() format through AnthropicAdapter directly rather than self.adapter_class, so _prepare_payload() is kept for the gateway path and now delegates to the adapter rather than repeating the transformation. test_adapter_class_matches_the_active_delegate is updated: Claude now resolves to the Vertex-specific adapter, which is the point of the change. Fixes mlflow#25543 Signed-off-by: Yash-Chindam <[email protected]>

github-actions · 2026-09-03T04:24:26Z

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/25556/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/25556/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/25556/merge

PR author's recent activity

In the last 14 days, @Yash-Chindam opened 6 PRs across 3 repos:

Repository	Open	Closed	Merged	Total
github/spec-kit	2	0	2	4
mlflow/mlflow	1	0	0	1
langchain-ai/langgraph	0	1	0	1

…aude-judge-anthropic-version

…rsion

kriscon-db

Thanks for the fix and the detailed write-up. The approach looks right, and I verified both the judge path and the gateway path locally (anthropic_version set once, model dropped in both). A few small suggestions inline.

-- Claude-drafted, reviewed before posting.

kriscon-db · 2026-09-04T05:45:15Z

+        return cls._apply_vertex_fields(super().chat_to_model(payload, config))
+
+    @classmethod
+    def chat_streaming_to_model(cls, payload, config):


AnthropicAdapter.chat_streaming_to_model already delegates to cls.chat_to_model, so the inherited method would pick up your chat_to_model override and apply the Vertex fields once. As written, this override applies them a second time. It's idempotent today so not a bug, but the redundant pass isn't desirable, and it becomes a latent trap if _apply_vertex_fields ever stops being idempotent. It also differs from AmazonBedrockAnthropicAdapter, which overrides only chat_to_model. Could we drop this override and let the base delegation carry it? The streaming assertion in the test should still pass.

-- Claude-drafted, reviewed before posting.

kriscon-db · 2026-09-04T05:45:15Z

    assert "model" not in call_kwargs[1]["json"]


+def test_claude_adapter_applies_vertex_fields_without_provider_hooks():


Nice that this formats through adapter_class directly. One gap: it stops at the adapter, whereas the bug surfaced through the judge integration (GatewayAdapter._invoke_and_handle_tools -> _get_provider_instance("vertex_ai", "claude-*") -> adapter_class.chat_to_model). Could we add a case that resolves the provider through that path too, so a future break in the resolution wiring is also caught? Optional, but it would pin the exact regression.

-- Claude-drafted, reviewed before posting.

kriscon-db · 2026-09-04T05:45:15Z

+        return payload
+
+    @classmethod
+    def chat_to_model(cls, payload, config):


nit: _apply_vertex_fields just above is fully typed, but this override isn't. The base signatures are untyped too, so this is optional. Could we add payload: dict[str, Any], config -> dict[str, Any] to keep the new class internally consistent?

-- Claude-drafted, reviewed before posting.

github-actions · 2026-09-04T06:09:31Z

Documentation preview for c37e9ec is available at:

https://pr-25556--mlflow-docs-preview.netlify.app/docs/latest/

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

…rsion

kriscon-db's review on mlflow#25556 flagged that _VertexAIClaudeAdapter's chat_streaming_to_model override applies the Vertex fields a second time, since AnthropicAdapter.chat_streaming_to_model already delegates to chat_to_model, which the adapter also overrides. Drop the redundant override and let base-class delegation carry it, matching how AmazonBedrockAnthropicAdapter only overrides chat_to_model. Also add a test that resolves the provider via _get_provider_instance, the same seam mlflow.genai.judges.adapters.gateway_adapter uses, so a future regression in that resolution wiring is caught too - not just the adapter-level formatting the existing test already covers. Co-Authored-By: Claude Sonnet 5 <[email protected]> Signed-off-by: Yash-Chindam <[email protected]>

Yash-Chindam · 2026-09-04T16:46:42Z

Pushed 9858353 addressing all three review comments:

Dropped the chat_streaming_to_model override — AnthropicAdapter.chat_streaming_to_model already delegates to cls.chat_to_model, which this adapter overrides, so the extra override was double-applying _apply_vertex_fields. Confirmed via direct execution that it now runs exactly once on the streaming path.
Added test_claude_adapter_applies_vertex_fields_via_judge_provider_resolution, which goes through _get_provider_instance("vertex_ai", ...) — the same resolution path gateway_adapter.py uses in production — rather than only the direct adapter-level formatting the existing test already covers.
Added type hints to chat_to_model to match _apply_vertex_fields above it.

ruff check and ruff format --check both pass.

…rsion

Co-Authored-By: Claude Fable 5.1 <[email protected]> Signed-off-by: Aaron Teo <[email protected]>

aaronteo-db

LGTM! @Yash-Chindam Thanks for addressing the comments from @kriscon-db .

I made some minor shortening on some verbose comments. Thanks for your contribution!

github-actions Bot added community Community/external contribution team-review Trigger a team review request size/M labels Sep 3, 2026

github-actions Bot requested review from aaronteo-db and mprahl September 3, 2026 04:24

github-actions Bot added area/evaluation MLflow Evaluation area/gateway MLflow AI Gateway client APIs, server, and third-party integrations rn/bug-fix Mention under Bug Fixes in Changelogs. labels Sep 3, 2026

Merge remote-tracking branch 'origin/master' into fix/25543-vertex-cl…

7eb4f90

…aude-judge-anthropic-version

github-actions Bot assigned kriscon-db Sep 3, 2026

Yash-Chindam added 2 commits September 3, 2026 19:11

Merge branch 'master' into fix/25543-vertex-claude-judge-anthropic-ve…

9763bbc

…rsion

Merge branch 'master' into fix/25543-vertex-claude-judge-anthropic-ve…

a45678c

…rsion

kriscon-db reviewed Sep 4, 2026

View reviewed changes

Yash-Chindam and others added 2 commits September 4, 2026 22:01

Merge branch 'master' into fix/25543-vertex-claude-judge-anthropic-ve…

97a8f7a

…rsion

Merge branch 'master' into fix/25543-vertex-claude-judge-anthropic-ve…

bc1aeb0

…rsion

Yash-Chindam requested a review from kriscon-db September 5, 2026 03:54

Shorten Vertex Claude adapter docstring and _prepare_payload comment

c37e9ec

Co-Authored-By: Claude Fable 5.1 <[email protected]> Signed-off-by: Aaron Teo <[email protected]>

aaronteo-db approved these changes Sep 6, 2026

View reviewed changes

github-actions Bot assigned aaronteo-db Sep 6, 2026

aaronteo-db merged commit 4f488cd into mlflow:master Sep 6, 2026
66 of 70 checks passed

Navigation Menu

Repository navigation

fix(gateway): apply Vertex Anthropic request fields in `adapter_class` so judges work - #25556

fix(gateway): apply Vertex Anthropic request fields in `adapter_class` so judges work#25556
aaronteo-db merged 8 commits into
mlflow:masterfrom
Yash-Chindam:fix/25543-vertex-claude-judge-anthropic-version

Yash-Chindam commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Install mlflow from this PR

Uh oh!

kriscon-db left a comment •

edited

Loading

Uh oh!

kriscon-db Sep 4, 2026

Uh oh!

kriscon-db Sep 4, 2026

Uh oh!

kriscon-db Sep 4, 2026

Uh oh!

github-actions Bot commented Sep 4, 2026 •

edited

Loading

Uh oh!

Yash-Chindam commented Sep 4, 2026

Uh oh!

aaronteo-db left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		assert "model" not in call_kwargs[1]["json"]


		def test_claude_adapter_applies_vertex_fields_without_provider_hooks():

Conversation

Yash-Chindam commented Sep 3, 2026

Related Issues/PRs

What changes are proposed in this pull request?

How is this PR tested?

Does this PR require documentation update?

Does this PR require updating the MLflow Skills repository?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Is this PR a critical bugfix or security fix that should go into the next patch release?

Uh oh!

github-actions Bot commented Sep 3, 2026

Install mlflow from this PR

Uh oh!

kriscon-db left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kriscon-db Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

kriscon-db Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

kriscon-db Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Yash-Chindam commented Sep 4, 2026

Uh oh!

aaronteo-db left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kriscon-db left a comment •

edited

Loading

github-actions Bot commented Sep 4, 2026 •

edited

Loading