Skip to content

Commit 9be8630

Browse files
ayupandey-engcnpierrepapi
authored andcommitted
feat(models): add LlmRequest.to_generate_content_body()
`LlmRequest.config` is a flat `GenerateContentConfig` namespace whose fields map to three different destinations on the wire: top-level request fields (such as `tools`, `toolConfig`, and `systemInstruction`), nested `generationConfig` sampling fields, and client-only fields (`httpOptions`, `automaticFunctionCalling`, `shouldReturnHttpResponse`, and `automaticContinuation`) that endpoints reject. Flattening `config` into `generationConfig` in a custom `BaseLlm` silently drops `tools` without raising an error. Add `LlmRequest.to_generate_content_body()` and `models._generate_content_body.to_generate_content_body()`, delegating to `google-genai`'s parameter converters (`_GenerateContentParameters_to_vertex` and `_GenerateContentParameters_to_mldev`) so the field split stays synchronized with `GenerateContentConfig`, finishing the body the way `google-genai` does before sending it so the result is JSON-serializable, and annotating API-incompatible field errors with the offending `LlmRequest.config` field name and target API. Fixes #6880 Merge #6935 Co-authored-by: cnpierrepapi <[email protected]> Co-authored-by: Ayush Pandey <[email protected]> PiperOrigin-RevId: 1016026963
1 parent 5c4fa92 commit 9be8630

3 files changed

Lines changed: 698 additions & 0 deletions

File tree

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
# Copyright 2026 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Turn an ``LlmRequest`` into a ``generateContent`` request body.
16+
17+
``LlmRequest.config`` is a ``types.GenerateContentConfig``: one flat namespace
18+
whose fields go to three different places on the wire. Some sit at the top level
19+
of the request, most belong inside ``generationConfig``, and a few mean
20+
something only to the client library and are rejected by the endpoint.
21+
22+
Nothing on the type says which is which, and getting it wrong does not raise.
23+
Bury ``tools`` inside ``generationConfig`` and the call still returns 200 with a
24+
well-formed response in it. The tools are simply absent, so the model answers
25+
from memory instead of calling anything, and the failure looks like a model that
26+
chose not to call the tool.
27+
28+
Rather than keep a hand-written list of which field goes where, this delegates
29+
to the same converter ``google-genai`` uses for its own built-in models. A
30+
hand-written list is a second copy of the mapping and goes stale the moment
31+
``GenerateContentConfig`` gains a field: every custom ``BaseLlm`` in the wild
32+
then starts silently dropping that field. Delegating means there is one copy and
33+
it moves when the type moves.
34+
"""
35+
36+
from __future__ import annotations
37+
38+
from typing import Any
39+
from typing import TYPE_CHECKING
40+
41+
from google.genai import types
42+
43+
from ..utils.variant_utils import get_google_llm_variant
44+
from ..utils.variant_utils import GoogleLLMVariant
45+
46+
if TYPE_CHECKING:
47+
from google.genai import Client
48+
from google.genai._api_client import BaseApiClient
49+
50+
from .llm_request import LlmRequest
51+
52+
__all__ = ["to_generate_content_body"]
53+
54+
55+
class _ApiMode:
56+
"""Stands in for a genai API client when the caller does not have one.
57+
58+
The converters read ``vertexai`` from the client they are handed. Passing a
59+
real client works and is preferred when the caller has one. This stands in
60+
when the caller does not, so the helper stays usable from a ``BaseLlm`` that
61+
talks REST directly and never builds a genai client at all.
62+
"""
63+
64+
__slots__ = ("vertexai",)
65+
66+
def __init__(self, vertexai: bool):
67+
self.vertexai = vertexai
68+
69+
@property
70+
def project(self) -> str:
71+
# On Vertex AI the converter expands a short cached_content name, such as
72+
# "cachedContents/123", with the client's project and location. A stand-in
73+
# has neither, and without this the caller would get an AttributeError
74+
# naming this private class instead of the field to fix.
75+
raise ValueError(
76+
"cached_content must be a full resource name"
77+
" (projects/{project}/locations/{location}/cachedContents/{id})"
78+
" unless an api_client is given, because Vertex AI needs a project"
79+
" and location to expand a short name."
80+
)
81+
82+
@property
83+
def location(self) -> str:
84+
return self.project
85+
86+
87+
def to_generate_content_body(
88+
llm_request: LlmRequest,
89+
*,
90+
vertexai: bool | None = None,
91+
api_client: Client | BaseApiClient | None = None,
92+
) -> dict[str, Any]:
93+
"""Return the request body for ``llm_request``.
94+
95+
Args:
96+
llm_request: The ``LlmRequest`` to convert.
97+
vertexai: Whether the body is bound for Vertex AI rather than the Gemini
98+
Developer API. The two differ in more than the URL: some config fields
99+
exist on one and not the other, so this is not cosmetic. Defaults to the
100+
backend ADK is configured for, which is the Gemini Developer API unless
101+
``GOOGLE_GENAI_USE_ENTERPRISE`` (or the deprecated
102+
``GOOGLE_GENAI_USE_VERTEXAI``) is enabled. Ignored when ``api_client``
103+
is given.
104+
api_client: A genai ``Client`` or ``BaseApiClient``, if the caller has one.
105+
Preferred over ``vertexai`` because it is the real client rather than a
106+
stand-in, and needed on Vertex AI to expand a short ``cached_content``
107+
name such as ``cachedContents/123``.
108+
109+
Returns:
110+
The request body as a JSON-serializable dict, the same one google-genai
111+
sends for this request: nested objects are plain dicts and bytes are
112+
base64-encoded. The model name is not in it; the caller already knows the
113+
model, and on Vertex it belongs in the URL.
114+
115+
Raises:
116+
ValueError: If a config field is not valid for the target API, or if
117+
``cached_content`` is a short name on Vertex AI and no ``api_client`` is
118+
given. The message names the ``GenerateContentConfig`` field so the error
119+
points at the line that set it rather than at the converter.
120+
"""
121+
from google.genai import _common as _genai_common
122+
from google.genai import models as _genai_models
123+
124+
client: Any
125+
if api_client is not None:
126+
inner = getattr(api_client, "_api_client", None)
127+
client = inner if inner is not None else api_client
128+
else:
129+
if vertexai is None:
130+
# Match the backend the rest of ADK picks, rather than guessing one.
131+
vertexai = get_google_llm_variant() == GoogleLLMVariant.VERTEX_AI
132+
client = _ApiMode(vertexai)
133+
134+
is_vertex = getattr(client, "vertexai", True)
135+
136+
params = types._GenerateContentParameters(
137+
model=llm_request.model,
138+
contents=llm_request.contents,
139+
config=llm_request.config,
140+
)
141+
142+
convert = (
143+
_genai_models._GenerateContentParameters_to_vertex
144+
if is_vertex
145+
else _genai_models._GenerateContentParameters_to_mldev
146+
)
147+
148+
try:
149+
body = convert(client, params)
150+
except ValueError as exc:
151+
raise ValueError(_annotate(str(exc), vertexai=is_vertex)) from exc
152+
153+
# Finish the body the way google-genai's own generate_content does before it
154+
# sends the request. Keys starting with "_", such as "_url", carry routing
155+
# information for the client rather than body content, and sending them is a
156+
# 400; google-genai drops a top-level "config" key as well. What remains
157+
# still holds pydantic objects and raw bytes, which json.dumps rejects.
158+
body = {
159+
key: value
160+
for key, value in body.items()
161+
if not key.startswith("_") and key != "config"
162+
}
163+
return _genai_common.encode_unserializable_types(
164+
_genai_common.convert_to_dict(body)
165+
)
166+
167+
168+
def _annotate(message: str, *, vertexai: bool) -> str:
169+
"""Point a genai validation error back at the ADK field that caused it.
170+
171+
The converters raise messages that start with the offending field name, e.g.
172+
"enable_enhanced_civic_answers parameter is only supported in ...". That is
173+
accurate and still hard to act on from a ``BaseLlm``, because the reader has
174+
no reason to connect it to the config they built several layers up.
175+
"""
176+
first = message.split(" ", 1)[0]
177+
if first in types.GenerateContentConfig.model_fields:
178+
mode = "Vertex AI" if vertexai else "the Gemini Developer API"
179+
return (
180+
f"{message} (set as LlmRequest.config.{first}; this request targets"
181+
f" {mode}.)"
182+
)
183+
return message

‎src/google/adk/models/llm_request.py‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@
1515
from __future__ import annotations
1616

1717
import logging
18+
from typing import Any
19+
from typing import TYPE_CHECKING
1820

1921
from google.genai import types
2022
from pydantic import BaseModel
@@ -28,6 +30,10 @@
2830
from ._service_tier import ServiceTier
2931
from .cache_metadata import CacheMetadata
3032

33+
if TYPE_CHECKING:
34+
from google.genai import Client
35+
from google.genai._api_client import BaseApiClient
36+
3137

3238
def _find_tool_with_function_declarations(
3339
llm_request: LlmRequest,
@@ -405,3 +411,46 @@ def set_output_schema(
405411

406412
self.config.response_schema = schema
407413
self.config.response_mime_type = "application/json"
414+
415+
def to_generate_content_body(
416+
self,
417+
*,
418+
vertexai: bool | None = None,
419+
api_client: Client | BaseApiClient | None = None,
420+
) -> dict[str, Any]:
421+
"""Returns this request as a ``generateContent`` request body.
422+
423+
``config`` is one flat namespace, but its fields go to three different
424+
places on the wire. Some sit at the top level, most belong inside
425+
``generationConfig``, and a few are consumed by the client library and
426+
rejected by the endpoint. Getting that split wrong does not raise: bury
427+
``tools`` in ``generationConfig`` and the call returns 200 with no function
428+
call in it, because none was ever offered.
429+
430+
Custom ``BaseLlm`` implementations that build their own request body should
431+
use this rather than flattening ``config`` by hand.
432+
433+
Args:
434+
vertexai: Whether the body is bound for Vertex AI rather than the Gemini
435+
Developer API. Defaults to the backend ADK is configured for, which is
436+
the Gemini Developer API unless ``GOOGLE_GENAI_USE_ENTERPRISE`` (or the
437+
deprecated ``GOOGLE_GENAI_USE_VERTEXAI``) is enabled. Ignored when
438+
``api_client`` is given.
439+
api_client: A genai ``Client`` or ``BaseApiClient``, if the caller has
440+
one. Needed on Vertex AI to expand a short ``cached_content`` name.
441+
442+
Returns:
443+
A JSON-serializable dict ready to send as the request body.
444+
445+
Raises:
446+
ValueError: If a config field is not valid for the target API, or if
447+
``cached_content`` is a short name on Vertex AI and no ``api_client``
448+
is given.
449+
"""
450+
# Imported here rather than at module scope: this pulls in google.genai's
451+
# model converters, which nothing else in this module needs.
452+
from ._generate_content_body import to_generate_content_body
453+
454+
return to_generate_content_body(
455+
self, vertexai=vertexai, api_client=api_client
456+
)

0 commit comments

Comments
 (0)