Google ADK Bug Report: before_model_callback LlmResponse Not Guaranteed to Skip LLM Call
Required Information
Describe the Bug:
When returning an LlmResponse from before_model_callback to skip the LLM call and return a deterministic response, the behavior is inconsistent. The LLM call is sometimes still made, resulting in non-deterministic output instead of the pre-generated content.
Additionally, the callback execution flow does not guarantee that returning LlmResponse will always prevent the LLM from being invoked - especially when other conditions in the callback may have already modified llm_request.config.tool_config.
Steps to Reproduce:
- Create an agent with
before_model_callback that tracks state across multiple tool calls
- Implement a multi-phase workflow:
- Phase 1: Load artifacts (images)
- Phase 2: Process artifacts with
save_datapoints tool
- Phase 3: First
save_final_corrected_datapoints call(we have multiple images from which different sections needs to be extracted and saved, so we have this extra final tool to give the final correct fields)
- Phase 4: Self-reflection with
load_artifacts for latest revision images ( for self reflecting to correct any errors, we pass latest revision of the image again)
- Phase 5: Post-reflection
save_final_corrected_datapoints
- Phase 6: Return pre-generated validation tables via
LlmResponse
- In
before_model_callback, when post_reflection_saved=True, return:
from google.adk.models import LlmResponse
from google.genai.types import Content, Part
return LlmResponse(
content=Content(
role="model",
parts=[Part.from_text(text=validation_tables)]
),
turn_complete=True
)
- Observe that sometimes the LLM is still called and generates a free-form response instead of the deterministic tables and also in 99% of such cases, the second load_artifacts also gets failed. But I have found a case where even the second load_artifacts happend(in self-reflection) and stilll the final output was llm response generated by llm instead of the tables. In all these cases I could see the state for the tables correctly populated. Will there be any race conditions within the state read/writes?
Expected Behavior:
When before_model_callback returns an LlmResponse object with turn_complete=True, the LLM should never be called, and the response should be exactly what was returned from the callback.
Observed Behavior:
- Intermittently, the LLM is still invoked even after returning
LlmResponse
- The agent produces free-form LLM-generated text instead of the pre-generated validation tables
- This breaks determinism in workflows that require strict output formatting
Environment Details:
- ADK Library Version:
1.22.1
- Desktop OS: macOS (Darwin)
- Python Version:
3.12.10
Model Information:
- Are you using LiteLLM: Yes
- Which model is being used:
azure/gpt-5.1-mini (Azure OpenAI)
Google ADK Bug Report:
before_model_callbackLlmResponse Not Guaranteed to Skip LLM CallRequired Information
Describe the Bug:
When returning an
LlmResponsefrombefore_model_callbackto skip the LLM call and return a deterministic response, the behavior is inconsistent. The LLM call is sometimes still made, resulting in non-deterministic output instead of the pre-generated content.Additionally, the callback execution flow does not guarantee that returning
LlmResponsewill always prevent the LLM from being invoked - especially when other conditions in the callback may have already modifiedllm_request.config.tool_config.Steps to Reproduce:
before_model_callbackthat tracks state across multiple tool callssave_datapointstoolsave_final_corrected_datapointscall(we have multiple images from which different sections needs to be extracted and saved, so we have this extra final tool to give the final correct fields)load_artifactsfor latest revision images ( for self reflecting to correct any errors, we pass latest revision of the image again)save_final_corrected_datapointsLlmResponsebefore_model_callback, whenpost_reflection_saved=True, return:Expected Behavior:
When
before_model_callbackreturns anLlmResponseobject withturn_complete=True, the LLM should never be called, and the response should be exactly what was returned from the callback.Observed Behavior:
LlmResponseEnvironment Details:
1.22.13.12.10Model Information:
azure/gpt-5.1-mini(Azure OpenAI)