Skip to content

before_model_callback LlmResponse Not Guaranteed to Skip LLM Call #4509

Description

@vishnu-MVP

Google ADK Bug Report: before_model_callback LlmResponse Not Guaranteed to Skip LLM Call

Required Information

Describe the Bug:
When returning an LlmResponse from before_model_callback to skip the LLM call and return a deterministic response, the behavior is inconsistent. The LLM call is sometimes still made, resulting in non-deterministic output instead of the pre-generated content.

Additionally, the callback execution flow does not guarantee that returning LlmResponse will always prevent the LLM from being invoked - especially when other conditions in the callback may have already modified llm_request.config.tool_config.

Steps to Reproduce:

  1. Create an agent with before_model_callback that tracks state across multiple tool calls
  2. Implement a multi-phase workflow:
    • Phase 1: Load artifacts (images)
    • Phase 2: Process artifacts with save_datapoints tool
    • Phase 3: First save_final_corrected_datapoints call(we have multiple images from which different sections needs to be extracted and saved, so we have this extra final tool to give the final correct fields)
    • Phase 4: Self-reflection with load_artifacts for latest revision images ( for self reflecting to correct any errors, we pass latest revision of the image again)
    • Phase 5: Post-reflection save_final_corrected_datapoints
    • Phase 6: Return pre-generated validation tables via LlmResponse
  3. In before_model_callback, when post_reflection_saved=True, return:
    from google.adk.models import LlmResponse
    from google.genai.types import Content, Part
    
    return LlmResponse(
        content=Content(
            role="model",
            parts=[Part.from_text(text=validation_tables)]
        ),
        turn_complete=True
    )
  4. Observe that sometimes the LLM is still called and generates a free-form response instead of the deterministic tables and also in 99% of such cases, the second load_artifacts also gets failed. But I have found a case where even the second load_artifacts happend(in self-reflection) and stilll the final output was llm response generated by llm instead of the tables. In all these cases I could see the state for the tables correctly populated. Will there be any race conditions within the state read/writes?

Expected Behavior:
When before_model_callback returns an LlmResponse object with turn_complete=True, the LLM should never be called, and the response should be exactly what was returned from the callback.

Observed Behavior:

  • Intermittently, the LLM is still invoked even after returning LlmResponse
  • The agent produces free-form LLM-generated text instead of the pre-generated validation tables
  • This breaks determinism in workflows that require strict output formatting

Environment Details:

  • ADK Library Version: 1.22.1
  • Desktop OS: macOS (Darwin)
  • Python Version: 3.12.10

Model Information:

  • Are you using LiteLLM: Yes
  • Which model is being used: azure/gpt-5.1-mini (Azure OpenAI)

Metadata

Metadata

Assignees

Labels

core[Component] This issue is related to the core interface and implementationrequest clarification[Status] The maintainer need clarification or more information from the authorstale[Status] Issues which have been marked inactive since there is no user response

Type

Fields

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions