Skip to content

fix(google_genai): publish AFC tool calls as output messages - #3798

Merged
caroger merged 6 commits into
Arize-ai:mainfrom
AbarnaaSree:fix/3788-google-genai-afc-output-messages
Nov 5, 2026
Merged

fix(google_genai): publish AFC tool calls as output messages#3798
caroger merged 6 commits into
Arize-ai:mainfrom
AbarnaaSree:fix/3788-google-genai-afc-output-messages

Conversation

@AbarnaaSree

@AbarnaaSree AbarnaaSree commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #3788.

Automatic function calling (AFC) tool calls from the Google GenAI instrumentation were emitted without the llm.output_messages.<messageIndex> prefix, producing orphaned message.tool_calls.* attributes.

This change publishes AFC tool calls under the appropriate output-message bucket and keeps tool-call indices scoped to each model message.

Changes

  • Prefix AFC tool-call attributes with llm.output_messages.<messageIndex>.
  • Reset tool-call indexing for each model message.
  • Skip request-seeded AFC history that belongs to input messages.
  • Avoid publishing candidate content twice when the same Content object appears in AFC history.
  • Use the Google GenAI SDK's content transformation when determining the seeded AFC history boundary.
  • Add regression tests covering multiple AFC model turns, duplicated candidate history, seeded request history, and merged request parts.

Testing

  • python -m pytest .\python\instrumentation\openinference-instrumentation-google-genai\tests\test_response_attributes_extractor.py -q
  • Result: 6 passed
  • git diff --check passes.

@AbarnaaSree
AbarnaaSree requested a review from a team as a code owner September 20, 2026 07:22
@github-actions

github-actions Bot commented Sep 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@AbarnaaSree

AbarnaaSree commented Sep 20, 2026

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

@AbarnaaSree

AbarnaaSree commented Sep 20, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@AbarnaaSree

AbarnaaSree commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

Hi, I’ve signed the CLA and triggered the recheck. The CLA workflow is currently waiting for maintainer approval. Thanks!

@mikeldking Claude

mikeldking commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Hi @AbarnaaSree, thanks for the AFC fix, and sorry for the slow response. Two small things are blocking it:

  1. The CLA check is still failing. Your signing comment ended with a line break, and the CLA workflow only accepts an exact match, so the signature was never recorded. Could you post a new comment with exactly I have read the CLA Document and I hereby sign the CLA, with nothing after it?
  2. Running ruff format (0.9.2, the version pinned in python/dev-requirements.txt) in the google-genai package would reformat _response_attributes_extractor.py and tests/test_response_attributes_extractor.py, so Python CI will fail until you push the formatted files.

Generated by Claude Code

github-actions Bot added a commit that referenced this pull request Oct 13, 2026
@github-actions
@AbarnaaSree has signed the CLA in #3798
555ae19
@AbarnaaSree

AbarnaaSree commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

Hi @mikeldking, thanks for the feedback! I've pushed the Ruff formatting changes, and the CLA check is now passing. The targeted regression tests pass locally (6 passed). I'll keep an eye on the CI checks. Thanks for your help!

@satyadevai
satyadevai force-pushed the fix/3788-google-genai-afc-output-messages branch from eba3178 to 6000c84 Compare October 22, 2026 08:44
@satyadevai

satyadevai commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Verified the changes with live examples using a local Phoenix instance. Each run generated one GenerateContent LLM span with status OK.

Existing vs Updated Behavior

Check Existing (main) Updated (This Branch)
Tool call attributes Stored at root level (message.tool_calls.*) Stored under llm.output_messages.1.message.tool_calls.*
Tool call message No separate output message Separate output message created for tool calls
Message role Not set for tool calls Set to model
Tool calls in Phoenix Not visible All 4 tool calls visible
Function arguments Stored at root level Moved under the corresponding output message
Reasoning signature Stored at root level Moved with the corresponding tool call
Final text response Present in llm.output_messages.0 Unchanged
Other span attributes Existing values Unchanged

Summary

Previously, tool calls were stored at the root level and were not visible in Phoenix.

With this change, tool calls are grouped under a separate output message (llm.output_messages.1) with the model role, making them visible in Phoenix.

The final text response and all other span attributes remain unchanged.

@mikeldking mikeldking self-assigned this Oct 22, 2026
@satyadevai

satyadevai commented Oct 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

I ran live Gemini examples against a local Phoenix with both builds: main and this branch. Both used google-genai 2.32.0 and gemini-3.5-flash. All spans returned status OK.

Scenario main This branch Result
Text, no tools (sync + async) no tool calls no tool calls ✅ unchanged
Text + tools (sync + async) message.tool_calls.* (no message prefix) llm.output_messages.1.message.tool_calls.* ✅ fixed
Image + text + tools (sync + async) message.tool_calls.* (no message prefix) llm.output_messages.1.message.tool_calls.* ✅ fixed
Stream + tools (sync + async) llm.output_messages.0.message.tool_calls.* llm.output_messages.0.message.tool_calls.* ✅ unchanged

Other checks

  • Apart from the tool-call keys, every scenario has identical attribute keys on both sides.
  • The fix also works on the oldest supported SDK, google-genai 2.0.1 (non-streaming calls, text and image, sync and async).

@satyadevai
satyadevai requested a review from caroger October 22, 2026 14:42
satyadevai and others added 3 commits October 22, 2026 20:57
@satyadevai
test(google-genai): assert afc tool calls in recorded test
36df0c3
@caroger
Merge branch 'main' into fix/3788-google-genai-afc-output-messages
b9eb0d0
@caroger
fix(google-genai): keep text and reasoning parts of AFC model turns
d0ffb30 
The AFC history loop rendered each model turn by hand and kept only its
function calls. A turn with "Let me check." and a get_weather call reached
the span as a bare tool call, and a thought part before a call was dropped
the same way. The loop now skips the request-seeded turns and the turns
already published as candidates, then renders each remaining model turn
with _get_attributes_from_generate_content_content, the path candidates
use. AFC turns now record their text and reasoning next to their tool calls.

_request_content_count no longer re-checks that request_parameters is a
Mapping. Its signature already says so, and every caller passes the dict
from _parse_args.
@caroger

caroger commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Thanks @AbarnaaSree. I pushed two commits to this branch on top of your work, b9eb0d0 and d0ffb30.

Merged main (b9eb0d0). main has since landed #3995 and #4230, which change how this package handles tuple contents and make the import safe with Pillow installed. The merge had no conflicts. CI on this branch now runs against current main, including the Pillow that test-requirements.txt now installs.

AFC model turns keep their text and reasoning (d0ffb30). The AFC history loop built each model turn's attributes by hand and kept only the function calls. When the model answered with "Let me check." plus a get_weather call, llm.output_messages.1 held the tool call and lost the text. A thought part before the call was dropped the same way. The loop still skips the request-seeded turns and the turn already published as a candidate. It now renders each remaining model turn with _get_attributes_from_generate_content_content, the same path the candidates use. So an AFC turn records its text, reasoning and tool calls exactly as a candidate would. test_afc_model_turn_keeps_its_text_and_reasoning_parts fails without this change and passes with it.

_request_content_count checks Mapping once (d0ffb30). It dropped the isinstance(request_parameters, Mapping) guard. The signature already takes a Mapping, and every caller passes the dict from _parse_args, so behavior is unchanged.

@caroger
caroger merged commit 5878e0b into Arize-ai:main Nov 5, 2026
25 checks passed
@openinference-factory openinference-factory Bot mentioned this pull request Nov 5, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@caroger caroger caroger approved these changes
+1 more reviewer
@satyadevai satyadevai satyadevai approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@mikeldking mikeldking

Labels

None yet

Projects

Instrumentation
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[bug] google-genai: automatic-function-calling tool calls are emitted outside every llm.output_messages bucket

4 participants

@AbarnaaSree @mikeldking @satyadevai @caroger