FunctionNode events invisible in BigQueryAgentAnalyticsPlugin in graph workflow adk 2

🔴 Required Information

Describe the Bug:

FunctionNodes in an ADK 2 graph Workflow are invisible to BigQueryAgentAnalyticsPlugin. A workflow made of function nodes produces BigQuery rows for the invocation and for any Agent nodes, but not a single row that identifies a function node or carries its result.

Two independent causes, both narrow:

NodeRunner never dispatches plugin callbacks. rg plugin_manager src/google/adk/workflow/ returns zero matches across all 29 files. Agent nodes are covered only incidentally, because _llm_agent_wrapper.py:325 calls agent.run_async(ic) and the callbacks fire from inside BaseAgent. FunctionNode._run_impl has no equivalent path, so before_agent_callback / after_agent_callback / the tool callbacks never fire for it.
BigQueryAgentAnalyticsPlugin.on_event_callback ignores Event.output. Function-node events carry their payload in Event.output with Event.content unset (_base_node.py:179, Event(output=validated)). Every content path in on_event_callback reads event.content or event.actions; rg 'event\.output' src/google/adk/plugins/bigquery_agent_analytics_plugin.py returns zero matches. So the one hook that does fire for function nodes discards their payload.

The net effect is that a node emits an event, the plugin receives it, and nothing is written.

Steps to Reproduce:

pip install google-adk==2.3.0
Run the script under Minimal Reproduction Code below. It needs no GCP access — _log_event is stubbed to record the event_type of each row the plugin would write.
Compare the recorded rows against the two nodes that ran.

Expected Behavior:

A two-node workflow should produce BigQuery rows that identify step_one and step_two and carry their outputs, comparable to the AGENT_STARTING / AGENT_COMPLETED coverage an Agent node gets.

Observed Behavior:

agent/tool callbacks fired: NONE

on_event_callback saw:
  path='repro_wf@1/step_one@1'    output=True  content=False
  path='repro_wf@1/step_two@1'    output=True  content=False

BigQuery rows the plugin would write: ['USER_MESSAGE_RECEIVED', 'INVOCATION_STARTING', 'INVOCATION_COMPLETED']

Three rows, none referencing either node. The events reach on_event_callback with correct node_info.path, so the identity information is present and simply unused.

In a real workflow the only node-derived rows are incidental STATE_DELTA entries from nodes that happen to write ctx.state — nodes that only return a value are entirely absent.

Environment Details:

ADK Library Version: google-adk==2.3.0 (also verified against main @ 27548e39)
Desktop OS: macOS 15 (darwin 24.6.0)
Python Version: 3.13

Model Information:

Are you using LiteLLM: No
Which model is being used: N/A — the repro contains no LLM call

@wdai-chwy

Hello @wdai-chwy ,

Could you please provide a minimal reproducible example to help us investigate this issue further? Additionally, please try testing with google-adk==2.6.0 and let us know your findings with this version.

@llalitkumarrr

@llalitkumarrr

example below for function node including mcp tool call. we can see events in adk web, but it is not logged into big query.

async def get_patient_by_id(ctx: Context, node_input: dict[str, Any]) -> dict[str, Any]:
    patient_id = node_input.get("patientId") if isinstance(node_input, dict) else None

    patient = await call_mcp_tool(
        ctx,
        constants.GET_PATIENT_BY_ID_TOOL,
        patient_id=patient_id,
    )
    return patient

i checked 2.6, https://github.com/google/adk-python/releases/tag/v2.6.0, looks like nothing related to this issue.

We have to deploy to test env to check big query, not locally.

@haiyuan-eng-google

Hello @haiyuan-eng-google,

Could you please take a look at this?

I'm working on this issue for now, would keep you posted

Thanks for the unusually precise diagnosis — both causes confirmed at current HEAD, and your repro's observed output matches what the code does:

The workflow/ package contains no plugin dispatch at all; NodeRunner only opens a telemetry span. Agent nodes are covered incidentally through agent.run_async inside the LLM-agent wrapper.
The plugin's on_event_callback has no branch for event.output — the function-node events do reach the plugin (via the runner's event queue, which is also why adk web displays them), and are then dropped because they match no content path.

Two additional gaps we found while verifying, which your MCP example touches:

Node failure events (Event(error_code=...) from NodeRunner) are equally invisible — no branch matches them either.
Tool callbacks only fire from the LLM function-calling flow, so a tool invoked imperatively inside a FunctionNode body (your call_mcp_tool) produces no TOOL_STARTING/TOOL_COMPLETED rows regardless of the above. This one needs its own design and will be tracked separately.

Fix plan, in two stages:

Stage 1 (plugin-side, no framework change): handle output-bearing events in on_event_callback — non-partial events with output set produce a NODE_OUTPUT row carrying the (truncated) output payload plus the node identity the event already has (attributes.adk.node.path / run_id / parent_run_id — that enrichment plumbing already exists and just isn't triggered). A sibling NODE_ERROR row covers node failures. event_type is a plain STRING column, so existing tables need no schema migration; consumer views get the new types projected. This makes your step_one/step_two identifiable with their outputs.

Stage 2 (framework): add node lifecycle plugin callbacks (before_node_callback/after_node_callback/on_node_error_callback) dispatched from NodeRunner, giving NODE_STARTING/NODE_COMPLETED rows with latency, symmetric with the AGENT_* coverage Agent nodes get today.

I'll post progress here as the stages land.

Follow-up after reproducing this end-to-end on current main: a workflow with two FunctionNode outputs still produces only USER_MESSAGE_RECEIVED, INVOCATION_STARTING, and INVOCATION_COMPLETED rows in BigQuery.

A scope refinement to my previous proposal: the missing node-lifecycle callback API is real, but it is not required to fix this issue. NodeRunner already sends each emitted node Event through the plugin’s on_event_callback; the BigQuery plugin currently ignores the event output. Also, a FunctionNode is neither an Agent nor a Tool, so reusing agent/tool callbacks would give misleading semantics.

The focused fix should therefore be:

Emit NODE_OUTPUT from on_event_callback for a non-partial node event with output/content, using the existing event envelope and node path.
Emit NODE_ERROR for node error events.
Preserve one row per emitted output: an async-generator node may yield multiple outputs, so NODE_OUTPUT is more accurate than a single NODE_COMPLETED row.
Cover dict/list/scalar output, multiple yields, None, output plus state delta, node exceptions, and an agent-wrapped workflow regression case.
Verify that the existing payload/privacy filtering still applies.

Exact NODE_STARTING/NODE_COMPLETED lifecycle hooks should be a separate API design: their behavior around retries, resumes, short-circuiting, and multi-output nodes needs to be defined first. Imperatively calling a tool inside a FunctionNode is likewise a separate instrumentation gap, not something this fix should infer as a normal tool callback.

One implementation pitfall to design into Stage 1 before writing the NODE_OUTPUT branch:

message_as_output / _output_delegated dedup semantics. The runner clears output on event copies when a node's message is being delegated as output (runners.py:939-942, _node_runner.py:345-348). The branch must key off the event as seen at the queue consumer (runners.py:945) and skip events where message_as_output applies — otherwise an LlmAgent node's final text gets logged twice: once as AGENT_RESPONSE (the existing content path) and again as NODE_OUTPUT. The FunctionNode case the reporter hit has content=None and no delegation, so it falls cleanly into the new branch once that guard exists.

Test anchors: the A2A on_event_callback test class (tests/unittests/plugins/test_bigquery_agent_analytics_plugin.py:8093+) is the right template for the new branch; workflow-side fixtures live under tests/unittests/workflow/ (test_function_node.py, test_node_runner_integration.py).

On Stage 2 vocabulary: reusing before/after_agent_callback for deterministic nodes is a poor fit — FunctionNode is not a BaseAgent, so CallbackContext.agent_name semantics break (the plugin already works around this in _resolve_agent_label, :5603-5628). The node-native NODE_* family proposed above avoids that entirely and matches how the plugin already special-cases workflow-driven invocations.

Stage 1 implementation follow-up: the plugin-only fix is now in draft PR caohy1988#12 (commit 438367d3). No workflow/framework files are changed.

on_event_callback now emits:

NODE_OUTPUT for non-partial workflow-node events with content is None and non-None output;
NODE_ERROR for workflow-node events with error_code.

Both use the existing event envelope, so node path/run/parent identity and payload truncation/privacy behavior are preserved. State deltas remain separate rows. The content-less guard avoids double-logging agent/message output, including the message_as_output path discussed above. Each output event is handled independently, so multi-yield nodes naturally produce one row per yielded event.

Tests cover dict/list/scalar outputs, node identity, output plus state delta, errors, None, message-output deduplication, and typed NODE_OUTPUT/NODE_ERROR views. The complete plugin unit file passes: 477 tests. Node lifecycle callbacks and imperative in-node tool instrumentation remain explicitly out of scope.

fix(plugins): Capture workflow node results in BigQuery

@wdai-chwy

Correction / final status: the Stage 1 fix is at upstream HEAD. It merged to main on 2026-08-10 in 04b8b72 (feat(plugins): add BigQuery Agent Analytics delivery and termination observability), landing via the internal review + Copybara flow. My earlier note describing it as still pending on a fork draft was out of date — apologies for the confusion.

What shipped (re-verified line-by-line at current HEAD):

on_event_callback now emits NODE_OUTPUT for final (non-partial) workflow-node events carrying output, and NODE_ERROR for node events carrying an error_code. Both reuse the existing envelope, so node_path / node_run_id / node_parent_run_id land in typed view columns, and output payloads go through the standard truncation/privacy pipeline. Multi-yield nodes produce one row per emitted output.
The dedup guards discussed above are in place: partial events are skipped; message_as_output delegation is excluded (an LlmAgent node's final text is not double-logged as both AGENT_RESPONSE and NODE_OUTPUT); model finish/block codes are excluded from NODE_ERROR, so an LLM SAFETY stop stays classified as LLM_RESPONSE rather than a node failure.

With this, the repro from the issue description produces identifiable rows for step_one / step_two with their outputs, and @wdai-chwy's function node produces a NODE_OUTPUT row carrying the returned patient payload.

Availability: the merge postdates v2.6.3, so it ships in the next release; until then it's available from main.

Closing this issue as fixed, since the reported defect — function nodes invisible to the plugin — is resolved. Two follow-ups remain tracked separately and were deliberately out of scope here: (1) explicit node lifecycle boundary rows (NODE_STARTING/completion with latency, symmetric with AGENT_* coverage), in progress; (2) instrumentation for tools invoked imperatively inside a node body (the call_mcp_tool case), which needs its own design. Please open/watch follow-up issues for those, and reopen here if the shipped behavior doesn't cover your workflow.

Follow-up clarification: lifecycle boundary instrumentation has been narrowed into RFC haiyuan-eng-google/BigQuery-Agent-Analytics-SDK#98. The RFC proposes gated event-observation windows, not exact node start/completion or FunctionNode wall-time. Exact lifecycle events remain deferred pending node-native framework callbacks.

Navigation Menu

Repository navigation

FunctionNode events invisible in BigQueryAgentAnalyticsPlugin in graph workflow adk 2 #6529

🔴 Required Information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

FunctionNode events invisible in BigQueryAgentAnalyticsPlugin in graph workflow adk 2 #6529

Description

🔴 Required Information

Activity

llalitkumarrr commented on Aug 3, 2026

wdai-chwy commented on Aug 3, 2026

llalitkumarrr commented on Aug 5, 2026

haiyuan-eng-google commented on Aug 5, 2026

caohy1988 commented on Aug 5, 2026

caohy1988 commented on Aug 5, 2026

caohy1988 commented on Aug 5, 2026

caohy1988 commented on Aug 5, 2026

caohy1988 commented on Aug 11, 2026

caohy1988 commented on Aug 11, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions