Repository navigation
Python: tool_call_args_match accepts missing null-valued arguments #8558
Description
Description
tool_call_args_match reports a match when an expected argument has value None but the actual tool call omits that key. The documented subset match requires every expected key-value pair to be present. This can make LocalEvaluator approve a call that used a default argument instead of explicitly passing null.
Code Sample
from agent_framework import Content, EvalItem, ExpectedToolCall, Message, tool_call_args_match
item = EvalItem(
conversation=[
Message("assistant", [Content.from_function_call("call_weather", "get_weather", arguments={"location": "NYC"})])
],
expected_tool_calls=[ExpectedToolCall("get_weather", {"location": "NYC", "units": None})],
)
print(tool_call_args_match(item).passed) # True; expected False because units is absent.Error Messages / Stack Traces
Package Versions
agent-framework-core 1.19.0; reproduced on main at 7232569.
Python Version
Python 3.12.14 on Windows.
Additional Context
The same result occurs with JSON-string arguments. Explicit "units": null should still match, and unrelated extra arguments should remain allowed.
The subset check uses actual_args.get(k) == v, which treats an absent key like a present null value. I plan to add an explicit key-presence check and regression coverage for dictionary and JSON-string arguments.
Metadata
Metadata
Labels
Type
Projects
- Status
Activity
github-actions commentedon Sep 20, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
The defect is in
python/packages/core/agent_framework/_evaluation.py::tool_call_args_matchat line 1224. It occurs when an expected argument has valueNonebut the actual dictionary or parsed JSON object omits that key. Minimal repro: expect{"location": "NYC", "units": None}and evaluate a tool call containing only{"location": "NYC"}; the matcher incorrectly returnspassed=True.python/packages/core/tests/core/test_local_eval.py1.19.01.19.0