Repository navigation
Description
What happened, and what did you expect?
When a tool parameter is a @dataclass or TypedDict whose fields have defaults (or are NotRequired), the generated JSON schema correctly marks those fields as not required (cls_json_schema in util/_json.py). But when the model omits one of them in the call, tool_param in model/_call_tools.py does input.get(name) for every field and coerces the resulting None through the field's type. Depending on the field type the outcome is:
int/floatdefault →ToolParsingError: Unable to convert 'None' to intis sent back to the model, even though the call is schema-valid.strdefault → the default is silently replaced by the literal string"None". The tool runs with a wrong value and nothing signals it.list[...]withdefault_factory→TypeError: 'NoneType' object is not iterableis raised out ofexecute_tools, which errors the sample.
Expected: a schema-valid call succeeds and the dataclass/TypedDict defaults are applied, i.e. the same behaviour top-level parameters already get in tool_params, which checks if param_name in input and otherwise falls back to param.default. Dataclass and TypedDict parameters are exercised by tests/tools/test_call_tools.py (MyDataClass, MyTypedDict), and the schema emitted to the model explicitly tells it these fields may be omitted.
Steps to reproduce
import asyncio
from dataclasses import dataclass, field
from inspect_ai.model._call_tools import execute_tools
from inspect_ai.model._chat_message import ChatMessageAssistant
from inspect_ai.tool import ToolCall, tool
from inspect_ai.tool._tool_def import ToolDef
@dataclass
class Opts:
query: str
limit: int = 10
label: str = "default-label"
@tool
def search():
async def execute(opts: Opts) -> str:
"""Search.
Args:
opts: Search options.
"""
return repr(opts)
return execute
async def main():
tdef = ToolDef(search())
print("schema required:", tdef.parameters.properties["opts"].required)
call = ToolCall(id="1", function="search", arguments={"opts": {"query": "cats"}})
msgs, _ = await execute_tools([ChatMessageAssistant(content="", tool_calls=[call])], [tdef])
print("error:", msgs[-1].error, "| content:", msgs[-1].content)
asyncio.run(main())Actual output:
schema required: ['query']
error: ToolCallError(type='parsing', message="Unable to convert 'None' to int") | content:
Drop limit and keep only label: str = "default-label" and the output becomes content: Opts(query='cats', label='None') with no error. Replace it with tags: list[str] = field(default_factory=list) and execute_tools raises TypeError: 'NoneType' object is not iterable. A TypedDict with NotRequired[int] fields hits the same Unable to convert 'None' to int path.
With the model sent a schema whose required is ['query'], omitting limit is exactly what a well-behaved model will do.
Version and environment
- inspect-ai 0.3.263, installed editable from
mainat 7aa7343 - Python 3.13.15, macOS
Root cause
src/inspect_ai/model/_call_tools.py, tool_param, the is_typeddict and is_dataclass branches: both iterate over all declared fields and call tool_param(hint, input.get(name)), so missing keys become None and are coerced as if the model had passed null. Only keys present in input should be coerced. Dataclass fields with a default/default_factory and TypedDict keys not in __required_keys__ should be left out so the constructor applies the default. I have a local fix along those lines that makes the three cases above return Opts(query='cats', limit=10, label='default-label', tags=[]) and keeps tests/tools/test_call_tools.py and tests/tools/test_tool_def.py green. Happy to be assigned and open a PR once this is accepted.
Disclosure: the code audit and reproduction were done with AI assistance (Claude Code); the repro output above is from running the script on this machine.
Activity
he-yufeng commentedon Sep 4, 2026
Confirmed on current main, and the report nails the mechanism. Both nested branches of
tool_param()rebuild the object from the declared fields instead of the received ones:src/inspect_ai/model/_call_tools.py:1068):for name, hint in annotations.items(): typeddict_data[name] = tool_param(hint, input.get(name))— every key the model left out is coerced fromNone, so an omittedNotRequired[str]comes back as the string"None".src/inspect_ai/model/_call_tools.py:1074): same pattern over__dataclass_fields__, so an omittedvalue: float = 1.5raisesToolParsingError: Unable to convert 'None' to float, and an omittedlistfield dies in the constructor with aTypeError.The BaseModel branch right below doesn't have the problem because it calls
type_hint(**input)and lets the model apply defaults itself.Minimal repro against the tool-call executor:
Fix I have working locally: skip what the model omitted and let the declared defaults do their job. In the TypedDict branch, only coerce keys that are present or required (
__optional_keys__); in the dataclass branch, only coerce fields that are present or have no default/default_factory, then let the constructor fill the rest. Required fields that are missing still go through the old coercion and fail exactly as before, and Optional fields without defaults keep their current None-coercion behavior, so the only behavior change is for omitted fields that actually declare a default.Validation (tests added to
tests/tools/test_call_tools.py, red before the patch, green after):{"td": {"count": 3}, "dc": {}}->td == {"count": 3},dc == Cfg(value=1.5, label="default-label", numbers=[]){"td": {"count": 1, "label": "x"}, "dc": {"label": "custom", "numbers": [1, 2]}}-> supplied values win, remaining fields take defaultstests/tools/test_call_tools.pypasses on asyncio and trio (16 + 15 tests), ruff format + check clean, mypy clean on the changed fileHappy to open the PR with this once the issue is accepted — per the contribution policy I'm holding off until then.