Repository navigation
Conversation
…stream In stream_chat/astream_chat, cur_tool_call was never reset when a tool block ended. A following server-side tool block (e.g. web_search via the tools/mcp_servers args) streams its input as InputJSONDelta, which then fell through to the client tool-call branch because cur_tool_call still pointed at the finished client ToolUseBlock. The server tool's partial JSON was appended to the stale cur_tool_json and, when it parsed, silently replaced the completed client tool call's input. Reset cur_tool_call when the tool block stops so that the existing server-side tool guard recognizes these deltas and ignores them.
…rrupted-by-server-tool-delta Resolve pyproject.toml version conflict: - upstream main bumped llama-index-llms-anthropic to 0.12.2 (run-llama#23305, Claude Sonnet 5.5 support, already published to PyPI), our branch had bumped it to 0.12.1 for this fix - take upstream's 0.12.2 as the new baseline and bump to 0.12.3 so this bug fix ships as a fresh release base.py auto-merged cleanly; both cur_tool_call = None resets in stream_chat/astream_chat (the actual fix for stale server-tool input_json_delta corrupting completed client tool calls) are intact, as are the added regression tests in tests/test_llms_anthropic.py.
There was a problem hiding this comment.
Community review (not a maintainer, does not clear the merge gate). I read the full diff and ran it.
What I ran, with anthropic 1.13.0 and the integration installed editable from this checkout:
- The two new tests (
test_stream_chat_server_tool_delta_does_not_corrupt_client_tool_calland the async one) pass on the PR branch. - I put the PR's tests on top of main's
base.py. Both fail there: the client tool callget_current_timeends up withtool_kwargs == {'query': 'weather in London'}, which is the web_search input from the later server tool block. So the bug is real on main and the two-line change fixes it. - The full
llama-index-llms-anthropictest directory on the PR branch: 54 passed, 15 skipped.
Why it happens, as I read it: cur_tool_call stays set to the finished ToolUseBlock after ContentBlockStopEvent. A later InputJSONDelta from a ServerToolUseBlock therefore passes the isinstance(cur_tool_call, ToolUseBlock) check, gets appended to cur_tool_json, and parse_partial_json writes the result into the old block's input. Resetting cur_tool_call = None at block stop in both gen() functions (sync and async) closes that.
One scoping note from my own probe, so the description is not read too broadly: I also tried a client tool that streams its arguments ({"a": 1} via InputJSONDelta) followed by a server tool. Main did not corrupt that case. I read the reason as cur_tool_json becoming {"a": 1}{"query": "x"}, which parse_partial_json should reject, with the except ValueError: pass keeping the earlier input, but I did not instrument that. So the corruption on main is limited to a client tool with empty input, which is what the PR test uses. Two client tools in a row, and a client tool followed by a text block, gave the same output on main and on the PR.
I did not run against the live API. The events are hand-built from the anthropic types. No issues found with the change.
|
Updated the description to record the scope you found: the overwrite only hits a client tool call that streamed no arguments; when the client tool streamed its own arguments the combined buffer fails |
Description
stream_chat/astream_chatinllama-index-llms-anthropicnever resetcur_tool_callwhen a tool block ended. When a streamed turn contains a clienttool_useblock followed by a server-side tool block (e.g.web_search_20250305passed viatools, ormcp_servers), the finished clientToolUseBlockstayed assigned tocur_tool_call, so the guard that is meant to ignore server-toolinput_json_deltaevents (llama_index/llms/anthropic/base.py:562-566) did not fire. The server tool's partial JSON was appended to the stalecur_tool_jsonand written back viacur_tool_call.input(base.py:567-579on main). For a client tool call that streamed no arguments the write succeeds and the finished call ends up with the server tool's arguments; for one that streamed its own arguments the combined buffer does not parse and theexcept ValueError: passkeeps the earlier input. The non-streaming path (_get_blocks_and_tool_calls_and_thinking) is not affected.The fix resets
cur_tool_call = Noneonce the finished call has been appended, in theContentBlockStopEventbranch of both the sync (base.py:615) and async (base.py:873) generators, so the existing guard recognizes the server tool'sinput_json_deltaand ignores it. Client tool calls now keep the input the API delivered.Fixes #23459
New Package?
Did I fill in the
tool.llamahubsection in thepyproject.tomland provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)Type of Change
How Has This Been Tested?
Two mock-based regression tests (
test_stream_chat_server_tool_delta_does_not_corrupt_client_tool_call,test_astream_chat_server_tool_delta_does_not_corrupt_client_tool_call) stream a clienttool_useblock followed by aserver_tool_useblock whose input arrives asInputJSONDelta, and assert the client tool call keeps its original arguments (tool_kwargs == {}). Both tests fail on currentmain:Commands run from
llama-index-integrations/llms/llama-index-llms-anthropic:python -m pytest tests/test_llms_anthropic.py -k server_tool_delta_does_not_corrupt— 2 passed (2 failed before the fix)python -m pytest— 54 passed, 15 skippedAt the repo root:
uv run make formatanduv run make lint— all pre-commit hooks passed (ruff, ruff-format, mypy, codespell, etc.).Suggested Checklist:
uv run make format; uv run make lintto appease the lint gods