Repository navigation
Conversation
ReActAgent.finalize applied the ReAct parser cleanup (strip everything up to and including the first "Answer:") to every response. The return_direct path in aggregate_tool_results builds that response straight from the tool output, so a tool result containing "Answer:" was silently truncated before the caller saw it. Skip the cleanup when the final response text was fabricated from a return_direct tool observation instead of produced by ReActOutputParser. Handoff tools are also return_direct, but they are followed by a fresh LLM response that differs from the handoff observation, so requiring the final text to match the observation keeps the cleanup active for that case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ReActAgent.finalizestrips everything up to and including the first'Answer:'from the final response text, assuming that text was parsed from an LLM turn byReActOutputParser. On thereturn_directpath, however,aggregate_tool_results(base_agent.py) builds the final response straight from the tool output, so areturn_directtool whose output contains'Answer:'(e.g.Sources:\n doc-1: pricing faq\nAnswer: 42 dollars per seat) is silently truncated to42 dollars per seatbefore the caller sees it.This skips the cleanup when the final response text was fabricated from a
return_directtool observation rather than an LLM turn. The skip additionally requires the final text block to match the observation exactly: handoff tools are alsoreturn_direct=True, but after a handoff the next agent produces a fresh LLM response that differs from the handoff observation, so the'Answer:'cleanup still applies to real LLM answers in multi-agent workflows.New Package?
Did I fill in the
tool.llamahubsection in thepyproject.tomland provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)Type of Change
How Has This Been Tested?
Two new deterministic unit tests (mock LLMs, no network) in
llama-index-core/tests/agent/workflow/test_react_agent.py:test_react_agent_return_direct_preserves_answer_substring: areturn_directtool whose output contains'Answer:'is returned verbatim.test_react_agent_handoff_still_strips_answer_scaffolding: agent A hands off to agent B (handoff tools are alsoreturn_direct); B answers withThought: ... / Answer: ...text and the final response must keep the cleanup (B final answer 42, no raw scaffolding).Commands and verbatim summaries (run in
llama-index-corewithllama-index-workflows==2.23.3):python -m pytest tests/agent/workflow/test_react_agent.py -q->3 passed, 69 warnings in 0.09sgit checkout HEAD~1 -- llama-index-core/llama_index/core/agent/workflow/react_agent.py):test_react_agent_return_direct_preserves_answer_substringfails on the original bug:AssertionError: assert '42 dollars per seat' == 'Sources:\n ...lars per seat'->1 failed, 2 passed, 69 warnings in 0.20s. The handoff test passes on the unpatched tree (it guards the fix, not the original bug).return_directobservation (dropping the text-match condition, which would wrongly cover handoff tools) the handoff test fails:AssertionError: assert 'Thought: I k...nal answer 42' == 'B final answer 42'->1 failed, 2 passed, 69 warnings in 0.15s.python -m pytest tests/agent/workflow/ -q->67 passed, 3 skipped, 1890 warnings in 0.47suvx [email protected] check <changed files>->All checks passed!;uvx [email protected] --check <changed files>->2 files would be left unchanged.;pre-commit run --files <changed files>(ruff, ruff-format, mypy, codespell, whitespace hooks) -> all Passed.Suggested Checklist:
uv run make format; uv run make lintto appease the lint gods