Summary
When the active provider returns composite tool_call ids of the form call_xxx|fc_yyy (OpenAI-compatible gateways that bridge Chat Completions to the Responses API encode the function_call item id after a |), every tool result is silently dropped before the follow-up LLM call. The tool executes (side effects happen on the host), but the model only ever receives the sanitizer stub:
[Result unavailable — see context summary above]
The model then either reports the result as unavailable or — far worse — confabulates a plausible-looking tool output (in our case it consistently "quoted" a stale number from its hindsight memory as the raw output of a grep -c it never saw). This is a dangerous silent failure: the agent's answers look verified but are invented.
Environment
- Hermes 0.16.0 (
v2026.6.5-519-g5a4297a11), bug confirmed still present on current main (2026-07-11, v2026.7.7.2) by code inspection — see line refs below
- Provider:
custom: provider → LiteLLM proxy → OpenAI-compatible gateway that bridges Chat Completions to the Responses API (api_mode: chat_completions)
- Platform: gateway / api_server; Linux, Python 3.12
Root cause (verified end-to-end with a logging proxy between Hermes and the provider)
The composite id is split in one place but not the other, and the strict id matching in the repair pass then eats the result:
- Assistant side (short id):
agent/chat_completion_helpers.py splits the raw id via _split_responses_tool_id and persists the assistant tool_calls entry with id/call_id = short call_xxx (the fc_yyy part goes to response_item_id).
- Tool-result side (raw composite id):
agent/tool_executor.py (line 1656 on current main) builds the result message with the raw SDK object id:
tool_message = make_tool_result_message(function_name, _tool_content, tool_call.id)
→ tool_call_id = call_xxx|fc_yyy.
- Repair pass drops it:
agent/agent_runtime_helpers.py → repair_message_sequence pass 1 collects known_tool_ids from the assistant tool_calls dicts (short ids) and exact-matches msg["tool_call_id"] against them. The composite id never matches → the fresh tool result is classified as a stray orphan and removed.
- Sanitizer stubs it:
sanitize_api_messages then sees an assistant tool_call with no matching result and injects the [Result unavailable — see context summary above] stub, which is what the model receives.
The same raw-id append also happens on the error paths in agent/conversation_loop.py (invalid tool name / invalid JSON / execution error), so those messages are dropped identically.
Reproduction
Any OpenAI-compatible upstream whose tool_calls[].id differs from the internally normalized call_id triggers it — a |-composite id is the concrete case:
- Configure a
custom_providers entry pointing at a gateway that returns ids like call_abc|fc_def (e.g. a Chat-Completions→Responses bridge behind LiteLLM).
- Ask the agent to run any tool, e.g.
terminal with grep -c foo somefile | tee /tmp/out.txt.
/tmp/out.txt contains the real output (tool ran), but the outbound follow-up request contains the stub instead of the result (easily verified with a local logging proxy as base_url), and the model answers from priors.
Note the provider itself is fine: replaying the identical two-turn exchange manually against the same gateway (both with the composite id and with only the call_ part echoed back, streaming and non-streaming) works correctly. The drop happens entirely inside Hermes between tool execution and the next API call.
Suggested fix
Normalize at the source so the result message always carries the same id shape as the persisted assistant message, e.g. in make_tool_result_message:
if isinstance(tool_call_id, str) and "|" in tool_call_id:
tool_call_id = tool_call_id.split("|", 1)[0]
plus (defense in depth) tolerate both variants in repair_message_sequence pass 1 and in sanitize_api_messages. We are running exactly this 3-point patch locally and it fully restores tool round-trips; short ids are accepted end-to-end by the bridge (response_item_id already carries the fc_ part separately for the Codex adapter).
Possibly related
#55626 reports the same symptom ([Result unavailable] for tools that demonstrably executed, fresh session, OpenRouter) with no root cause identified — a provider-specific id-shape mismatch in the same exact-match pipeline would explain its "some tools work, some never do" pattern.
Summary
When the active provider returns composite tool_call ids of the form
call_xxx|fc_yyy(OpenAI-compatible gateways that bridge Chat Completions to the Responses API encode thefunction_callitem id after a|), every tool result is silently dropped before the follow-up LLM call. The tool executes (side effects happen on the host), but the model only ever receives the sanitizer stub:The model then either reports the result as unavailable or — far worse — confabulates a plausible-looking tool output (in our case it consistently "quoted" a stale number from its hindsight memory as the raw output of a
grep -cit never saw). This is a dangerous silent failure: the agent's answers look verified but are invented.Environment
v2026.6.5-519-g5a4297a11), bug confirmed still present on currentmain(2026-07-11,v2026.7.7.2) by code inspection — see line refs belowcustom:provider → LiteLLM proxy → OpenAI-compatible gateway that bridges Chat Completions to the Responses API (api_mode: chat_completions)Root cause (verified end-to-end with a logging proxy between Hermes and the provider)
The composite id is split in one place but not the other, and the strict id matching in the repair pass then eats the result:
agent/chat_completion_helpers.pysplits the raw id via_split_responses_tool_idand persists the assistanttool_callsentry withid/call_id= shortcall_xxx(thefc_yyypart goes toresponse_item_id).agent/tool_executor.py(line 1656 on current main) builds the result message with the raw SDK object id:tool_call_id=call_xxx|fc_yyy.agent/agent_runtime_helpers.py→repair_message_sequencepass 1 collectsknown_tool_idsfrom the assistanttool_callsdicts (short ids) and exact-matchesmsg["tool_call_id"]against them. The composite id never matches → the fresh tool result is classified as a stray orphan and removed.sanitize_api_messagesthen sees an assistanttool_callwith no matching result and injects the[Result unavailable — see context summary above]stub, which is what the model receives.The same raw-id append also happens on the error paths in
agent/conversation_loop.py(invalid tool name / invalid JSON / execution error), so those messages are dropped identically.Reproduction
Any OpenAI-compatible upstream whose
tool_calls[].iddiffers from the internally normalizedcall_idtriggers it — a|-composite id is the concrete case:custom_providersentry pointing at a gateway that returns ids likecall_abc|fc_def(e.g. a Chat-Completions→Responses bridge behind LiteLLM).terminalwithgrep -c foo somefile | tee /tmp/out.txt./tmp/out.txtcontains the real output (tool ran), but the outbound follow-up request contains the stub instead of the result (easily verified with a local logging proxy asbase_url), and the model answers from priors.Note the provider itself is fine: replaying the identical two-turn exchange manually against the same gateway (both with the composite id and with only the
call_part echoed back, streaming and non-streaming) works correctly. The drop happens entirely inside Hermes between tool execution and the next API call.Suggested fix
Normalize at the source so the result message always carries the same id shape as the persisted assistant message, e.g. in
make_tool_result_message:plus (defense in depth) tolerate both variants in
repair_message_sequencepass 1 and insanitize_api_messages. We are running exactly this 3-point patch locally and it fully restores tool round-trips; short ids are accepted end-to-end by the bridge (response_item_idalready carries thefc_part separately for the Codex adapter).Possibly related
#55626 reports the same symptom (
[Result unavailable]for tools that demonstrably executed, fresh session, OpenRouter) with no root cause identified — a provider-specific id-shape mismatch in the same exact-match pipeline would explain its "some tools work, some never do" pattern.