Repository navigation
Python: [Bug]: AG-UI history replay can send invalid assistant/tool sequence to OpenAI (tool_calls without matching tool messages) #5855
Description
Description
When using agent_framework.ag_ui with persisted thread history, later requests can intermittently fail with OpenAI/Azure validation errors:
An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'.
The issue appears related to replay/history reconstruction logic where assistant tool-call messages and tool-result messages become inconsistently paired during outbound payload generation.
Code Sample
Error Messages / Stack Traces
{
"error": {
"message": "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_xxx",
"type": "invalid_request_error",
"param": "messages.[N].role"
}
}Package Versions
agent_framework_ag_ui: 1.0.0b260507
Python Version
No response
Additional Context
Observed History Behavior
Persisted history files may contain both:
- assistant tool-call messages
- matching tool-result messages
However, during replay/reconstruction, outbound model payloads can still contain:
- unmatched assistant
tool_calls - reordered tool messages
- duplicated call entries
- missing tool-result messages
This eventually causes provider-side validation failures.
Expected Behavior
Framework-generated outbound payloads should always satisfy provider message contracts:
- Every assistant
tool_callmust have a matching tool-result message. - Tool messages must appear immediately after their originating assistant tool-call message.
- No orphan tool messages should exist.
- History replay and deduplication should remain deterministic under:
- retries
- interruptions
- resumed execution flows
Actual Behavior
Under some replay/history scenarios, invalid message sequences are sent to the model provider, resulting in hard 400 failures and unusable conversation threads.
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
github-actions commentedon May 14, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
Repro: _sanitize_tool_history() in python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py has two defects. (1) Line 76 unconditionally overwrites pending_tool_call_ids when a new assistant tool_call message is encountered, so consecutive assistant messages with tool_calls leave the first one's calls orphaned without synthetic results. (2) The function returns at line 197 without injecting synthetic results for any remaining pending_tool_call_ids, so history ending with an assistant tool_call message produces orphaned calls. Minimal repro: call _sanitize_tool_history with [assistant(tool_calls=[call_1]), assistant(tool_calls=[call_2]), user("hi")] and verify call_1 has no matching tool result in the output.
python/packages/ag-ui/tests/ag_ui/test_issue_5855_tool_history_replay.py1.0.0b2605071.0.0rc1norika1207-lab commentedon Jun 6, 2026
The history sanitizer should probably enforce the provider message contract as an invariant, not as a best-effort cleanup.
Before any outbound OpenAI/Azure payload is sent, the reconstructed sequence should be validated as:
tool_call.idhas exactly one following tool messageThe dangerous case is synthetic repair hiding a real incomplete run. It can prevent a 400, but the runtime should still expose that a tool result was missing or reconstructed. Otherwise the next model call may continue from a transcript that is schema-valid but operationally false.
lordlinus commentedon Jun 15, 2026
Still reproduces on the latest release
agent-framework-ag-ui==1.0.0rc4(withagent-framework-openai==1.8.1) —_message_adapters._sanitize_tool_history()is unchanged from the beta this was filed against, so a replayed assistantfunction_callwith no matching tool result still produces:Two observations from testing against Azure OpenAI Chat Completions on rc4:
_sanitize_tool_history()(~line 76 unconditional overwrite ofpending_tool_call_ids, and ~line 197 early return without injecting synthetic results).Related but distinct: the HITL approve → replay variant (an approved
approval_mode="always_require"tool whose executed result isn't journaled back into the snapshot) is #6513.Happy to help validate a fix.