Repository navigation
Description
Where do you use Phoenix
Self-hosted
What version of Phoenix are you using?
main @ d0dc9e5ff0a33a1962da6aae83e6f56ac72c444f (__version__ = "20.21.0", src/phoenix/version.py). Python 3.12.14, openai 3.29.0 (repo pins openai>=3.1.0,<4, pyproject.toml:122), macOS 26.6.2 (arm64).
What happened?
In the playground, the OpenAI streaming path keys streamed tool-call fragments by their position inside the current delta (enumerate) instead of by the fragment's own index field. Parallel tool calls get garbled arguments, and a first fragment that arrives without an id crashes the whole playground run.
In OpenAICompatibleClient._chat_completion_create (src/phoenix/server/api/helpers/playground_clients.py:775-781):
if (tool_calls := delta.tool_calls) is not None:
for tool_call_index, tool_call in enumerate(tool_calls): # position inside THIS delta
tool_call_id = (
tool_call.id
if tool_call.id is not None
else tool_call_ids[tool_call_index] # KeyError if never seen
)
tool_call_ids[tool_call_index] = tool_call_idIn OpenAI's streaming protocol a delta normally carries a fragment for only one tool call, so the enumerate() position is almost always 0 no matter which call the fragment belongs to; delta.tool_calls[i].index is the field that identifies the call, and id is only sent on the first fragment of each call. Two consequences:
- Parallel tool calls get garbled arguments. Argument fragments for
index 0are attributed to whichever call id last landed at map key0. Reassembled byid— which is what both the frontend and the span do — one call ends up with empty arguments and the other gets both calls' fragments concatenated into invalid JSON. - A first fragment without an
idraisesKeyError: 0. The SDK typesChoiceDeltaToolCall.idasOptional[str], so a provider that starts a tool call without sendingidkills the entire playground run with the bare error message0.
Minimal reproduction (offline, no API key — synthetic ChatCompletionChunk deltas fed through _chat_completion_create):
import asyncio
from contextlib import asynccontextmanager
from typing import Any, AsyncIterator, Sequence
from phoenix.db.types.model_provider import LLMClientFactory
from phoenix.server.api.helpers.playground_clients import (
OpenAIChatCompletionsClient,
ToolCallChunk,
)
class FakeSpan:
def set_attribute(self, *a: Any, **k: Any) -> None:
pass
def set_attributes(self, *a: Any, **k: Any) -> None:
pass
def chunk(tool_calls: list[dict[str, Any]]) -> Any:
from openai.types.chat import ChatCompletionChunk
from openai.types.chat.chat_completion_chunk import (
Choice,
ChoiceDelta,
ChoiceDeltaToolCall,
ChoiceDeltaToolCallFunction,
)
return ChatCompletionChunk(
id="chatcmpl-x",
choices=[
Choice(
delta=ChoiceDelta(
tool_calls=[
ChoiceDeltaToolCall(
index=tc.get("index"),
id=tc.get("id"),
function=ChoiceDeltaToolCallFunction(
name=tc["function"].get("name"),
arguments=tc["function"].get("arguments"),
),
)
for tc in tool_calls
]
),
finish_reason=None,
index=0,
)
],
created=1700000000,
model="gpt-4o",
object="chat.completion.chunk",
)
def make_client(stream_chunks: Sequence[Any]) -> OpenAIChatCompletionsClient:
@asynccontextmanager
async def create_client() -> Any:
class S:
_client = None
yield S()
client = OpenAIChatCompletionsClient(
client_factory=LLMClientFactory(create=create_client, rate_limit_key=("openai",)),
model_name="gpt-4o",
provider="openai",
)
async def fake_stream(**kwargs: Any) -> Any:
async def gen() -> AsyncIterator[Any]:
for c in stream_chunks:
yield c
return gen()
client._openai_chat_completion_create_stream = fake_stream # type: ignore[method-assign]
return client
async def run(label: str, stream_chunks: Sequence[Any]) -> None:
print(f"=== {label} ===")
client = make_client(stream_chunks)
per_id: dict[str, str] = {}
try:
async for c in client._chat_completion_create(
messages=[{"role": "user", "content": "weather and time in Paris?"}],
tools=None,
response_format=None,
invocation_parameters=None,
span=FakeSpan(),
stream_model_output=True,
):
assert isinstance(c, ToolCallChunk)
per_id.setdefault(c.id, "")
per_id[c.id] += c.function.arguments
for tid, args in per_id.items(): # reassembled by id, as the frontend/span do
print(f" reassembled id={tid!r} arguments={args!r}")
except Exception as e:
print(f" CRASH: {type(e).__name__}: {e}")
print()
async def main() -> None:
await run(
"Scenario 1: interleaved parallel tool calls (ids first, then args)",
[
chunk([{"index": 0, "id": "call_A", "function": {"name": "get_weather"}}]),
chunk([{"index": 1, "id": "call_B", "function": {"name": "get_time"}}]),
chunk([{"index": 0, "function": {"arguments": '{"city": "Paris"}'}}]),
chunk([{"index": 1, "function": {"arguments": '{"tz": "CET"}'}}]),
],
)
print(" expected: call_A -> '{\"city\": \"Paris\"}' call_B -> '{\"tz\": \"CET\"}'\n")
await run(
"Scenario 1b: same, id deltas arriving in reverse order",
[
chunk([{"index": 1, "id": "call_B", "function": {"name": "get_time"}}]),
chunk([{"index": 0, "id": "call_A", "function": {"name": "get_weather"}}]),
chunk([{"index": 0, "function": {"arguments": '{"city": "Paris"}'}}]),
chunk([{"index": 1, "function": {"arguments": '{"tz": "CET"}'}}]),
],
)
await run(
"Scenario 2: first fragment arrives without an id (id: Optional[str])",
[chunk([{"index": 0, "function": {"name": "get_weather", "arguments": "{}"}}])],
)
if __name__ == "__main__":
asyncio.run(main())Actual output (verbatim at the commit above):
=== Scenario 1: interleaved parallel tool calls (ids first, then args) ===
reassembled id='call_A' arguments=''
reassembled id='call_B' arguments='{"city": "Paris"}{"tz": "CET"}'
expected: call_A -> '{"city": "Paris"}' call_B -> '{"tz": "CET"}'
=== Scenario 1b: same, id deltas arriving in reverse order ===
reassembled id='call_B' arguments=''
reassembled id='call_A' arguments='{"city": "Paris"}{"tz": "CET"}'
=== Scenario 2: first fragment arrives without an id (id: Optional[str]) ===
CRASH: KeyError: 0
The trace is corrupted too: _llm_output_messages (playground_clients.py:4268) reassembles from the same id-keyed map, so for Scenario 1 the span records
llm.output_messages.0.message.tool_calls.0.tool_call.function.name = 'get_weather'
llm.output_messages.0.message.tool_calls.0.tool_call.id = 'call_A'
llm.output_messages.0.message.tool_calls.1.tool_call.function.arguments = '{"city": "Paris"}{"tz": "CET"}'
llm.output_messages.0.message.tool_calls.1.tool_call.function.name = 'get_time'
llm.output_messages.0.message.tool_calls.1.tool_call.id = 'call_B'
i.e. call_A has no arguments at all and call_B carries invalid concatenated JSON.
Expected behavior: fragments should be keyed by delta.tool_calls[i].index — the field OpenAI's streaming protocol defines for exactly this purpose (the openai SDK requires ChoiceDeltaToolCall.index: int while id is Optional[str], so index is the reliable key and id is the optional one). A fragment that has no id and no recorded id for its index should get a synthesized id instead of raising KeyError.
This is also how the sibling branches in the same file already handle it:
- Bedrock keys active tool calls by
contentBlockIndex("Get the actual index"),playground_clients.py:1801-1840. - The Gemini branch synthesizes stable ids precisely because "The frontend merges streamed tool-call chunks by
id",playground_clients.py:2843-2853. - The frontend merge-by-id with argument concatenation is
addRepetitionPartialToolCallinjs/app/src/store/playground/playgroundStore.tsx:1303-1323.
Root cause: src/phoenix/server/api/helpers/playground_clients.py:775 — enumerate(tool_calls) yields the fragment's position within the current delta, which is unrelated to which tool call the fragment belongs to, and that position is then used both to look up and to store the id (:779-781). The map key should be the fragment's own index (falling back to the enumerate position when absent), and a missing id should synthesize one rather than raise.
This affects every playground client inheriting OpenAICompatibleClient whenever the model streams tool calls: OpenAI (playground_clients.py:1973), Azure (:2056), DeepSeek, xAI, Ollama, Cerebras, Fireworks, Groq, Moonshot, MiniMax, Perplexity, Together, ZAI and Meta (:1274-1491) — reached from every streamed playground chat completion (src/phoenix/server/api/subscriptions.py:75 → chat_completion_create at :105).
Additional information
- I searched open issues and did not find this reported. The closest are [BUG]: parallel_tool_calls is dropped when an OpenAI prompt is converted back to model kwargs #16329 (
parallel_tool_callsdropped when a prompt is converted back to model kwargs — a different layer) and [BUG]: gen_ai conversion keeps only the last of several tool_call_response parts #16916 (gen_ai conversion keeps only the last of severaltool_call_responseparts). - Happy to be assigned and open a small PR if maintainers want it: key the id map by the fragment's own
indexfield (falling back to the enumerate position when absent) and synthesize an id for a first fragment that arrives without one, mirroring the Gemini branch.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fields📘 Todo
Activity
BlueX888 commentedon Oct 12, 2026
Reproduced this. Root cause: src/phoenix/server/api/helpers/playground_clients.py:775 - enumerate(tool_calls) keyed the tool_call_ids map by the fragment's position within the current delta instead of by the fragment's own index field, so parallel tool-call fragments collided on position 0 and a first fragment without an id raised KeyError: 0
Minimal reproduction:
Observed output:
Happy to open a PR with this fix, or to be assigned if you prefer to take it.