Skip to content

[BUG]: [playground] AWS Bedrock run crashes with JSONDecodeError when a tool result message contains plain text #16950

Closed

Description

@BlueX888

Where do you use Phoenix

Self-hosted

What version of Phoenix are you using?

Current main (after arize-phoenix 20.21.1)

What happened?

Running a playground prompt against an AWS Bedrock model fails immediately with json.decoder.JSONDecodeError whenever the prompt contains a TOOL message (a tool result) whose content is not valid JSON — which is the normal case for tool output.

BedrockClient._build_converse_messages builds every tool result as a JSON block, so the request cannot even be serialized (main, src/phoenix/server/api/helpers/playground_clients.py lines 1931-1933):

tool_result = ToolResultBlockTypeDef(
    toolUseId=tool_call_id or "",
    content=[ToolResultContentBlockTypeDef(json=json.loads(content))],
)

PlaygroundMessage content is a plain string, and tool results are usually plain text (e.g. "72 degrees and sunny"), so json.loads raises before any request is sent to AWS.

Minimal reproduction (no AWS credentials needed):

from contextlib import asynccontextmanager
from typing import Any

from phoenix.db.types.model_provider import LLMClientFactory
from phoenix.server.api.helpers.message_helpers import create_playground_message
from phoenix.server.api.helpers.playground_clients import BedrockClient
from phoenix.server.api.types.ChatCompletionMessageRole import ChatCompletionMessageRole


@asynccontextmanager
async def create_client() -> Any:
    yield None


client: Any = BedrockClient(
    client_factory=LLMClientFactory(create_client, ("aws", "test")),
    model_name="anthropic.claude-3-5-sonnet-20240620-v1:0",
    provider="aws",
)
client._build_converse_messages(
    [
        create_playground_message(
            ChatCompletionMessageRole.TOOL,
            "72 degrees and sunny",  # any non-JSON tool output
            tool_call_id="tooluse_1",
        )
    ]
)
json.decoder.JSONDecodeError: Extra data: line 1 column 4 (char 3)

Expected: the tool result should be forwarded to Converse as text and the run should proceed, like the other provider clients do — the OpenAI client passes content straight into ChatCompletionToolMessageParam(content=content) and the Anthropic client into ToolResultBlockParam(content=content or ""). Bedrock's ToolResultContentBlock accepts a text field, so the string can be sent as-is.

Additional information

Reproduced on current main through BedrockClient._converse_build_request, which the playground chatCompletion subscription calls before every Converse request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triageissues that need triage

    Type

    No type

    Projects

    • Status
      ✅ Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions