Skip to content

[BUG]: Anthropic prompt adapter ignores {{ variables }} in str system messages, sending the literal template as the system kwarg #16228

Closed

Description

@BlueX888

Where do you use Phoenix

Self-hosted (SDK / library use — arize-phoenix-client)

What version of Phoenix are you using?

main at commit e12748298b366605cc0e2aa60e659f473efbff99 (arize-phoenix-client, package version is dynamic — no released pin applies)

What happened?

The Anthropic prompt adapter leaves {{ variables }} unsubstituted when a stored system message content is a plain str. The sibling branches in the same function — and the equivalent system-message paths in the OpenAI and Google GenAI adapters — all substitute variables, so the Anthropic system kwarg is the only one that reaches the API as a literal template.

This is the normal Anthropic request shape: create_prompt_version_from_anthropic({..., "system": "You are {{ persona }}.", "messages": [...]}) stores system as a str (messages.py:134-140). Calling the documented PromptVersion.format(variables=..., sdk="anthropic") then returns the raw template in kwargs["system"], while the user/assistant messages in the same call are correctly formatted.

Minimal reproduction

from phoenix.client.helpers.sdk.anthropic.messages import to_chat_messages_and_kwargs
from phoenix.client.helpers.sdk.openai.chat import to_chat_messages_and_kwargs as openai_fn
from phoenix.client.types.prompts import PromptVersion

data = {
    "model": "claude-3-5-sonnet-latest",
    "system": "You are {{ persona }}.",          # str system message — normal Anthropic shape
    "messages": [
        {"role": "user", "content": "Hello {{ persona }}."},
    ],
}

Reproduction script: /tmp/pxrepro/repro_anthropic_system.py (run under a venv with arize-phoenix-client installed from the commit above).

Actual output

Verbatim output observed by a verifier at HEAD e12748298 (line 184 present):

stored system content: You are {{ persona }}.
anthropic to_chat_messages_and_kwargs: system = 'You are {{ persona }}.'
                                       user   = 'Hello a pirate.'
openai    to_chat_messages_and_kwargs: system = 'You are a pirate.'
anthropic, system stored as list of parts: system = 'You are a pirate.'
PromptVersion.format(sdk='anthropic'): system = 'You are {{ persona }}.'

A second verifier, same commit, reported:

UNPATCHED (HEAD e1274829):
stored system content: You are {{ persona }}.
anthropic system = 'You are {{ persona }}.'  |  user = 'Hello a pirate.'
openai    system = 'You are a pirate.'
anthropic, system stored as list of parts: system = 'You are a pirate.'
PromptVersion.format(sdk='anthropic'): system = 'You are {{ persona }}.'
PATCHED (formatter.format on the str branch): all five lines -> 'You are a pirate.'; 271 tests pass (baseline 271).

Both were independently reproduced; the same output was reproduced again while filing this issue.

Expected behaviour

system should be 'You are a pirate.', matching the user message in the same call. Concrete basis — sibling code paths substitute variables here:

  • anthropic/messages.py:713, _TextContentPartConversion.to_anthropic: text = formatter.format(obj["text"], variables=variables) — the module's own system-content path when the stored system content is a list of parts.
  • openai/chat.py:984, _SystemMessageConversion.to_openai: "content": formatter.format(obj["content"], variables=variables).
  • google_genai/generate_content.py:201-202: if text := formatter.format(content, variables=variables):.
  • PromptVersion.format docstring (client/types/prompts.py:271-272): "variables (Mapping[str, str]): A mapping of variable names to values to use in the prompt."

The only difference between the working and failing cases is whether the stored system content is a str or a list of text parts.

Root cause

packages/phoenix-client/src/phoenix/client/helpers/sdk/anthropic/messages.py:184 — in to_chat_messages_and_kwargs, the system-role branch appends str content verbatim:

if isinstance(message["content"], str):
    system_messages.append(message["content"])
    continue

It never calls formatter.format(...), so variables is ignored for str system content, while the list-of-parts path immediately below it (via _ContentConversion.to_anthropic → _TextContentPartConversion.to_anthropic) and all non-system messages in the same loop are formatted.

Related issues/PRs

Searched for existing reports of this (anthropic system format variable, formatter.format system in issues; anthropic system in PRs) — nothing matching. No collision found by either verifier.

Additional information

  • Environment: Python 3.12.14, macOS 26.6.2 (25G83), arize-phoenix-client at commit e12748298b366605cc0e2aa60e659f473efbff99
  • Also reachable via the public create_prompt_version_from_anthropic(...) + PromptVersion.format(variables=..., sdk="anthropic") API; both to_chat_messages_and_kwargs and to_anthropic are exported in __all__ and appear in docs/source/api/helpers.rst.
  • Happy to be assigned — per CONTRIBUTING I'll wait for a maintainer to confirm before sending a PR. The fix is a one-line change at messages.py:184 to route str system content through formatter.format(message["content"], variables=variables), matching _TextContentPartConversion.to_anthropic and the OpenAI/Google system paths.

Activity

  1. BlueX888 commented on Sep 16, 2026

    @BlueX888
    Author

    Reproduced this. Root cause: packages/phoenix-client/src/phoenix/client/helpers/sdk/anthropic/messages.py:184 -- the system-role branch appends str content verbatim, skipping the formatter.format() call every sibling path makes.

    Minimal reproduction:

    /tmp/pxenv/bin/python /tmp/pxrepro/repro_anthropic_system.py
    

    Observed output:

    stored system content: You are {{ persona }}.
    anthropic to_chat_messages_and_kwargs: system = 'You are {{ persona }}.'
                                           user   = 'Hello a pirate.'
    openai    to_chat_messages_and_kwargs: system = 'You are a pirate.'
    anthropic, system stored as list of parts: system = 'You are a pirate.'
    PromptVersion.format(sdk='anthropic'): system = 'You are {{ persona }}.'
    

    Happy to open a PR with this fix, or to be assigned if you prefer to take it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triageissues that need triage

    Type

    No type

    Projects

    • Status
      ✅ Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions