Skip to content

Python: [Bug]: Hosted /responses drops finish_reason="content_filter" — filtered turns are indistinguishable from successful ones #8475

Closed

Description

@jiks-hue

Description

What happened?

When a turn is stopped by an Azure OpenAI content filter, the hosted agent's
/responses HTTP output reports a completely successful turn. An HTTP
caller has no way to tell a filtered turn from a normal one.

The upstream chat completion clearly carries the signal. Captured from the
agent's own telemetry for the request in question:

{"index": 0, "finish_reason": "content_filter",
 "message": {"content": "I'm sorry, but I cannot assist with that request."}}

But the /responses payload returned to the caller for that same turn is:

HTTP 200
{
  "status": "completed",
  "error": null,
  "id": "caresp_...",
  "output": [
    {"type": "message", "role": "assistant", "status": "completed",
     "content": [{"type": "output_text",
                  "text": "I'm sorry, but I cannot assist with that request."}]}
  ]
}

Probing the full payload for any trace of the filter finds none:

probe present
finish_reason ❌
content_filter ❌
incomplete ❌
refusal ❌

Top-level keys are agent_reference, agent_session_id, background, completed_at, created_at, id, model, object, output, parallel_tool_calls, response_id, status, usage — the finish_reason: "content_filter" from the
underlying completion is dropped, and both the response and the output item
say "completed".

What did you expect to happen?

The /responses output should preserve the fact that the turn was
content-filtered — e.g. status: "incomplete" with an
incomplete_details.reason, or a finish_reason on the output item, or an
error — so a caller can react to it.

Why this matters in practice

We run a hosted agent behind a small HTTP relay. When a user's message trips
the filter, the relay hands them back the raw
"I'm sorry, but I cannot assist with that request." string, which is
out-of-character for the agent's persona and unexplained — the user re-sent the
identical message, which deterministically tripped the same filter again.

Because nothing in the response distinguishes this case, the only workaround
available to us is matching on the refusal text, which is obviously
brittle: it breaks on wording changes and localisation, and risks false
positives on legitimate short replies.

Note this is not #7025 (ChatResponse.finish_reason always None on
Responses-based clients, closed 2026-07-17). This report is about the
hosting/server side — what /responses emits to an HTTP caller. Related,
but a different layer.

It is also distinct from #8385 (OpenAIContentFilterException raising
ValueError on the ContentFiltered inner code): on this hosted path no
exception surfaces at all, because the service returns a normal completion with
finish_reason: content_filter rather than an error.

Versions

  • agent-framework-core 1.18.0
  • agent-framework-foundry 1.13.0
  • agent-framework-foundry-hosting 1.0.0b260910
  • agent-framework-openai 1.14.3
  • azure-ai-agentserver-core 2.1.0, -responses 2.2.0b1

Agent hosted with ResponsesHostServer(agent, history_source="agent") on
Azure AI Foundry, gpt-5.4, called over the OpenAI-compatible /responses
protocol.

Repro

  1. Host any agent with ResponsesHostServer on Foundry.
  2. POST a prompt to its /responses endpoint that trips the deployment's
    content filter (a violent or sexual plot summary will do — ours was an
    ordinary historical-drama recap that scored high on sexual).
  3. Observe HTTP 200, status: "completed", no error, and the canned
    refusal as output_text, while the agent's telemetry for the same request
    shows finish_reason: "content_filter".

Activity

  1. added
    pythonUsage: [Issues, PRs], Target: Python
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Sep 17, 2026
  2. added
    reproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow
    on Sep 17, 2026
  3. github-actions commented on Sep 17, 2026

    @github-actions
    Contributor

    🤖 Automated triage reproduction notes (agent-authored — trust but verify)

    Agent analysis

    Repro: python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py::_handle_inner_agent at lines 848-859 ignores AgentResponseUpdate.finish_reason, and _handle_response at lines 695-701 consequently emits response.completed. It triggers when an agent update contains refusal text with finish_reason="content_filter". Minimal repro: use the existing ASGI harness with a mock agent emitting that update and POST a non-streaming request to /responses; the response returns HTTP 200 with status="completed".

    • Failing test: python/packages/foundry_hosting/tests/test_responses.py::TestNonStreaming::test_content_filter_finish_reason_marks_response_incomplete
    • Files examined: python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py, python/packages/foundry_hosting/tests/test_responses.py, python/packages/core/agent_framework/_types.py, python/packages/foundry_hosting/pyproject.toml, python/packages/hosting-responses/agent_framework_hosting_responses/_parsing.py
    • Tests run: TestNonStreaming::test_basic_text_response, TestNonStreaming::test_content_filter_finish_reason_marks_response_incomplete
    • Reported version: 1.0.0b260910
    • Current version: 1.0.0b260910
  4. added
    foundryUsage: [Issues, PRs], Target: all Foundry integrations
    hostingUsage: [Issues, PRs], Target: all hosting related solutions
    and removed
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Sep 17, 2026
  5. anishmehta24 commented on Sep 17, 2026

    @anishmehta24
    Contributor

    I'd like to take this one. Plan: track AgentResponseUpdate.finish_reason in the hosted /responses path (both the plain-agent and workflow branches) and end the response with response.incomplete + incomplete_details.reason="content_filter" instead of response.completed when a turn is content-filtered, mirroring the OpenAI Responses shape. Tests for streaming and non-streaming. PR shortly.

  6. added a commit that references this issue on Sep 19, 2026
    5e684a2
  7. added a commit that references this issue on Sep 21, 2026
    f80498b
  8. added a commit that references this issue on Sep 29, 2026
    758ec03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

foundryUsage: [Issues, PRs], Target: all Foundry integrationshostingUsage: [Issues, PRs], Target: all hosting related solutionspythonUsage: [Issues, PRs], Target: PythonreproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions