Skip to content

messages→responses 翻译的 input_image 缺 detail，严格校验的 Responses 上游（vLLM 等）返回 400 "ResponseInputImageParam detail Field required" #1474

New issue
New issue
Closed
Closed
messages→responses 翻译的 input_image 缺 detail，严格校验的 Responses 上游（vLLM 等）返回 400 "ResponseInputImageParam detail Field required"#1474

Description

@zacharyxz
zacharyxz
opened on Oct 10, 2026

环境：Claude Code（Anthropic Messages API /v1/messages）→ magpie（main，2026-10-10 的报告）→ 自建 provider，协议 Responses（endpoint 是 vLLM 的 Responses API）。

现象：对话里带一张图（粘贴截图，或工具返回图片）之后，请求稳定被上游 400：

API Error: 400 <provider>: 1 validation error for ResponseInputImageParam
  detail
    Field required [type=missing, input_value={'type': 'input_image', '...4EEQAAAABJRU5ErkJggg=='}, input_type=dict]
      For further information visit https://errors.pydantic.dev/2.13/v/missing

纯文字对话一切正常；但一旦图片进入会话历史，之后每次请求都会带上它并重复 400（会话基本卡死）。

复现：

  1. magpie 配置一个 Responses 协议的 provider，其端点按 openai-python 的类型对请求做严格校验（如 vLLM 的 Responses API）；
  2. Claude Code（ANTHROPIC_BASE_URL 指向 magpie）使用该 provider 的模型；
  3. 在 Claude Code 里粘贴一张截图（或让工具读取一张 png）后发送；
  4. 请求 400，如上。

根因分析（供参考）：报错中的 ResponseInputImageParam 是 openai-python 的 Responses 类型；自 v2.10.0 起它把 detail 标为 Required（值 low/high/auto/original），OpenAPI 规范的 InputImageContent 也同样 required: [type, detail]。用这套类型生成 pydantic 校验器的服务会拒收缺 detail 的请求。

magpie 在 messages→responses 翻译里构造的 input_image 只带 type+image_url，没带 detail。涉及 internal/gateway/responses.go 三处构造：

  • buildResponses：用户消息里的图片（用户贴图正是这一处），以及把工具结果图片放进 function_call_output.output 时；
  • orphanedToolOutputs：把没有 call_id 的 tool output 合成用户消息时。

（对照：Responses→Responses 的原样转发路径不改字节，不受影响。另外按 SDK 类型，function_call_output 内的 input_image 其 detail 是可选的，必填要求出在 message content 的 union——用户贴图与合成消息都走这个 union。）

Activity

  1. ump45nose commented on Oct 10, 2026

    @ump45nose
    ump45nose
    on Oct 10, 2026
    Contributor

    Here is a fix on a fork branch. This repo doesn't take PRs, so I'm leaving it here in case it's useful.

    Cause: in internal/gateway/responses.go, the messages→responses translation builds input_image with only type and image_url. That happens in three places: a user message's image and a tool result's image list in buildResponses, and the user message that orphanedToolOutputs builds from a tool output with no call_id. openai-python ≥ 2.10 and the OpenAPI InputImageContent mark detail as required, so validators generated from those types (vLLM's Responses API, for example) return 400.

    Change: each of those three places now sends "detail": "auto", which is the default OpenAI applies when detail is left out. Responses→Responses passthrough doesn't change.

    Test (internal/gateway/input_image_detail_test.go). With the old responses.go it fails:

    --- FAIL: TestResponsesInputImageHasDetail (0.01s)
    input_image_detail_test.go:56: input_image without detail: map[image_url:data:image/png;base64,aGVsbG8= type:input_image]
    input_image_detail_test.go:56: input_image without detail: map[image_url:data:image/png;base64,d29ybGQhIQ== type:input_image]
--- FAIL: TestOrphanedToolOutputImageHasDetail (0.00s)
    input_image_detail_test.go:71: input_image without detail: {"input":[{"content":[{"text":"shot","type":"input_text"},{"image_url":"data:image/png;base64,aGVsbG8=","type":"input_image"}],"role":"user","type":"message"}],"model":"m"}

    With the fix, both tests pass.

    Checks (Go 1.26.8, linux/amd64, temporary HOME):

    • gofmt -l on the changed files: no output
    • GOOS=linux go vet -tags nogui ./..., GOOS=darwin go vet -tags nogui ./..., GOOS=windows go vet ./...: pass
    • GOOS=linux go build -tags nogui, GOOS=darwin go build -tags nogui, GOOS=windows go build: pass
    • Plain go vet / go build on linux and darwin: not run successfully. The box has no GTK/WebKit, and there's no cgo for darwin.
    • go test -tags nogui ./...: everything passes except tests that go through a proxy, which time out in this sandbox: TestPluginModelListThroughAccountProxy (gateway), TestPluginHTTP2TakesTheProxy and TestPluginProxy (plugin). TestPluginModelListThroughAccountProxy fails the same way on unmodified main here.
    • Not run against a real vLLM server.

    Commit 2033ba78 on top of a5114d93 (it merges cleanly onto the current main): main...ump45nose:magpie:fix/1474-input-image-detail

    --- a/internal/gateway/responses.go
+++ b/internal/gateway/responses.go
@@ -537 +537 @@ func orphanedToolOutputs(body []byte) []byte {
-			content = append(content, map[string]any{"type": "input_image", "image_url": dataURL(image)})
+			content = append(content, map[string]any{"type": "input_image", "detail": "auto", "image_url": dataURL(image)})
@@ -805 +805 @@ func buildResponses(r *Request, model, host string, rejectTemp bool) []byte {
-					content = append(content, map[string]any{"type": "input_image", "image_url": dataURL(p)})
+					content = append(content, map[string]any{"type": "input_image", "detail": "auto", "image_url": dataURL(p)})
@@ -839 +839 @@ func buildResponses(r *Request, model, host string, rejectTemp bool) []byte {
-						items = append(items, map[string]any{"type": "input_image", "image_url": dataURL(im)})
+						items = append(items, map[string]any{"type": "input_image", "detail": "auto", "image_url": dataURL(im)})

    (The test file is on the branch.)

    AI assistance disclosure: an AI coding assistant helped write this fix, the test and this comment. I reviewed the change and ran the checks listed above.

  2. yetone commented on Oct 12, 2026

    @yetone
    yetone
    on Oct 12, 2026
    Owner

    @zacharyxz 感谢详细的根因分析，抱歉这么久才回复。@ump45nose thanks for the fork branch and the failing test, and sorry for the wait.

    我核对了当前 main（091593f7d）：messages→responses 翻译构造的 input_image 仍然只有 type 和 image_url，三处都还在（internal/gateway/responses.go 的 orphanedToolOutputs 约 661 行，buildResponses 约 1036、1070 行），所以严格校验的 Responses 上游（vLLM 等）依然会 400，这个问题还没修。

    ump45nose 的补丁（commit 2033ba7，main...ump45nose:magpie:fix/1474-input-image-detail ）正在审查：三处都补 "detail": "auto"，加上它的测试。合入时提交会带上 ump45nose 的署名（作者或 Co-authored-by）。合入并发版后会在这里回复版本号，issue 保持打开等你确认。

  3. yetone
    added 2 commits that reference this issue on Oct 12, 2026

    gateway: an image Claude Code sends to a Responses provider carries d…

    bbc7ee0

    gateway: the saved Anthropic-to-Responses body carries the image's de…

    1e080cc

  4. yetone commented on Oct 12, 2026

    @yetone
    yetone
    on Oct 12, 2026
    Owner

    @zacharyxz 已修好：Claude Code 发给 Responses 供应商的图片（input_image）现在带上 detail: "auto"，vLLM 这类严格校验的上游不会再因为缺 detail 回 400。三处构造 input_image 的地方（用户消息里的图、工具结果里的图、孤立的工具输出）都覆盖了，测试在旧代码上会失败。

    修复来自 @ump45nose 的补丁，以 TA 为作者合入：bbc7ee0a9；另有 1e080cc 更新了一份保存的上游请求体样例。感谢 @ump45nose！

    会随下一个版本发布。这个 issue 先保持打开，等你确认在你的环境里能用了再关。

  5. yetone commented on Oct 12, 2026

    @yetone
    yetone
    on Oct 12, 2026
    Owner

    @zacharyxz @ump45nose 图片的 detail: auto 已在 v0.1.1219 发布（magpie 里检查更新即可）。麻烦更新后确认一下，有问题直接在这里回复。

  6. zacharyxz commented on Oct 13, 2026

    @zacharyxz
    zacharyxz
    on Oct 13, 2026
    Author

    感谢修复，已确认该补丁对 vllm/vllm-openai:nightly (0.30.1rc1.dev396+gac68c3087) 部署的 POST /v1/responses 请求有效。

  7. yetone commented on Oct 13, 2026

    @yetone
    yetone
    on Oct 13, 2026
    Owner

    感谢在 vLLM nightly 上实测确认 🙏 关闭了，后续 /v1/responses 有别的兼容问题再开。

  8. yetone
    closed this as completedon Oct 13, 2026
  9. yetone
    added a commit that references this issue on Oct 26, 2026

    gateway: an image's detail reaches a Chat or Responses upstream; Resp…

    44a626b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions