fix #92688: [Bug]: Qwen vision models fail with 400 "Unexpected item type in content" on DashScope#92782
fix #92688: [Bug]: Qwen vision models fail with 400 "Unexpected item type in content" on DashScope#92782zhangguiping-xydt wants to merge 11 commits into
Conversation
|
ClawSweeper review: did not complete due to Codex infrastructure failure. Reviewed June 16, 2026, 5:02 AM ET / 09:02 UTC. Summary PR surface: Source +507, Tests +762. Total +1269 across 25 files. Reproducibility: unclear. The review failed before ClawSweeper could establish a reproduction path. Review metrics: none identified. Stored data model Merge readiness This is a ClawSweeper/Codex infrastructure failure, not a PR readiness or patch-quality verdict. Risk before merge
Maintainer options:
Next step before merge
Review detailsBest possible solution: Retry the Codex review after fixing the execution failure. Do we have a high-confidence way to reproduce the issue? Unclear. The review failed before ClawSweeper could establish a reproduction path. Is this the best way to solve the issue? Unclear. Retry the review first so ClawSweeper can evaluate the actual issue and fix direction. AGENTS.md: unclear because the file could not be read completely. Codex review notes: model internal, reasoning high; reviewed against 6d22b8eb2420. Label changesLabel changes:
Evidence reviewedPR surface: Source +507, Tests +762. Total +1269 across 25 files. View PR surface stats
What I checked:
Likely related people:
How this review workflow works
|
7096e53 to
4cb00f1
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
Keep Qwen image defaults and text-only model capability facts at the Qwen plugin seam while shared media-understanding code consumes a generic provider-owned override contract.
Update the bundled plugin registration contract to reflect the Qwen Cloud media-understanding aliases now owned by the Qwen plugin.
6c07b8b to
7499f5e
Compare
Summary
What problem does this PR solve?
OpenClaw's Qwen image-understanding routing had two coupled failure points for DashScope:
input: ["text", "image"]metadata as sufficient proof that a Qwen model could handle image content. That allowed known text-only Qwen chat models such asqwen3.7-maxto be selected for image requests when config metadata was overly broad.modelstudio-nativeimage requests did not use the same prompt placement as OpenRouter-style multimodal providers, so the image prompt could be sent as a separate system/developer prompt instead of user content before the image block.Together, that can produce DashScope's
400 Unexpected item type in contentfor an image-understanding request instead of routing to the bundled Qwen vision default.Why does this matter now?
A valid Qwen setup can lose the user's image request before an image-capable model is reached. The reported path configured Qwen/DashScope with a chat model such as
qwen3.7-max; image understanding should skip that text-only family and selectqwen-vl-max-latest.What is the intended outcome?
Fix classification: root-cause fix. Qwen provider-owned runtime metadata now marks the known text-only family as non-image for media-understanding routing, and the resolver entry points consume that metadata before accepting configured image metadata. DashScope-native image prompts are placed in the user content before the image block.
What is intentionally out of scope?
Scope boundary: this PR does not change Qwen auth, API-key lookup, network transport, public plugin SDK shape, public manifest schema, migrations, or provider endpoint selection. The Qwen-specific capability fact stays internal to the Qwen provider/runtime metadata.
What does success look like?
When config marks
qwen3.7-maxas[text,image], OpenClaw still resolves image understanding toqwen/qwen-vl-max-latest, and Qwen/DashScope image prompts are serialized as user text before the image payload.What should reviewers focus on?
Please check the source-of-truth boundary and resolver consistency: Qwen model capability facts are provider-owned runtime metadata, not a new public manifest/plugin SDK contract, and the same filtering rule is applied across default media model resolution, configured provider discovery, image-tool routing, PDF-tool routing, manifest-backed defaults, and bundled Qwen aliases.
Linked context
Closes #92688.
This refresh addresses the review concern that the previous shape risked making a Qwen-specific model fact part of a broader public contract. The current shape keeps the compatibility-sensitive decision in internal provider/runtime metadata while preserving the existing public manifest metadata surface for capabilities and defaults.
Real behavior proof (required for external PRs)
Behavior or issue addressed: image routing must select an image-capable Qwen model when the active/configured Qwen chat model is
qwen3.7-maxand its configured metadata incorrectly includesimage.Real environment tested: OpenClaw's resolver and image-tool/media-understanding paths with Qwen/DashScope-style config, provider registry construction, manifest-backed defaults, and bundled Qwen image defaults.
Exact steps or command run after this patch: configured the active model as
qwen/qwen3.7-max, configured the Qwen provider model entry asinput: ["text", "image"], then exercised the OpenClaw resolver/image-tool paths.Evidence after fix:
qwen/qwen3.7-max, the configured provider metadata can still declareqwen3.7-maxas[text,image], and the image request path selectsqwen/qwen-vl-max-latest.Tests and validation
Which commands did you run?
pnpm exec vitest run \ src/media-understanding/defaults.test.ts \ src/media-understanding/provider-registry.test.ts \ extensions/qwen/index.test.tspnpm exec vitest run src/agents/tools/image-tool.test.tspnpm exec vitest run \ src/agents/tools/image-tool.test.ts \ src/agents/tools/pdf-tool.model-config.test.ts \ src/media-understanding/config-provider-models.test.ts \ src/media-understanding/image.test.tsenv OPENCLAW_VITEST_MAX_WORKERS=2 NODE_OPTIONS=--max-old-space-size=8192 OPENCLAW_VITEST_SHARD_NAME=agentic-agents-tools \ pnpm exec vitest run --config test/vitest/vitest.agents-tools.config.ts src/agents/tools/image-tool.test.tsEarlier validation for the same submitted change also included the extension package-boundary compile/canary checks, additional boundary shards, plugin contracts, and CI artifact build after removing generated catalog artifacts from the commit.
What regression coverage was added or updated?
Coverage now locks the Qwen image-routing behavior at the resolver boundaries that can see configured model metadata: active image model resolution, configured provider vision resolution, default media model resolution with runtime metadata, manifest-backed default resolution, config-derived provider models, PDF tool model config, Qwen/DashScope prompt placement, and bundled Qwen provider registration.
What failed before this fix, if known?
The RED coverage reproduced
qwen/qwen3.7-maxbeing selected from configured[text,image]metadata. A later manifest-backed proof reproduced the static/default path selectingqwen3.7-max; after the follow-up, that same path selectsqwen-vl-max-latest.If no test was added, why not?
Regression tests were added or updated across the image tool, media-understanding defaults/metadata, config provider discovery, PDF tool model config, prompt placement, provider registry, and Qwen provider paths.
Risk checklist
Did user-visible behavior change?
YesDid config, environment, or migration behavior change?
NoDid security, auth, secrets, network, or tool execution behavior change?
NoWhat is the highest-risk area?
Risk labels considered: compatibility and auth-provider. The highest-risk area is model selection for Qwen image understanding, because provider-owned capability facts now take precedence over overly broad configured
inputmetadata for known text-only Qwen families.Risk explanation: the compatibility-sensitive behavior is narrow. Qwen models matching the internal text-only family facts are filtered from image routing, while the existing Qwen image default remains the selected fallback target. The public manifest and plugin SDK contract shapes are not expanded.
How is that risk mitigated?
The same internal capability facts are consumed by the resolver paths covered in this patch, and regression tests exercise configured-model, provider-registry, manifest-backed default, image-tool, PDF-tool, prompt-placement, and bundled Qwen provider flows.
Current review state
What is the next action?
Maintainer-ready confidence: high after issue-specific resolver/image-tool proof, manifest-backed fallback proof, focused Qwen/media-understanding regression coverage, lint, and production/test type checks on the submitted head.
What is still waiting on author, maintainer, CI, or external proof?
The author-side refresh is complete for this scoped routing fix. The next step is maintainer review and CI on the refreshed head.
Which bot or reviewer comments were addressed?
qwen3.7-maxconfigured as[text,image]andqwen/qwen-vl-max-latestselected.qwen-vl-max-latest.qwen-vl-max-latestreplacingqwen3.7-maxfor image understanding.src/**internals for capability facts.Patch quality notes: the diff touches multiple resolver entry points because the same capability source-of-truth is consumed by defaults, configured model discovery, image runtime, image tool config, PDF tool config, provider registry inputs, and manifest-backed defaults. Covering those consumers together keeps the Qwen image-routing behavior consistent for the reported configuration path.