Skip to content

fix: Xiaomi MiMo vision_analyze uses wrong path — images degraded to text #49388

Description

@likunoc

Bug Description

vision_analyze on Xiaomi MiMo (mimo-v2.5) returns incorrect/meaningless analysis results because images are silently degraded to text summaries instead of being sent to the auxiliary vision model.

Root Cause

The bug involves two related issues in tools/vision_tools.py:

1. _should_use_native_vision_fast_path() conflated two independent capabilities

The function returned True for Xiaomi because _lookup_supports_vision() was True (Xiaomi profile has supports_vision=True). But supports_vision means the model accepts images in user messages, not in tool-result messages.

The condition was:

return (
    _supports_media_in_tool_results(provider, model)
    or _lookup_supports_vision(provider, model, cfg) is True
)

The or _lookup_supports_vision(...) clause bypassed the supports_media_in_tool_results check, enabling the native fast path for providers that support vision in user messages but reject list-type tool content.

2. _supports_media_in_tool_results() profile fallback ignored supports_vision_tool_messages

The profile fallback only checked profile.supports_vision:

if profile is not None and profile.supports_vision:
    return True

For Xiaomi, supports_vision=True but supports_vision_tool_messages=False. The fallback returned True despite the profile explicitly declaring tool-message incompatibility.

What Happens at Runtime

  1. _should_use_native_vision_fast_path() returns True for Xiaomi
  2. _vision_analyze_native() returns a _multimodal envelope (image in tool result)
  3. Agent loop _tool_result_content_for_active_model() detects Xiaomi cannot handle list-type tool content, then downgrades to text summary
  4. Model only sees text description, not pixels -> wrong analysis results

Fix

  1. _supports_media_in_tool_results(): In the profile fallback, also check supports_vision_tool_messages -- if False, return False
  2. _should_use_native_vision_fast_path(): Remove or _lookup_supports_vision(...) -- only enable fast path when _supports_media_in_tool_results() is True
  3. For Xiaomi, vision_analyze now correctly falls through to the aux-LLM path (images sent in user messages, where Xiaomi supports them)

Provider Profile

The Xiaomi profile is correctly configured:

supports_vision=True,                    # accepts images in user messages
supports_vision_tool_messages=False,     # rejects list-type tool content (400 "text is not set")

The issue was that the code did not respect supports_vision_tool_messages in the relevant decision points.

Testing

All existing tests pass:

  • tests/tools/test_vision_native_fast_path.py: 22 passed
  • tests/run_agent/test_vision_tool_messages.py: 17 passed

Added new test test_xiaomi_falls_through_to_aux to verify Xiaomi routes to aux-LLM path.

Impact

  • Affects: Xiaomi MiMo provider (mimo-v2.5)
  • Other providers (Anthropic, OpenAI, OpenRouter, Gemini 3) are unaffected
  • The fix ensures Xiaomi vision analysis works via the aux-LLM path (images in user messages) instead of the broken native fast path

Activity

  1. added
    type/bugSomething isn't working
    comp/agentCore agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint
    tool/visionVision analysis and image generation
    P2Medium — degraded but workaround exists
    duplicateThis issue or pull request already exists
    on Jun 20, 2026
  2. alt-glitch commented on Jun 20, 2026

    @alt-glitch

    Duplicate of #39685 — same root cause and code path. The native vision fast path in tools/vision_tools.py routes images through tool-result messages for Xiaomi MiMo despite the profile declaring supports_vision_tool_messages=False, so the image is degraded to a text summary (or rejected with a 400 "text is not set"). The fix you describe (gate _supports_media_in_tool_results() on supports_vision_tool_messages and change the _should_use_native_vision_fast_path() or clause to and) is exactly the one tracked by the open fix PRs #39692, #45571, and #47747. Confirmed still live on main (the or _lookup_supports_vision(...) clause is present at tools/vision_tools.py:626). Consolidating under #39685.

  3. Nandoca95 commented on Jul 26, 2026

    @Nandoca95

    Current reproduction: opencode-go/mimo-v2.5 after browser_vision

    I am adding current evidence to this existing issue rather than opening a duplicate.

    Environment

    • Hermes main commit: fe3dd9106a1c9d2327fb3795e55b09742c43bafd
    • Effective provider: opencode-go
    • Model: mimo-v2.5
    • Base URL: https://opencode.ai/zen/go/v1
    • Command path: --provider opencode-go -m mimo-v2.5

    Reproduction sequence

    In a fresh temporary Hermes home, with no permanent configuration changes:

    1. browser_navigate completed successfully.
    2. browser_vision completed successfully.
    3. The next model call, still using opencode-go/mimo-v2.5, failed with:
    HTTP 400: Error from provider (Xiaomi): Param Incorrect
    param: `text` is not set
    

    Hermes did not silently fall back to another provider. The logs show that the fallback entry matched the current opencode-go/mimo-v2.5 route, and the error was classified as non-retryable.

    This behavior is also consistent with earlier Hermes sessions using the same effective route, both directly and after a real fallback to opencode-go/mimo-v2.5.

    What this establishes

    The current main branch still reproduces the failure when browser_vision is followed by another request through opencode-go/mimo-v2.5.

    The evidence is consistent with a compatibility problem in the request generated after browser_vision and the OpenCode Go/MiMo request path. At this point I cannot determine whether the invalid or incomplete text field is introduced by Hermes, transformed by OpenCode Go, or produced at the downstream adapter boundary.

    Limitations

    • This was not a direct test of the Xiaomi provider; the effective Hermes provider was opencode-go.
    • Xiaomi is mentioned only because it appears as the upstream source in the returned error.
    • The exact wire payload was not captured.
    • I have not established which historical commit introduced the behavior.
    • The DeepSeek route is only an operational mitigation and is not presented as a fix for this issue.

    Could you confirm whether Hermes is expected to normalize this request state before sending it through opencode-go/mimo-v2.5, and whether the request should be normalized in Hermes or in the provider/relay layer?

    Related OpenCode Go report: anomalyco/opencode#32613

  4. added
    provider/otherCalling models through a small provider without its own label (for example Kilo Code, Hugging Face I
    and removed on Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium — degraded but workaround existscomp/agentCore agent runtime: loop, agent_init, prompt builder, context-compression, responses endpointduplicateThis issue or pull request already existsprovider/otherCalling models through a small provider without its own label (for example Kilo Code, Hugging Face Itool/visionVision analysis and image generationtype/bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions