logs.txt

Summary

When loading Qwen3.5-397B-A17B-MLX-8bit in oMLX, the model is discovered as a vlm model and the server attempts to initialize the VLM engine. VLM loading then fails every time because Qwen2VLImageProcessor requires PyTorch and Torchvision, which are not present in the environment. oMLX falls back to LLM mode.

This happens consistently across restarts and after changing memory settings.

Environment

• oMLX
• macOS
• Selector shown in logs: KqueueSelector
• Metal memory enforcement enabled during most attempts
• API key auth enabled
• Model: Qwen3.5-397B-A17B-MLX-8bit

What I expected

One of these:

1. The model should load successfully in VLM mode, or
2. oMLX should clearly validate/report required VLM dependencies before load, or
3. The model should be classified/treated as LLM-only if the current environment cannot support its VLM processor stack.

What actually happened

• Model is discovered as:
  • type: vlm
  • engine: vlm
• oMLX tries to load it as VLM
• VLM initialization fails with missing dependency errors:
  • Qwen2VLImageProcessor requires the PyTorch library but it was not found
  • Qwen2VLImageProcessor requires the Torchvision library but it was not found
• oMLX falls back to LLM mode

Reproduction steps

1. Start oMLX with model directories containing Qwen3.5-397B-A17B-MLX-8bit
2. Let model discovery complete
3. Load Qwen3.5-397B-A17B-MLX-8bit from the admin UI or API
4. Observe VLM initialization failure and fallback to LLM

Relevant log excerpts

Model discovery

2026-03-27 22:28:00,122 - omlx.model_discovery - INFO - [-] - Discovered model: Qwen3.5-397B-A17B-MLX-8bit (type: vlm, engine: vlm, size: 412.70GB)
2026-03-27 22:28:00,123 - omlx.server - INFO - [-] - Default model: Qwen3.5-397B-A17B-MLX-8bit