-
Notifications
You must be signed in to change notification settings - Fork 718
Qwen3-VL-8B-Instruct benchmark test error #487
Description
Describe the bug
This error shows up for v.0.3.0 and all previous releases as well.
When benchmarking the model "Qwen3-VL-8B-Instruct ".
The test error out right away with "ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'"
To Reproduce
Steps to reproduce the behavior:
0. Download "Qwen3-VL-8B-Instruct" model first.
- Go to 'Bench.'
- Click on 'Performance'
- Select 'Qwen3-VL-8B-Instruct' from the dropdown and hit "Run Benchmark".
- See error
Expected behavior
The benchmark should run and generate results.
Screenshots
Desktop (please complete the following information):
- OS: [macOS Tahoe]
- Browser [Safari Version 26.4 (21624.1.16.11.4)]
- Version [26.4]
Additional context
2026-03-30 22:26:20,398 - omlx.admin.routes - INFO - [-] - Benchmark started: bench-36638558ac59 model=Qwen3-VL-8B-Instruct tests=6
2026-03-30 22:26:20,401 - omlx.engine_pool - INFO - [-] - Loading model as LM (force_lm=True): Qwen3-VL-8B-Instruct
2026-03-30 22:26:20,417 - omlx.engine_pool - WARNING - [-] - VLM loading failed for Qwen3-VL-8B-Instruct, falling back to LLM: ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'
2026-03-30 22:26:20,417 - omlx.engine.batched - INFO - [-] - BatchedEngine stopped
2026-03-30 22:26:20,466 - omlx.admin.benchmark - ERROR - [-] - Benchmark error: ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'
Traceback (most recent call last):
File "/Applications/oMLX.app/Contents/Resources/omlx/engine_pool.py", line 560, in _load_engine
await engine.start()
File "/Applications/oMLX.app/Contents/Resources/omlx/engine/batched.py", line 175, in start
self._model, self._tokenizer = await loop.run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/cpython-3.11/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Resources/omlx/engine/batched.py", line 169, in _load_model_sync
return load(
^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/utils.py", line 491, in load
model, config = load_model(model_path, lazy, model_config=model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/utils.py", line 343, in load_model
model = model_class(model_args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/models/qwen3_vl.py", line 31, in init
self.language_model = qwen3.Model(qwen3.ModelArgs.from_dict(args.text_config))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/models/base.py", line 15, in from_dict
return cls(
^^^^
TypeError: ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Applications/oMLX.app/Contents/Resources/omlx/admin/benchmark.py", line 605, in run_benchmark
engine = await engine_pool.get_engine(request.model_id, force_lm=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Resources/omlx/engine_pool.py", line 407, in get_engine
await self._load_engine(model_id, force_lm=force_lm)
File "/Applications/oMLX.app/Contents/Resources/omlx/engine_pool.py", line 584, in _load_engine
await engine.start()
File "/Applications/oMLX.app/Contents/Python/cpython-3.11/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Resources/omlx/engine/batched.py", line 169, in _load_model_sync
return load(
^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/utils.py", line 491, in load
model, config = load_model(model_path, lazy, model_config=model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/utils.py", line 343, in load_model
model = model_class(model_args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/models/qwen3_vl.py", line 31, in init
self.language_model = qwen3.Model(qwen3.ModelArgs.from_dict(args.text_config))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/models/base.py", line 15, in from_dict
return cls(
^^^^
TypeError: ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'