Skip to content

Qwen3-VL-8B-Instruct benchmark test error #487

@skchang99

Description

@skchang99

Describe the bug
This error shows up for v.0.3.0 and all previous releases as well.
When benchmarking the model "Qwen3-VL-8B-Instruct ".
The test error out right away with "ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'"

To Reproduce
Steps to reproduce the behavior:
0. Download "Qwen3-VL-8B-Instruct" model first.

  1. Go to 'Bench.'
  2. Click on 'Performance'
  3. Select 'Qwen3-VL-8B-Instruct' from the dropdown and hit "Run Benchmark".
  4. See error

Expected behavior
The benchmark should run and generate results.

Screenshots

Image

Desktop (please complete the following information):

  • OS: [macOS Tahoe]
  • Browser [Safari Version 26.4 (21624.1.16.11.4)]
  • Version [26.4]

Additional context
2026-03-30 22:26:20,398 - omlx.admin.routes - INFO - [-] - Benchmark started: bench-36638558ac59 model=Qwen3-VL-8B-Instruct tests=6
2026-03-30 22:26:20,401 - omlx.engine_pool - INFO - [-] - Loading model as LM (force_lm=True): Qwen3-VL-8B-Instruct
2026-03-30 22:26:20,417 - omlx.engine_pool - WARNING - [-] - VLM loading failed for Qwen3-VL-8B-Instruct, falling back to LLM: ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'
2026-03-30 22:26:20,417 - omlx.engine.batched - INFO - [-] - BatchedEngine stopped
2026-03-30 22:26:20,466 - omlx.admin.benchmark - ERROR - [-] - Benchmark error: ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'
Traceback (most recent call last):
File "/Applications/oMLX.app/Contents/Resources/omlx/engine_pool.py", line 560, in _load_engine
await engine.start()
File "/Applications/oMLX.app/Contents/Resources/omlx/engine/batched.py", line 175, in start
self._model, self._tokenizer = await loop.run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/cpython-3.11/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Resources/omlx/engine/batched.py", line 169, in _load_model_sync
return load(
^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/utils.py", line 491, in load
model, config = load_model(model_path, lazy, model_config=model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/utils.py", line 343, in load_model
model = model_class(model_args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/models/qwen3_vl.py", line 31, in init
self.language_model = qwen3.Model(qwen3.ModelArgs.from_dict(args.text_config))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/models/base.py", line 15, in from_dict
return cls(
^^^^
TypeError: ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Applications/oMLX.app/Contents/Resources/omlx/admin/benchmark.py", line 605, in run_benchmark
engine = await engine_pool.get_engine(request.model_id, force_lm=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Resources/omlx/engine_pool.py", line 407, in get_engine
await self._load_engine(model_id, force_lm=force_lm)
File "/Applications/oMLX.app/Contents/Resources/omlx/engine_pool.py", line 584, in _load_engine
await engine.start()
File "/Applications/oMLX.app/Contents/Python/cpython-3.11/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Resources/omlx/engine/batched.py", line 169, in _load_model_sync
return load(
^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/utils.py", line 491, in load
model, config = load_model(model_path, lazy, model_config=model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/utils.py", line 343, in load_model
model = model_class(model_args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/models/qwen3_vl.py", line 31, in init
self.language_model = qwen3.Model(qwen3.ModelArgs.from_dict(args.text_config))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/oMLX.app/Contents/Python/framework-mlx-framework/lib/python3.11/site-packages/mlx_lm/models/base.py", line 15, in from_dict
return cls(
^^^^
TypeError: ModelArgs.init() missing 1 required positional argument: 'tie_word_embeddings'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions