Skip to content

Cannot load ONNX whisper small on Mac M1 #2273

Description

@pivonroll

System Info

Optimum version: 1.25.2
Python Version 3.10.13
OS: Mac OS 15.5
Installed packages
Package            Version
------------------ -----------
aiohappyeyeballs   2.6.1
aiohttp            3.11.18
aiosignal          1.3.2
async-timeout      5.0.1
attrs              25.3.0
audioread          3.0.1
certifi            2025.4.26
cffi               1.17.1
charset-normalizer 3.4.2
coloredlogs        15.0.1
datasets           3.6.0
decorator          5.2.1
dill               0.3.8
filelock           3.18.0
flatbuffers        25.2.10
frozenlist         1.6.0
fsspec             2025.3.2
huggingface-hub    0.31.2
humanfriendly      10.0
idna               3.10
Jinja2             3.1.6
joblib             1.5.0
lazy_loader        0.4
librosa            0.11.0
llvmlite           0.44.0
MarkupSafe         3.0.2
mpmath             1.3.0
msgpack            1.1.0
multidict          6.4.3
multiprocess       0.70.16
networkx           3.4.2
numba              0.61.2
numpy              2.2.5
onnx               1.18.0
onnxruntime        1.22.0
optimum            1.25.2
packaging          25.0
pandas             2.2.3
pillow             11.2.1
pip                23.0.1
platformdirs       4.3.8
pooch              1.8.2
propcache          0.3.1
protobuf           6.31.0
pyarrow            20.0.0
pycparser          2.22
python-dateutil    2.9.0.post0
pytz               2025.2
PyYAML             6.0.2
regex              2024.11.6
requests           2.32.3
safetensors        0.5.3
scikit-learn       1.6.1
scipy              1.15.3
sentencepiece      0.2.0
setuptools         65.5.0
six                1.17.0
soundfile          0.13.1
soxr               0.5.0.post1
sympy              1.14.0
threadpoolctl      3.6.0
timm               1.0.15
tokenizers         0.21.1
torch              2.7.0
torchaudio         2.7.0
torchvision        0.22.0
tqdm               4.67.1
transformers       4.51.3
typing_extensions  4.13.2
tzdata             2025.2
urllib3            2.4.0
xxhash             3.5.0
yarl               1.20.0

Who can help?

@philschmid tagging you as this error originates in pipelines.

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)

from optimum.pipelines import pipeline

model_kwargs = {"encoder_file_name": "encoder_model.onnx", "decoder_file_name": "decoder_model.onnx"}
onnx_qa = pipeline(
    "automatic-speech-recognition",
    model="onnx-community/whisper-small-ONNX",
    subfolder="onnx",
    model_kwargs=model_kwargs,
    accelerator="ort",
)

it fails with an error:

Moving the following attributes in the config to the generation config: {'max_length': 448, 'suppress_tokens': [1, 2, 7, 8, 9, 10, 14, 25, 26, 27, 28, 29, 31, 58, 59, 60, 61, 62, 63, 90, 91, 92, 93, 359, 503, 522, 542, 873, 893, 902, 918, 922, 931, 1350, 1853, 1982, 2460, 2627, 3246, 3253, 3268, 3536, 3846, 3961, 4183, 4667, 6585, 6647, 7273, 9061, 9383, 10428, 10929, 11938, 12033, 12331, 12562, 13793, 14157, 14635, 15265, 15618, 16553, 16604, 18362, 18956, 20075, 21675, 22520, 26130, 26161, 26435, 28279, 29464, 31650, 32302, 32470, 36865, 42863, 47425, 49870, 50254, 50258, 50360, 50361, 50362], 'begin_suppress_tokens': [220, 50257]}. You are seeing this warning because you've set generation parameters in the model config, as opposed to in the generation config.
Error evaluating: thread_id: pid_22869_id_4352066944
frame_id: 6425813264
scope: FRAME
attrs: model
Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_vars.py", line 316, in resolve_compound_variable_fields
    return _typeName, _resolve_default_variable_fields(var, resolver, offset)
  File "/Applications/PyCharm.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_vars.py", line 266, in _resolve_default_variable_fields
    return resolver.get_dictionary(VariableWithOffset(var, offset) if offset else var)
  File "/Applications/PyCharm.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 86, in get_dictionary
    return self._getPyDictionary(var, names)
  File "/Applications/PyCharm.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 159, in _getPyDictionary
    names = self.get_names(var)
  File "/Applications/PyCharm.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 147, in get_names
    names = dir(var)
  File "/Users/radovanzivkovic/.pyenv/versions/ms_speech_ht5_tts-3.10.13/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2971, in __dir__
    parameters = list(self._parameters.keys())
  File "/Users/radovanzivkovic/.pyenv/versions/ms_speech_ht5_tts-3.10.13/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1940, in __getattr__
    raise AttributeError(
AttributeError: '_ORTModelForWhisper' object has no attribute '_parameters'
Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/plugins/python-ce/helpers/pydev/pydevd.py", line 1570, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Applications/PyCharm.app/Contents/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/radovanzivkovic/Development/python_projects/ai/huggingface_models/optimum_exporter/src/optimum_exporter/whisper_onnx.py", line 11, in <module>
    onnx_qa = pipeline(
  File "/Users/radovanzivkovic/.pyenv/versions/ms_speech_ht5_tts-3.10.13/lib/python3.10/site-packages/optimum/pipelines/pipelines_base.py", line 353, in pipeline
    model, model_id, tokenizer, feature_extractor = MAPPING_LOADING_FUNC[accelerator](
  File "/Users/radovanzivkovic/.pyenv/versions/ms_speech_ht5_tts-3.10.13/lib/python3.10/site-packages/optimum/pipelines/pipelines_base.py", line 276, in load_ort_pipeline
    return model, model_id, tokenizer, feature_extractor
UnboundLocalError: local variable 'model_id' referenced before assignment

Expected behavior

Model should load.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions