System Info
optimum[onnxruntime]==1.25.0
python==3.13
os: debian bookworm slim
Who can help?
Geting the following error when using SentenceTransformers with backend="onnx". It was working fine with optimum[onnxruntime]==1.24.0.
Traceback (most recent call last):
File "/app/test.py", line 6, in <module>
embeddings = model.encode(sentences)
File "/app/.venv/lib/python3.13/site-packages/sentence_transformers/SentenceTransformer.py", line 685, in encode
out_features = self.forward(features, **kwargs)
File "/app/.venv/lib/python3.13/site-packages/sentence_transformers/SentenceTransformer.py", line 758, in forward
input = module(input, **module_kwargs)
File "/app/.venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
File "/app/.venv/lib/python3.13/site-packages/sentence_transformers/models/Transformer.py", line 442, in forward
outputs = self.auto_model(**trans_features, **kwargs, return_dict=True)
File "/app/.venv/lib/python3.13/site-packages/optimum/modeling_base.py", line 99, in __call__
return self.forward(*args, **kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
TypeError: ORTModelForFeatureExtraction.forward() got an unexpected keyword argument 'return_dict'
I think the commit that introduced the issue was f93964c by @Abdennacer-Badaoui, which removed **kwargs from ORTModelForFeatureExtraction.forward(). Will it be ok to add back **kwargs?
Related: huggingface/sentence-transformers#3363
ONNX Runtime: @JingyaHuang, @echarlaix
Information
Tasks
Reproduction (minimal, reproducible, runnable)
# test.py
# https://sbert.net/docs/sentence_transformer/usage/efficiency.html#onnx
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2", backend="onnx", model_kwargs={"file_name": "onnx/model.onnx"})
sentences = ["This is an example sentence", "Each sentence is converted"]
embeddings = model.encode(sentences)
print(embeddings)# Dockerfile
FROM ghcr.io/astral-sh/uv:debian-slim AS base
WORKDIR /app
RUN apt-get update && apt-get install --yes git
COPY test.py test.py
ARG OPTIMUM_REF=cf5c6ecdf62e48b0070da953ff5089b85bd4519f
ARG OPTIMUM_PACKAGE=optimum[onnxruntime]@git+https://github.com/huggingface/optimum.git@${OPTIMUM_REF}
RUN uv venv --python 3.13 /app/.venv && \
uv pip install hf_xet sentence-transformers "${OPTIMUM_PACKAGE}" && \
uv pip list && \
uv run python test.py
Errors with optimum[onnxruntime]==1.25.0.
> docker build --progress plain --build-arg "OPTIMUM_PACKAGE=optimum[onnxruntime]==1.25.0" .
#0 building with "default" instance using docker-container driver
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 493B done
#1 DONE 0.0s
#2 [internal] load metadata for ghcr.io/astral-sh/uv:debian-slim
#2 DONE 1.3s
#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s
#4 [1/5] FROM ghcr.io/astral-sh/uv:debian-slim@sha256:f013f0a1c9f8b3186d2d3ab8d79ad4806f205e55430a262a7baa9dd6afbbfd12
#4 resolve ghcr.io/astral-sh/uv:debian-slim@sha256:f013f0a1c9f8b3186d2d3ab8d79ad4806f205e55430a262a7baa9dd6afbbfd12 done
#4 DONE 0.0s
#5 [internal] load build context
#5 transferring context: 29B done
#5 DONE 0.0s
#6 [2/5] WORKDIR /app
#6 CACHED
#7 [3/5] RUN apt-get update && apt-get install --yes git
#7 CACHED
#8 [4/5] COPY test.py test.py
#8 CACHED
#9 [5/5] RUN uv venv --python 3.13 /app/.venv && uv pip install hf_xet sentence-transformers "optimum[onnxruntime]==1.25.0" && uv pip list && uv run python test.py
#9 1.056 Downloading cpython-3.13.3-linux-aarch64-gnu (download) (16.8MiB)
#9 1.912 Downloading cpython-3.13.3-linux-aarch64-gnu (download)
#9 2.013 Using CPython 3.13.3
#9 2.013 Creating virtual environment at: .venv
#9 2.542 Resolved 56 packages in 427ms
#9 2.570 Downloading setuptools (1.1MiB)
#9 2.579 Downloading pillow (4.3MiB)
#9 2.579 Downloading networkx (1.6MiB)
#9 2.579 Downloading sympy (6.0MiB)
#9 2.579 Downloading scipy (33.5MiB)
#9 2.579 Downloading transformers (9.9MiB)
#9 2.579 Downloading pyarrow (38.8MiB)
#9 2.580 Downloading pandas (14.5MiB)
#9 2.580 Downloading scikit-learn (11.6MiB)
#9 2.580 Downloading onnxruntime (13.8MiB)
#9 2.580 Downloading aiohttp (1.6MiB)
#9 2.580 Downloading numpy (13.4MiB)
#9 2.580 Downloading torch (94.4MiB)
#9 2.586 Downloading hf-xet (23.3MiB)
#9 2.586 Downloading tokenizers (2.8MiB)
#9 2.586 Downloading onnx (16.6MiB)
#9 3.274 Downloading setuptools
#9 3.519 Downloading aiohttp
#9 3.540 Downloading networkx
#9 3.984 Downloading tokenizers
#9 4.416 Downloading pillow
#9 4.815 Downloading sympy
#9 5.616 Downloading transformers
#9 5.861 Downloading scikit-learn
#9 6.105 Downloading numpy
#9 6.212 Downloading onnxruntime
#9 6.277 Downloading pandas
#9 6.461 Downloading onnx
#9 7.090 Downloading hf-xet
#9 7.698 Downloading scipy
#9 7.972 Downloading pyarrow
#9 10.19 Downloading torch
#9 10.19 Prepared 56 packages in 7.65s
#9 10.41 Installed 56 packages in 212ms
#9 10.41 + aiohappyeyeballs==2.6.1
#9 10.41 + aiohttp==3.11.18
#9 10.41 + aiosignal==1.3.2
#9 10.41 + attrs==25.3.0
#9 10.41 + certifi==2025.4.26
#9 10.41 + charset-normalizer==3.4.2
#9 10.41 + coloredlogs==15.0.1
#9 10.41 + datasets==3.6.0
#9 10.41 + dill==0.3.8
#9 10.41 + filelock==3.18.0
#9 10.41 + flatbuffers==25.2.10
#9 10.41 + frozenlist==1.6.0
#9 10.41 + fsspec==2025.3.0
#9 10.41 + hf-xet==1.1.1
#9 10.41 + huggingface-hub==0.31.2
#9 10.41 + humanfriendly==10.0
#9 10.41 + idna==3.10
#9 10.41 + jinja2==3.1.6
#9 10.41 + joblib==1.5.0
#9 10.41 + markupsafe==3.0.2
#9 10.41 + mpmath==1.3.0
#9 10.41 + multidict==6.4.3
#9 10.41 + multiprocess==0.70.16
#9 10.41 + networkx==3.4.2
#9 10.41 + numpy==2.2.5
#9 10.41 + onnx==1.18.0
#9 10.41 + onnxruntime==1.22.0
#9 10.41 + optimum==1.25.0
#9 10.41 + packaging==25.0
#9 10.41 + pandas==2.2.3
#9 10.41 + pillow==11.2.1
#9 10.41 + propcache==0.3.1
#9 10.41 + protobuf==6.30.2
#9 10.41 + pyarrow==20.0.0
#9 10.41 + python-dateutil==2.9.0.post0
#9 10.41 + pytz==2025.2
#9 10.41 + pyyaml==6.0.2
#9 10.41 + regex==2024.11.6
#9 10.41 + requests==2.32.3
#9 10.41 + safetensors==0.5.3
#9 10.41 + scikit-learn==1.6.1
#9 10.41 + scipy==1.15.3
#9 10.41 + sentence-transformers==4.1.0
#9 10.41 + setuptools==80.4.0
#9 10.41 + six==1.17.0
#9 10.41 + sympy==1.14.0
#9 10.41 + threadpoolctl==3.6.0
#9 10.41 + tokenizers==0.21.1
#9 10.41 + torch==2.7.0
#9 10.41 + tqdm==4.67.1
#9 10.41 + transformers==4.51.3
#9 10.41 + typing-extensions==4.13.2
#9 10.41 + tzdata==2025.2
#9 10.41 + urllib3==2.4.0
#9 10.41 + xxhash==3.5.0
#9 10.41 + yarl==1.20.0
#9 10.42 Package Version
#9 10.42 --------------------- -----------
#9 10.42 aiohappyeyeballs 2.6.1
#9 10.42 aiohttp 3.11.18
#9 10.42 aiosignal 1.3.2
#9 10.42 attrs 25.3.0
#9 10.42 certifi 2025.4.26
#9 10.42 charset-normalizer 3.4.2
#9 10.42 coloredlogs 15.0.1
#9 10.42 datasets 3.6.0
#9 10.42 dill 0.3.8
#9 10.42 filelock 3.18.0
#9 10.42 flatbuffers 25.2.10
#9 10.42 frozenlist 1.6.0
#9 10.42 fsspec 2025.3.0
#9 10.42 hf-xet 1.1.1
#9 10.42 huggingface-hub 0.31.2
#9 10.42 humanfriendly 10.0
#9 10.42 idna 3.10
#9 10.42 jinja2 3.1.6
#9 10.42 joblib 1.5.0
#9 10.42 markupsafe 3.0.2
#9 10.42 mpmath 1.3.0
#9 10.42 multidict 6.4.3
#9 10.42 multiprocess 0.70.16
#9 10.42 networkx 3.4.2
#9 10.42 numpy 2.2.5
#9 10.42 onnx 1.18.0
#9 10.42 onnxruntime 1.22.0
#9 10.42 optimum 1.25.0
#9 10.42 packaging 25.0
#9 10.42 pandas 2.2.3
#9 10.42 pillow 11.2.1
#9 10.42 propcache 0.3.1
#9 10.42 protobuf 6.30.2
#9 10.42 pyarrow 20.0.0
#9 10.42 python-dateutil 2.9.0.post0
#9 10.42 pytz 2025.2
#9 10.42 pyyaml 6.0.2
#9 10.42 regex 2024.11.6
#9 10.42 requests 2.32.3
#9 10.42 safetensors 0.5.3
#9 10.42 scikit-learn 1.6.1
#9 10.42 scipy 1.15.3
#9 10.42 sentence-transformers 4.1.0
#9 10.42 setuptools 80.4.0
#9 10.42 six 1.17.0
#9 10.42 sympy 1.14.0
#9 10.42 threadpoolctl 3.6.0
#9 10.42 tokenizers 0.21.1
#9 10.42 torch 2.7.0
#9 10.42 tqdm 4.67.1
#9 10.42 transformers 4.51.3
#9 10.42 typing-extensions 4.13.2
#9 10.42 tzdata 2025.2
#9 10.42 urllib3 2.4.0
#9 10.42 xxhash 3.5.0
#9 10.42 yarl 1.20.0
#9 23.88 Too many ONNX model files were found in onnx/model.onnx ,onnx/model_O1.onnx ,onnx/model_O2.onnx ,onnx/model_O3.onnx ,onnx/model_O4.onnx ,onnx/model_qint8_arm64.onnx ,onnx/model_qint8_avx512.onnx ,onnx/model_qint8_avx512_vnni.onnx ,onnx/model_quint8_avx2.onnx. specify which one to load by using the `file_name` and/or the `subfolder` arguments. Loading the file model.onnx in the subfolder onnx.
#9 34.54 Traceback (most recent call last):
#9 34.55 File "/app/test.py", line 6, in <module>
#9 34.55 embeddings = model.encode(sentences)
#9 34.55 File "/app/.venv/lib/python3.13/site-packages/sentence_transformers/SentenceTransformer.py", line 685, in encode
#9 34.55 out_features = self.forward(features, **kwargs)
#9 34.55 File "/app/.venv/lib/python3.13/site-packages/sentence_transformers/SentenceTransformer.py", line 758, in forward
#9 34.55 input = module(input, **module_kwargs)
#9 34.55 File "/app/.venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
#9 34.55 return self._call_impl(*args, **kwargs)
#9 34.55 ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
#9 34.55 File "/app/.venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
#9 34.55 return forward_call(*args, **kwargs)
#9 34.55 File "/app/.venv/lib/python3.13/site-packages/sentence_transformers/models/Transformer.py", line 442, in forward
#9 34.55 outputs = self.auto_model(**trans_features, **kwargs, return_dict=True)
#9 34.55 File "/app/.venv/lib/python3.13/site-packages/optimum/modeling_base.py", line 99, in __call__
#9 34.55 return self.forward(*args, **kwargs)
#9 34.55 ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
#9 34.55 TypeError: ORTModelForFeatureExtraction.forward() got an unexpected keyword argument 'return_dict'
#9 ERROR: process "/bin/sh -c uv venv --python 3.13 /app/.venv && uv pip install hf_xet sentence-transformers \"${OPTIMUM_PACKAGE}\" && uv pip list && uv run python test.py" did not complete successfully: exit code: 1
------
> [5/5] RUN uv venv --python 3.13 /app/.venv && uv pip install hf_xet sentence-transformers "optimum[onnxruntime]==1.25.0" && uv pip list && uv run python test.py:
34.55 return self._call_impl(*args, **kwargs)
34.55 ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
34.55 File "/app/.venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
34.55 return forward_call(*args, **kwargs)
34.55 File "/app/.venv/lib/python3.13/site-packages/sentence_transformers/models/Transformer.py", line 442, in forward
34.55 outputs = self.auto_model(**trans_features, **kwargs, return_dict=True)
34.55 File "/app/.venv/lib/python3.13/site-packages/optimum/modeling_base.py", line 99, in __call__
34.55 return self.forward(*args, **kwargs)
34.55 ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
34.55 TypeError: ORTModelForFeatureExtraction.forward() got an unexpected keyword argument 'return_dict'
------
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
Dockerfile:10
--------------------
9 | ARG OPTIMUM_PACKAGE=optimum[onnxruntime]@git+https://github.com/huggingface/optimum.git@${OPTIMUM_REF}
10 | >>> RUN uv venv --python 3.13 /app/.venv && \
11 | >>> uv pip install hf_xet sentence-transformers "${OPTIMUM_PACKAGE}" && \
12 | >>> uv pip list && \
13 | >>> uv run python test.py
14 |
--------------------
ERROR: failed to solve: process "/bin/sh -c uv venv --python 3.13 /app/.venv && uv pip install hf_xet sentence-transformers \"${OPTIMUM_PACKAGE}\" && uv pip list && uv run python test.py" did not complete successfully: exit code: 1
Expected behavior
Works with the commit in the dockerfile
> docker build --progress plain .
#0 building with "default" instance using docker-container driver
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 493B done
#1 DONE 0.0s
#2 [internal] load metadata for ghcr.io/astral-sh/uv:debian-slim
#2 DONE 1.3s
#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s
#4 [1/5] FROM ghcr.io/astral-sh/uv:debian-slim@sha256:f013f0a1c9f8b3186d2d3ab8d79ad4806f205e55430a262a7baa9dd6afbbfd12
#4 resolve ghcr.io/astral-sh/uv:debian-slim@sha256:f013f0a1c9f8b3186d2d3ab8d79ad4806f205e55430a262a7baa9dd6afbbfd12 done
#4 DONE 0.0s
#5 [internal] load build context
#5 transferring context: 332B done
#5 DONE 0.0s
#6 [2/5] WORKDIR /app
#6 CACHED
#7 [3/5] RUN apt-get update && apt-get install --yes git
#7 CACHED
#8 [4/5] COPY test.py test.py
#8 DONE 0.0s
#9 [5/5] RUN uv venv --python 3.13 /app/.venv && uv pip install hf_xet sentence-transformers "optimum[onnxruntime]@git+https://github.com/huggingface/optimum.git@cf5c6ecdf62e48b0070da953ff5089b85bd4519f" && uv pip list && uv run python test.py
#9 0.477 Downloading cpython-3.13.3-linux-aarch64-gnu (download) (16.8MiB)
#9 1.272 Downloading cpython-3.13.3-linux-aarch64-gnu (download)
#9 1.388 Using CPython 3.13.3
#9 1.388 Creating virtual environment at: .venv
#9 1.930 Updating https://github.com/huggingface/optimum.git (cf5c6ecdf62e48b0070da953ff5089b85bd4519f)
#9 3.695 Updated https://github.com/huggingface/optimum.git (cf5c6ecdf62e48b0070da953ff5089b85bd4519f)
#9 4.935 Resolved 56 packages in 3.43s
#9 4.939 Building optimum @ git+https://github.com/huggingface/optimum.git@cf5c6ecdf62e48b0070da953ff5089b85bd4519f
#9 4.963 Downloading transformers (9.9MiB)
#9 4.966 Downloading sympy (6.0MiB)
#9 4.971 Downloading numpy (13.4MiB)
#9 4.972 Downloading pillow (4.3MiB)
#9 4.972 Downloading networkx (1.6MiB)
#9 4.972 Downloading aiohttp (1.6MiB)
#9 4.972 Downloading scipy (33.5MiB)
#9 4.972 Downloading scikit-learn (11.6MiB)
#9 4.972 Downloading torch (94.4MiB)
#9 4.972 Downloading hf-xet (23.3MiB)
#9 4.973 Downloading tokenizers (2.8MiB)
#9 4.973 Downloading pandas (14.5MiB)
#9 4.973 Downloading onnxruntime (13.8MiB)
#9 4.977 Downloading onnx (16.6MiB)
#9 4.986 Downloading pyarrow (38.8MiB)
#9 5.427 Built optimum @ git+https://github.com/huggingface/optimum.git@cf5c6ecdf62e48b0070da953ff5089b85bd4519f
#9 5.645 Downloading networkx
#9 5.646 Downloading aiohttp
#9 5.978 Downloading tokenizers
#9 6.345 Downloading pillow
#9 6.787 Downloading sympy
#9 7.515 Downloading transformers
#9 7.890 Downloading scikit-learn
#9 8.269 Downloading numpy
#9 8.365 Downloading onnxruntime
#9 8.560 Downloading pandas
#9 8.798 Downloading onnx
#9 9.303 Downloading hf-xet
#9 10.07 Downloading scipy
#9 10.30 Downloading pyarrow
#9 12.30 Downloading torch
#9 12.30 Prepared 55 packages in 7.36s
#9 12.58 Installed 56 packages in 282ms
#9 12.58 + aiohappyeyeballs==2.6.1
#9 12.58 + aiohttp==3.11.18
#9 12.58 + aiosignal==1.3.2
#9 12.58 + attrs==25.3.0
#9 12.58 + certifi==2025.4.26
#9 12.58 + charset-normalizer==3.4.2
#9 12.58 + coloredlogs==15.0.1
#9 12.58 + datasets==3.6.0
#9 12.58 + dill==0.3.8
#9 12.58 + filelock==3.18.0
#9 12.58 + flatbuffers==25.2.10
#9 12.58 + frozenlist==1.6.0
#9 12.58 + fsspec==2025.3.0
#9 12.58 + hf-xet==1.1.1
#9 12.58 + huggingface-hub==0.31.2
#9 12.58 + humanfriendly==10.0
#9 12.58 + idna==3.10
#9 12.58 + jinja2==3.1.6
#9 12.58 + joblib==1.5.0
#9 12.58 + markupsafe==3.0.2
#9 12.58 + mpmath==1.3.0
#9 12.58 + multidict==6.4.3
#9 12.58 + multiprocess==0.70.16
#9 12.58 + networkx==3.4.2
#9 12.58 + numpy==2.2.5
#9 12.58 + onnx==1.18.0
#9 12.58 + onnxruntime==1.22.0
#9 12.58 + optimum==1.25.0.dev0 (from git+https://github.com/huggingface/optimum.git@cf5c6ecdf62e48b0070da953ff5089b85bd4519f)
#9 12.58 + packaging==25.0
#9 12.58 + pandas==2.2.3
#9 12.58 + pillow==11.2.1
#9 12.58 + propcache==0.3.1
#9 12.58 + protobuf==6.30.2
#9 12.58 + pyarrow==20.0.0
#9 12.58 + python-dateutil==2.9.0.post0
#9 12.58 + pytz==2025.2
#9 12.58 + pyyaml==6.0.2
#9 12.58 + regex==2024.11.6
#9 12.58 + requests==2.32.3
#9 12.58 + safetensors==0.5.3
#9 12.58 + scikit-learn==1.6.1
#9 12.58 + scipy==1.15.3
#9 12.58 + sentence-transformers==4.1.0
#9 12.58 + setuptools==80.4.0
#9 12.58 + six==1.17.0
#9 12.58 + sympy==1.14.0
#9 12.58 + threadpoolctl==3.6.0
#9 12.58 + tokenizers==0.21.1
#9 12.58 + torch==2.7.0
#9 12.58 + tqdm==4.67.1
#9 12.58 + transformers==4.51.3
#9 12.58 + typing-extensions==4.13.2
#9 12.58 + tzdata==2025.2
#9 12.58 + urllib3==2.4.0
#9 12.58 + xxhash==3.5.0
#9 12.58 + yarl==1.20.0
#9 12.60 Package Version
#9 12.60 --------------------- -----------
#9 12.60 aiohappyeyeballs 2.6.1
#9 12.60 aiohttp 3.11.18
#9 12.60 aiosignal 1.3.2
#9 12.60 attrs 25.3.0
#9 12.60 certifi 2025.4.26
#9 12.60 charset-normalizer 3.4.2
#9 12.60 coloredlogs 15.0.1
#9 12.60 datasets 3.6.0
#9 12.60 dill 0.3.8
#9 12.60 filelock 3.18.0
#9 12.60 flatbuffers 25.2.10
#9 12.60 frozenlist 1.6.0
#9 12.60 fsspec 2025.3.0
#9 12.60 hf-xet 1.1.1
#9 12.60 huggingface-hub 0.31.2
#9 12.60 humanfriendly 10.0
#9 12.60 idna 3.10
#9 12.60 jinja2 3.1.6
#9 12.60 joblib 1.5.0
#9 12.60 markupsafe 3.0.2
#9 12.60 mpmath 1.3.0
#9 12.60 multidict 6.4.3
#9 12.60 multiprocess 0.70.16
#9 12.60 networkx 3.4.2
#9 12.60 numpy 2.2.5
#9 12.60 onnx 1.18.0
#9 12.60 onnxruntime 1.22.0
#9 12.60 optimum 1.25.0.dev0
#9 12.60 packaging 25.0
#9 12.60 pandas 2.2.3
#9 12.60 pillow 11.2.1
#9 12.60 propcache 0.3.1
#9 12.60 protobuf 6.30.2
#9 12.60 pyarrow 20.0.0
#9 12.60 python-dateutil 2.9.0.post0
#9 12.60 pytz 2025.2
#9 12.60 pyyaml 6.0.2
#9 12.60 regex 2024.11.6
#9 12.60 requests 2.32.3
#9 12.60 safetensors 0.5.3
#9 12.60 scikit-learn 1.6.1
#9 12.60 scipy 1.15.3
#9 12.60 sentence-transformers 4.1.0
#9 12.60 setuptools 80.4.0
#9 12.60 six 1.17.0
#9 12.60 sympy 1.14.0
#9 12.60 threadpoolctl 3.6.0
#9 12.60 tokenizers 0.21.1
#9 12.60 torch 2.7.0
#9 12.60 tqdm 4.67.1
#9 12.60 transformers 4.51.3
#9 12.60 typing-extensions 4.13.2
#9 12.60 tzdata 2025.2
#9 12.60 urllib3 2.4.0
#9 12.60 xxhash 3.5.0
#9 12.60 yarl 1.20.0
#9 28.07 Too many ONNX model files were found in onnx/model.onnx ,onnx/model_O1.onnx ,onnx/model_O2.onnx ,onnx/model_O3.onnx ,onnx/model_O4.onnx ,onnx/model_qint8_arm64.onnx ,onnx/model_qint8_avx512.onnx ,onnx/model_qint8_avx512_vnni.onnx ,onnx/model_quint8_avx2.onnx. specify which one to load by using the `file_name` and/or the `subfolder` arguments. Loading the file model.onnx in the subfolder onnx.
#9 38.40 [[ 6.76569194e-02 6.34959638e-02 4.87131067e-02 7.93049335e-02
#9 38.40 3.74480560e-02 2.65279319e-03 3.93749848e-02 -7.09842052e-03
#9 38.40 5.93614578e-02 3.15369815e-02 6.00979626e-02 -5.29052541e-02
#9 38.40 4.06067669e-02 -2.59309039e-02 2.98428722e-02 1.12687389e-03
#9 38.40 7.35149160e-02 -5.03818877e-02 -1.22386597e-01 2.37028282e-02
#9 38.40 2.97265127e-02 4.24767956e-02 2.56337766e-02 1.99518143e-03
#9 38.40 -5.69191538e-02 -2.71598324e-02 -3.29034999e-02 6.60248846e-02
#9 38.40 1.19007163e-01 -4.58790734e-02 -7.26214200e-02 -3.25840004e-02
#9 38.40 5.23413755e-02 4.50553149e-02 8.25300720e-03 3.67023796e-02
#9 38.40 -1.39415339e-02 6.53918684e-02 -2.64272597e-02 2.06399651e-04
#9 38.40 -1.36643089e-02 -3.62810418e-02 -1.95043422e-02 -2.89738514e-02
#9 38.40 3.94270271e-02 -8.84090811e-02 2.62431917e-03 1.36713684e-02
#9 38.40 4.83062677e-02 -3.11565474e-02 -1.17329203e-01 -5.11690304e-02
#9 38.40 -8.85289088e-02 -2.18963176e-02 1.42986188e-02 4.44167964e-02
#9 38.40 -1.34815322e-02 7.43392482e-02 2.66382899e-02 -1.98762491e-02
#9 38.40 1.79191604e-02 -1.06052393e-02 -9.04263780e-02 2.13269368e-02
#9 38.40 1.41204804e-01 -6.47172937e-03 -1.40381209e-03 -1.53609440e-02
#9 38.40 -8.73571783e-02 7.22174570e-02 2.01402996e-02 4.25587334e-02
#9 38.40 -3.49014252e-02 3.19537008e-04 -8.02971274e-02 -3.27472799e-02
#9 38.40 2.85268035e-02 -5.13658077e-02 1.09389171e-01 8.19327235e-02
#9 38.40 -9.84039828e-02 -9.34095904e-02 -1.51292104e-02 4.51249033e-02
#9 38.40 4.94172201e-02 -2.51867492e-02 1.57077257e-02 -1.29290789e-01
#9 38.40 5.31890942e-03 4.02343133e-03 -2.34572235e-02 -6.72983527e-02
#9 38.40 2.92281378e-02 -2.60845013e-02 1.30624967e-02 -3.11662946e-02
#9 38.40 -4.82713692e-02 -5.58859669e-02 -3.87505442e-02 1.20010801e-01
#9 38.40 -1.03924265e-02 4.89705056e-02 5.53536750e-02 4.49358188e-02
#9 38.40 -4.00971808e-03 -1.02959715e-01 -2.92968303e-02 -5.83402738e-02
#9 38.40 2.70473063e-02 -2.20169723e-02 -7.22240806e-02 -4.13870141e-02
#9 38.40 -1.93297844e-02 2.73328787e-03 2.76943581e-04 -9.67588425e-02
#9 38.40 -1.00574724e-01 -1.41922571e-02 -8.07892531e-02 4.53925319e-02
#9 38.40 2.45041382e-02 5.97613826e-02 -7.38184303e-02 1.19844424e-02
#9 38.40 -6.63403124e-02 -7.69044831e-02 3.85157652e-02 -5.59362513e-33
#9 38.40 2.80013364e-02 -5.60784936e-02 -4.86601442e-02 2.15569846e-02
#9 38.40 6.01981133e-02 -4.81402390e-02 -3.50246914e-02 1.93313658e-02
#9 38.40 -1.75151769e-02 -3.89210060e-02 -3.81069398e-03 -1.70287974e-02
#9 38.40 2.82099321e-02 1.28290467e-02 4.71601486e-02 6.21030256e-02
#9 38.40 -6.43589124e-02 1.29285619e-01 -1.31230596e-02 5.23069240e-02
#9 38.40 -3.73680517e-02 2.89094392e-02 -1.68981105e-02 -2.37330385e-02
#9 38.40 -3.33491936e-02 -5.16762510e-02 1.55356992e-02 2.08802652e-02
#9 38.40 -1.25371413e-02 4.59578931e-02 3.72720994e-02 2.80566830e-02
#9 38.40 -5.90005033e-02 -1.16987973e-02 4.92182374e-02 4.70329113e-02
#9 38.40 7.35487565e-02 -3.70530188e-02 3.98458308e-03 1.06412442e-02
#9 38.40 -1.61521471e-04 -5.27165905e-02 2.75928210e-02 -3.92921641e-02
#9 38.40 8.44717845e-02 4.86860536e-02 -4.85873176e-03 1.79948974e-02
#9 38.40 -4.28570583e-02 1.23374788e-02 6.39955420e-03 4.04822901e-02
#9 38.40 1.48887476e-02 -1.53941428e-02 7.62948170e-02 2.37043127e-02
#9 38.40 4.45237420e-02 5.08196093e-02 -2.31253868e-03 -1.88737176e-02
#9 38.40 -1.23335803e-02 4.66001965e-02 -5.63438162e-02 6.29927218e-02
#9 38.40 -3.15535031e-02 3.24912146e-02 2.34673452e-02 -6.55438825e-02
#9 38.40 2.01709718e-02 2.57081520e-02 -1.23868529e-02 -8.36487953e-03
#9 38.40 -6.64377362e-02 9.43073630e-02 -3.57092842e-02 -3.42482813e-02
#9 38.40 -6.66355062e-03 -8.01521633e-03 -3.09711006e-02 4.33012024e-02
#9 38.40 -8.21392145e-03 -1.50795013e-01 3.07691768e-02 4.00718674e-02
#9 38.40 -3.79293263e-02 1.93216477e-03 4.00530510e-02 -8.77074823e-02
#9 38.40 -3.68491597e-02 8.57965183e-03 -3.19252238e-02 -1.25258025e-02
#9 38.40 7.35538825e-02 1.34736614e-03 2.05918420e-02 2.71098330e-33
#9 38.40 -5.18576913e-02 5.78360260e-02 -9.18985605e-02 3.94421369e-02
#9 38.40 1.05576575e-01 -1.96912251e-02 6.18402436e-02 -7.63465539e-02
#9 38.40 2.40880363e-02 9.40048695e-02 -1.16535537e-01 3.71197872e-02
#9 38.40 5.22425137e-02 -3.95861501e-03 5.72213680e-02 5.32851275e-03
#9 38.40 1.24016874e-01 1.39022209e-02 -1.10249780e-02 3.56052890e-02
#9 38.40 -3.30754630e-02 8.16573948e-02 -1.52003653e-02 6.05584718e-02
#9 38.40 -6.01397380e-02 3.26102152e-02 -3.48296873e-02 -1.69882309e-02
#9 38.40 -9.74907055e-02 -2.71484517e-02 1.74711552e-03 -7.68982172e-02
#9 38.40 -4.31858376e-02 -1.89984739e-02 -2.91660856e-02 5.77488095e-02
#9 38.40 2.41821241e-02 -1.16902022e-02 -6.21435605e-02 2.84352321e-02
#9 38.40 -2.37452885e-04 -2.51782909e-02 4.39629704e-03 8.12839791e-02
#9 38.40 3.64184640e-02 -6.04006089e-02 -3.65518108e-02 -7.93748125e-02
#9 38.40 -5.08525688e-03 6.69699162e-02 -1.17784359e-01 3.23743634e-02
#9 38.40 -4.71251979e-02 -1.34459585e-02 -9.48444679e-02 8.24947841e-03
#9 38.40 -1.06748957e-02 -6.81881830e-02 1.11821503e-03 2.48020105e-02
#9 38.40 -6.35889769e-02 2.84492355e-02 -2.61303950e-02 8.58111009e-02
#9 38.40 1.14682242e-01 -5.35345934e-02 -5.63588180e-02 4.26008999e-02
#9 38.40 1.09454403e-02 2.09578965e-02 1.00131169e-01 3.26051600e-02
#9 38.40 -1.84208766e-01 -3.93208414e-02 -6.91454932e-02 -6.38104826e-02
#9 38.40 -6.56385943e-02 -6.41255826e-03 -4.79612909e-02 -7.68132955e-02
#9 38.40 2.95384098e-02 -2.29948536e-02 4.17037159e-02 -2.50047892e-02
#9 38.40 -4.54509538e-03 -4.17136848e-02 -1.32288942e-02 -6.38357699e-02
#9 38.40 -2.46474869e-03 -1.37337595e-02 1.68976877e-02 -6.30397648e-02
#9 38.40 8.98880884e-02 4.18171100e-02 -1.85687654e-02 -1.80442132e-08
#9 38.40 -1.67998262e-02 -3.21577825e-02 6.30383566e-02 -4.13092300e-02
#9 38.40 4.44819145e-02 2.02468364e-03 6.29592165e-02 -5.17372647e-03
#9 38.40 -1.00444118e-02 -3.05640027e-02 3.52672748e-02 5.58581762e-02
#9 38.40 -4.67125326e-02 3.45104188e-02 3.29577997e-02 4.30114754e-02
#9 38.40 2.94360649e-02 -3.03165242e-02 -1.71107706e-02 7.37485513e-02
#9 38.40 -5.47909588e-02 2.77515799e-02 6.20161463e-03 1.58801377e-02
#9 38.40 3.42978574e-02 -5.15752705e-03 2.35079620e-02 7.53135532e-02
#9 38.40 1.92842726e-02 3.36197540e-02 5.09103425e-02 1.52497008e-01
#9 38.40 1.64207295e-02 2.70528905e-02 3.75162102e-02 2.18553953e-02
#9 38.40 5.66333793e-02 -3.95746715e-02 7.12313578e-02 -5.41376993e-02
#9 38.40 1.03767740e-03 2.11854074e-02 -3.56309190e-02 1.09016977e-01
#9 38.40 2.76530627e-03 3.13996859e-02 1.38420053e-03 -3.45738195e-02
#9 38.40 -4.59278226e-02 2.88084038e-02 7.16899941e-03 4.84684817e-02
#9 38.40 2.61018351e-02 -9.44071636e-03 2.82169543e-02 3.48724052e-02
#9 38.40 3.69099155e-02 -8.58949311e-03 -3.53205800e-02 -2.47856248e-02
#9 38.40 -1.91921238e-02 3.80707383e-02 5.99653795e-02 -4.22287174e-02]
#9 38.40 [ 8.64385590e-02 1.02762602e-01 5.39447879e-03 2.04442511e-03
#9 38.40 -9.96332243e-03 2.53854804e-02 4.92874719e-02 -3.06265969e-02
#9 38.40 6.87254369e-02 1.01365549e-02 7.75397271e-02 -9.00807530e-02
#9 38.40 6.10618712e-03 -5.69899119e-02 1.41715081e-02 2.80491561e-02
#9 38.40 -8.68464336e-02 7.64399543e-02 -1.03491217e-01 -6.77437708e-02
#9 38.40 6.99947625e-02 8.44250917e-02 -7.24915508e-03 1.04770213e-02
#9 38.40 1.34020960e-02 6.77576736e-02 -9.42085981e-02 -3.71689722e-02
#9 38.40 5.22617325e-02 -3.10853086e-02 -9.63406712e-02 1.57717112e-02
#9 38.40 2.57866718e-02 7.85244629e-02 7.89949745e-02 1.91517472e-02
#9 38.40 1.64356939e-02 3.10081034e-03 3.81311402e-02 2.37090662e-02
#9 38.40 1.05389431e-02 -4.40644287e-02 4.41738144e-02 -2.58727949e-02
#9 38.40 6.15378246e-02 -4.05427627e-02 -8.64140168e-02 3.19722518e-02
#9 38.40 -8.90732161e-04 -2.44437065e-02 -9.19721350e-02 2.33939476e-02
#9 38.40 -8.30293968e-02 4.41510528e-02 -2.49693077e-02 6.23020716e-02
#9 38.40 -1.30343717e-03 7.51395598e-02 2.46384218e-02 -6.47244826e-02
#9 38.40 -1.17727816e-01 3.83392349e-02 -9.11767408e-02 6.35445863e-02
#9 38.40 7.62739852e-02 -8.80241692e-02 9.54561122e-03 -4.69717085e-02
#9 38.40 -8.41740072e-02 3.88823636e-02 -1.14393659e-01 6.28858944e-03
#9 38.40 -3.49361561e-02 2.39750966e-02 -3.31316814e-02 -1.57244094e-02
#9 38.40 -3.78955863e-02 -8.81245546e-03 7.06118643e-02 3.28066237e-02
#9 38.40 2.03671725e-03 -1.12278923e-01 6.79719634e-03 1.22765861e-02
#9 38.40 3.35302874e-02 -1.36199929e-02 -2.25490108e-02 -2.25228760e-02
#9 38.40 -2.03194320e-02 5.04297577e-02 -7.48652965e-02 -8.22822005e-02
#9 38.40 7.65963495e-02 4.93392311e-02 -3.75552736e-02 1.44634414e-02
#9 38.40 -5.72458245e-02 -1.79954115e-02 1.09697968e-01 1.19462714e-01
#9 38.40 8.09161400e-04 6.17057942e-02 3.26322094e-02 -1.30780146e-01
#9 38.40 -1.48636654e-01 -6.16232865e-02 4.33885865e-02 2.67129447e-02
#9 38.40 1.39786145e-02 -3.94002460e-02 -2.52710450e-02 3.87744419e-03
#9 38.40 3.58664542e-02 -6.15421385e-02 3.76661085e-02 2.67565101e-02
#9 38.40 -3.82658802e-02 -3.54794189e-02 -2.39227712e-02 8.67977440e-02
#9 38.40 -1.84063390e-02 7.71039724e-02 1.39868143e-03 7.00383261e-02
#9 38.40 -4.77878079e-02 -7.89819658e-02 5.10813817e-02 -2.99868297e-33
#9 38.40 -3.91646735e-02 -2.56212987e-03 1.65210236e-02 9.48935747e-03
#9 38.40 -5.66219725e-02 6.57782853e-02 -4.77002785e-02 1.11662177e-02
#9 38.40 -5.73558137e-02 -9.16253030e-03 -2.17521302e-02 -5.59532158e-02
#9 38.40 -1.11422781e-02 9.32792872e-02 1.66764371e-02 -1.36724133e-02
#9 38.40 4.34388556e-02 1.87246385e-03 7.29945302e-03 5.16331978e-02
#9 38.40 4.80609015e-02 1.35341510e-01 -1.71739422e-02 -1.29697751e-02
#9 38.40 -7.50109106e-02 2.61108652e-02 2.69802064e-02 7.83044437e-04
#9 38.40 -4.87269387e-02 1.17842630e-02 -4.59580347e-02 -4.83214557e-02
#9 38.40 -1.95670668e-02 1.93889663e-02 1.98807381e-02 1.67431980e-02
#9 38.40 9.87800509e-02 -2.74087843e-02 2.34808996e-02 3.70233902e-03
#9 38.40 -6.14514090e-02 -1.21232111e-03 -9.50473454e-03 9.25161596e-03
#9 38.40 2.38444246e-02 8.61231312e-02 2.26790253e-02 5.45136281e-04
#9 38.40 3.47130187e-02 6.25460409e-03 -6.92769326e-03 3.92400846e-02
#9 38.40 1.15674939e-02 3.26280333e-02 6.22155145e-02 2.76113693e-02
#9 38.40 1.86884273e-02 3.55805792e-02 4.11795825e-02 1.54781695e-02
#9 38.40 4.22691293e-02 3.82248424e-02 1.00313425e-02 -2.83245221e-02
#9 38.40 4.47052270e-02 -4.10459489e-02 -4.50548576e-03 -5.44734895e-02
#9 38.40 2.62320433e-02 1.79861747e-02 -1.23118810e-01 -4.66952473e-02
#9 38.40 -1.35913473e-02 6.46710619e-02 3.57355247e-03 -1.22234495e-02
#9 38.40 -1.79382432e-02 -2.55502444e-02 2.37224326e-02 4.08658944e-03
#9 38.40 -6.51475191e-02 4.43651751e-02 4.68595475e-02 -3.25175300e-02
#9 38.40 4.02270583e-03 -3.97600979e-03 1.11939199e-02 -9.95597988e-02
#9 38.40 3.33168544e-02 8.01060051e-02 9.42692608e-02 -6.38294816e-02
#9 38.40 3.23151015e-02 -5.13553545e-02 -7.49871181e-03 5.30048173e-34
#9 38.40 -4.13194261e-02 9.49646756e-02 -1.06401399e-01 4.96590510e-02
#9 38.40 -3.41913961e-02 -3.16746160e-02 -1.71555914e-02 1.70098001e-03
#9 38.40 5.79758249e-02 -1.21782674e-03 -1.68536566e-02 -5.16912304e-02
#9 38.40 5.52998781e-02 -3.42647657e-02 3.08179352e-02 -3.10481712e-02
#9 38.40 9.27532092e-02 3.72663736e-02 -2.37397701e-02 4.45893779e-02
#9 38.40 1.46153541e-02 1.16239257e-01 -5.00111952e-02 3.88715863e-02
#9 38.40 4.24754061e-03 2.56976355e-02 3.27243805e-02 4.29908074e-02
#9 38.40 -1.36144143e-02 2.56122202e-02 1.06262416e-02 -8.46864581e-02
#9 38.40 -9.52982157e-02 1.08399846e-01 -7.51600564e-02 -1.37774246e-02
#9 38.40 6.37338832e-02 -4.49666847e-03 -3.25321630e-02 6.23613857e-02
#9 38.40 3.48052159e-02 -3.54922265e-02 -2.00222358e-02 3.66608165e-02
#9 38.40 -2.48837601e-02 1.01819290e-02 -7.01232776e-02 -4.31950577e-02
#9 38.40 2.95333099e-02 -2.95016944e-04 -3.45386006e-02 1.46675799e-02
#9 38.40 -9.83970389e-02 -4.70488556e-02 -8.85490235e-03 -8.89914855e-02
#9 38.40 3.50995921e-02 -1.29601985e-01 -4.98865806e-02 -6.12047426e-02
#9 38.40 -5.97797707e-02 9.46316496e-03 4.91218120e-02 -7.75026903e-02
#9 38.40 8.09726641e-02 -4.79256921e-02 2.34376360e-03 7.57031664e-02
#9 38.40 -2.40175221e-02 -1.52545599e-02 4.86739166e-02 -3.85968536e-02
#9 38.40 -7.04831481e-02 -1.20348241e-02 -3.88790853e-02 -7.76016340e-02
#9 38.40 -1.07243182e-02 1.04188025e-02 -2.13753432e-02 -9.17386711e-02
#9 38.40 -1.11344801e-02 -2.96066236e-02 2.46458501e-02 4.65711579e-03
#9 38.40 -1.63448974e-02 -3.95219997e-02 7.73373023e-02 -2.84733325e-02
#9 38.40 -3.69938300e-03 8.27665105e-02 -1.10409129e-02 3.13984193e-02
#9 38.40 5.35094030e-02 5.75145707e-02 -3.17622237e-02 -1.52911230e-08
#9 38.40 -7.99661279e-02 -4.76796776e-02 -8.59788582e-02 5.69616444e-02
#9 38.40 -4.08866145e-02 2.23833006e-02 -4.64451779e-03 -3.80130969e-02
#9 38.40 -3.10670696e-02 -1.07278153e-02 1.97698064e-02 7.77000329e-03
#9 38.40 -6.09470997e-03 -3.86376083e-02 2.80272160e-02 6.78138211e-02
#9 38.40 -2.35350169e-02 3.21748294e-02 8.02539289e-03 -2.39106901e-02
#9 38.40 -1.22002384e-03 3.14599462e-02 -5.24924062e-02 -8.06814618e-03
#9 38.40 3.14772525e-03 5.11496663e-02 -4.44104671e-02 6.36013150e-02
#9 38.40 3.85084078e-02 3.30432653e-02 -4.18727938e-03 4.95593026e-02
#9 38.40 -5.69604822e-02 -6.49716053e-03 -2.49793027e-02 -1.60866417e-02
#9 38.40 6.62288889e-02 -2.06309818e-02 1.08045779e-01 1.68546829e-02
#9 38.40 1.43813500e-02 -1.32126985e-02 -1.29387408e-01 6.95216358e-02
#9 38.40 -5.55773266e-02 -6.75414056e-02 -5.45818266e-03 -6.13590935e-03
#9 38.40 3.90841179e-02 -6.28779605e-02 3.74063849e-02 -1.16571207e-02
#9 38.40 1.29150013e-02 -5.52495383e-02 5.16075455e-02 -4.30838298e-03
#9 38.40 5.80247529e-02 1.86944921e-02 2.27810424e-02 3.21666002e-02
#9 38.40 5.37978634e-02 7.02848807e-02 7.49311820e-02 -8.41774866e-02]]
#9 DONE 40.4s
System Info
Who can help?
Geting the following error when using
SentenceTransformerswithbackend="onnx". It was working fine withoptimum[onnxruntime]==1.24.0.I think the commit that introduced the issue was f93964c by @Abdennacer-Badaoui, which removed
**kwargsfromORTModelForFeatureExtraction.forward(). Will it be ok to add back**kwargs?Related: huggingface/sentence-transformers#3363
ONNX Runtime: @JingyaHuang, @echarlaix
Information
Tasks
examplesfolder (such as GLUE/SQuAD, ...)Reproduction (minimal, reproducible, runnable)
Errors with
optimum[onnxruntime]==1.25.0.Expected behavior
Works with the commit in the dockerfile