Skip to content

Faiss compatibility with NumPy 2 (ImportError: numpy.core.multiarray failed to import) #3526

@pdarulewski

Description

@pdarulewski

Summary

I'm not really sure if this is something on my side or Faiss but from the logs it looks like Faiss needs to be recompiled with the newest NumPy release. AFAIK, there are a couple of C API changes: https://numpy.org/doc/stable/release/2.0.0-notes.html

Platform

OS: macOS 14.4.1 23E224 arm64

Faiss version: faiss-cpu==1.8.0

Installed from: pypi through poetry and pip

Faiss compilation options: none

Running on:

  • CPU
  • GPU

Interface:

  • C++
  • Python

Reproduction instructions

To reproduce:

  1. install numpy, faiss-cpu, packaging (apparently is required by faiss but it's not automatically installed by pip 24.0)
  2. open REPL and import faiss
∇ python3.11 -m venv .venv
∇ source .venv/bin/activate
∇ pip install numpy faiss-cpu packaging
Collecting numpy
  Using cached numpy-2.0.0-cp311-cp311-macosx_14_0_arm64.whl.metadata (60 kB)
Collecting faiss-cpu
  Using cached faiss_cpu-1.8.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (3.6 kB)
Collecting packaging
  Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Using cached numpy-2.0.0-cp311-cp311-macosx_14_0_arm64.whl (5.2 MB)
Using cached faiss_cpu-1.8.0-cp311-cp311-macosx_11_0_arm64.whl (3.1 MB)
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Installing collected packages: packaging, numpy, faiss-cpu
Successfully installed faiss-cpu-1.8.0 numpy-2.0.0 packaging-24.1
∇ python
Python 3.11.8 (main, Jun  7 2024, 13:22:20) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import faiss

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "<stdin>", line 1, in <module>
  File "/Users/pd/utils/scratch/.venv/lib/python3.11/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/Users/pd/utils/scratch/.venv/lib/python3.11/site-packages/faiss/loader.py", line 98, in <module>
    from .swigfaiss import *
  File "/Users/pd/utils/scratch/.venv/lib/python3.11/site-packages/faiss/swigfaiss.py", line 10, in <module>
    from . import _swigfaiss
AttributeError: _ARRAY_API not found
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/pd/utils/scratch/.venv/lib/python3.11/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/Users/pd/utils/scratch/.venv/lib/python3.11/site-packages/faiss/loader.py", line 98, in <module>
    from .swigfaiss import *
  File "/Users/pd/utils/scratch/.venv/lib/python3.11/site-packages/faiss/swigfaiss.py", line 10, in <module>
    from . import _swigfaiss
ImportError: numpy.core.multiarray failed to import

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    1.13.0

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions