Skip to content

Ability to use IndexIDMap add_with_ids function with GPU based Cagra index #4107

@navneet1v

Description

@navneet1v

Description

I am trying to create a GPU Cagra index in a IndexIDMap index. But facing error like this:

  Traceback (most recent call last):
  File "/tmp/faiss-test.py", line 54, in <module>
    indexData(d, xb, ids, {}, "l2", "testgpuIndex.cagra.graph")
  File "/tmp/faiss-test.py", line 38, in indexData
    indexDataInIndex(idMapIVFPQIndex, ids, xb)
  File "/tmp/faiss-test.py", line 47, in indexDataInIndex
    index.add_with_ids(xb, ids)
  File "/opt/conda/lib/python3.11/site-packages/faiss/class_wrappers.py", line 251, in replacement_add_with_ids
    self.add_with_ids_c(n, swig_ptr(x), swig_ptr(ids))
  File "/opt/conda/lib/python3.11/site-packages/faiss/swigfaiss_avx2.py", line 11249, in add_with_ids
    return _swigfaiss_avx2.IndexIDMap_add_with_ids(self, n, x, xids)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Error in virtual void faiss::gpu::GpuIndex::add_with_ids(faiss::idx_t, const float*, const idx_t*) at /home/runner/miniconda3/conda-bld/faiss-pkg_1728491363260/work/faiss/gpu/GpuIndex.cu:120: Error: '!(this->is_trained)' failed: Index not trained

Based on my code reading what I can see is add_with_ids functionality is not supported with CagraIndex, but I think for the first time we should able to create the index, but subsequent adds should be blocked.

GpuIndexCagra is inherited from GpuIndex here and the exception is coming from here: and IndexIDMap uses add_with_ids

Is there any specific reason why this is blocked? If there is no specific reason I would like to contribute related to this.

Version

Faiss: 1.9.0

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions