Summary
Currently, the indexes that are supported in GPU are clearly listed in the wiki: https://github.com/facebookresearch/faiss/wiki/Faiss-on-the-GPU#implemented-indexes, however, it would be great if we could throw an error when the user attempts to move an unsupported GPU index instead of failing silently and running in the CPU.
Platform
Faiss version: faiss-gpu 1.7.4
Installed from: conda
Running on:
Interface:
Reproduction instructions
I have created an IndexPQ, attempted to move it to the GPU and conducted search. This results in the knn search taking a long time 236.77 seconds plus nvidia-smi pmon showcases we indeed never use the GPU:
# gpu pid type sm mem enc dec command
# Idx # C/G % % % % name
0 2711586 C 0 0 - - python3.10
1 - - - - - - -
0 2711586 C 0 0 - - python3.10
1 - - - - - - -
0 2711586 C 0 0 - - python3.10
import faiss
import numpy as np
import time
dimension = 128
n = 1000000
db_vectors = np.random.random((n, dimension)).astype('float32')
k = 3
dimension=128
query_vectors = np.random.random((n, dimension)).astype('float32')
code_size=16
res = faiss.StandardGpuResources()
start = time.time()
#define the pq index in cpu
index_pq = faiss.IndexPQ(dimension, code_size, 8)
index_pq.train(db_vectors)
index_pq.add(db_vectors)
gpu_index_pq = faiss.index_cpu_to_gpu(res, 0, index_pq)
distances_pq, indices_pq = gpu_index_pq.search(query_vectors, k)
end = time.time()
total_time = end - start
print("\n"+ str(total_time))
In contrast, with an IVFPQ (which is GPU supported) the run time is 3.48 seconds utilising the same number of vectors and we use the GPU according to nvidia-smi pmon as expected.
gpu pid type sm mem enc dec command
# Idx # C/G % % % % name
0 2711586 C 0 0 - - python3.10
1 - - - - - - -
0 2711586 C 0 0 - - python3.10
1 - - - - - - -
0 2711586 C 0 0 - - python3.10
1 - - - - - - -
0 2711586 C 25 6 - - python3.10
1 - - - - - - -
0 2711586 C 10 3 - - python3.10
dimension = 128
n = 1000000
db_vectors = np.random.random((n, dimension)).astype('float32')
k = 3
dimension=128
query_vectors = np.random.random((n, dimension)).astype('float32')
code_size=16
nlist=math.floor(math.sqrt(n))
res = faiss.StandardGpuResources()
#define the ivfpq index in cpu
start = time.time()
quantizer = faiss.IndexFlatL2(dimension) # coarse quantizer
#define the inverted index
index_ivfpq = faiss.IndexIVFPQ(quantizer, dimension, nlist, code_size, 8)
index_ivfpq.train(db_vectors)
index_ivfpq.add(db_vectors)
gpu_index_ivfpq = faiss.index_cpu_to_gpu(res, 0, index_ivfpq)
distances_ivfpq, indices_ivfpq = gpu_index_ivfpq.search(query_vectors, k)
end = time.time()
total_time = end - start
print("\n"+ str(total_time))
Summary
Currently, the indexes that are supported in GPU are clearly listed in the wiki: https://github.com/facebookresearch/faiss/wiki/Faiss-on-the-GPU#implemented-indexes, however, it would be great if we could throw an error when the user attempts to move an unsupported GPU index instead of failing silently and running in the CPU.
Platform
Faiss version: faiss-gpu 1.7.4
Installed from: conda
Running on:
Interface:
Reproduction instructions
I have created an
IndexPQ, attempted to move it to the GPU and conducted search. This results in the knn search taking a long time236.77 secondsplusnvidia-smi pmonshowcases we indeed never use the GPU:In contrast, with an IVFPQ (which is GPU supported) the run time is
3.48 secondsutilising the same number of vectors and we use the GPU according tonvidia-smi pmonas expected.