LanceDB version
v0.13.0
What happened?
Query except FTS requires vector column name to be passed in.
Failed to infer vector column at
https://github.com/lancedb/lancedb/blob/main/python/python/lancedb/table.py#L1635
the vector_column_name is None without throwing exception.
vector_column_name = inf_vector_column_query(self.schema)
Proposal
- early error out if failed to infer vector column name from schema
- check why schema does not have vector column
Are there known steps to reproduce?
Test dataset
def _test_search_table() -> pa.Table:
return pa.Table.from_pandas(
pd.DataFrame(
{
"item_id": [1, 2, 3, 4, 5],
"inner_id": [1, 2, 3, 4, 5],
"category": ["a", "a", "b", "b", "c"],
"numeric_int": [2, 5, 3, 1, 4],
"numeric_float": [0.2, 0.5, 0.3, 0.1, 0.4],
"category_set": [
["d", "e"],
["d", "f"],
["d", "g"],
["g", "h"],
["h", "i"],
],
}
)
)
with search query
result = (
self._table_inner.search(query_type="hybrid")
.text(query_text)
.vector(text_query_encoder([query_text])[0])
.rerank(reranker=LinearCombinationReranker(text_semantic_search_weight))
.select([self.INNER_ID_NAME])
.limit(k)
.to_arrow()
.to_pylist()
)
throws exception
in text_semantic_search
.to_arrow()
../../.conda/envs/dev/lib/python3.11/site-packages/lancedb/query.py:1046: in to_arrow
vector_results = vector_future.result()
../../.conda/envs/dev/lib/python3.11/concurrent/futures/_base.py:449: in result
return self.__get_result()
../../.conda/envs/dev/lib/python3.11/concurrent/futures/_base.py:401: in __get_result
raise self._exception
../../.conda/envs/dev/lib/python3.11/concurrent/futures/thread.py:58: in run
result = self.fn(*self.args, **self.kwargs)
../../.conda/envs/dev/lib/python3.11/site-packages/lancedb/query.py:647: in to_arrow
return self.to_batches().read_all()
../../.conda/envs/dev/lib/python3.11/site-packages/lancedb/query.py:678: in to_batches
result_set = self._table._execute_query(query, batch_size)
../../.conda/envs/dev/lib/python3.11/site-packages/lancedb/table.py:1742: in _execute_query
return ds.scanner(
../../.conda/envs/dev/lib/python3.11/site-packages/lance/dataset.py:369: in scanner
builder = builder.nearest(**nearest)
../../.conda/envs/dev/lib/python3.11/site-packages/lance/dataset.py:2441: in nearest
if self.ds.schema.get_field_index(column) < 0:
pyarrow/types.pxi:3012: in pyarrow.lib.Schema.get_field_index
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E TypeError: expected bytes, NoneType found
schema.get_field_index(column) throws exception
pyarrow schema
'inner_id: int64, has_vector: bool, vector: fixed_size_list<item: double>[384], item_id: int64, category: string, numeric_int: int64, numeric_float: double, category_set: list<item: string>'
column is None
(Pdb) pp nearest
{'column': None,
'k': 3,
'metric': 'L2',
'nprobes': 20,
'q': <vector_values>,
'refine_factor': None
}
After pass in search(query, vector_column_name), the test passed.
LanceDB version
v0.13.0
What happened?
Query except FTS requires vector column name to be passed in.
Failed to infer vector column at
https://github.com/lancedb/lancedb/blob/main/python/python/lancedb/table.py#L1635
the
vector_column_nameis None without throwing exception.Proposal
Are there known steps to reproduce?
Test dataset
with search query
throws exception
schema.get_field_index(column) throws exception
pyarrow schema
column is None
After pass in search(query, vector_column_name), the test passed.