Skip to content

Commit 6e5a4b7

Browse files
author
Patrick Ribbsaeter
authored
Python: fix(python/redis): restore vector search broken by redisvl>=0.5 API change and KeyError on include_vectors=False (#14278)
## Summary Fixes #13896. Vector search (`FT.SEARCH`) via the Python Redis connector was completely broken due to two independent bugs. Both have been fixed in this PR. --- ## Bug 1 — `process_results()` API break with redisvl >= 0.5 ### Root cause `redisvl` 0.5.0 changed the signature of `process_results()`: ```python # redisvl < 0.5 (old API) process_results(results, query, storage_type: StorageType) # redisvl >= 0.5 (new API) process_results(results, query, schema: IndexSchema) ``` The `pyproject.toml` pin `redisvl ~= 0.4` resolves to `>=0.4, <1.0` under PEP 440, so `uv.lock` silently picks up redisvl 0.15.x where the old API is gone. Every call to `collection.search()` raised: ``` VectorSearchExecutionException: An error occurred during the search: 'StorageType' object has no attribute 'index' ``` ### Fix Detect the redisvl API signature at runtime via `inspect.signature` by introspecting the third parameter name: - If `third_parameter == "storage_type"` (redisvl < 0.5), pass the `StorageType` enum. - If `third_parameter == "schema"` (redisvl >= 0.5), load the schema via `AsyncSearchIndex.from_existing(name=collection_name, redis_client=redis_database)` and pass `index.schema`. - If an unsupported third parameter is encountered, raise `VectorSearchExecutionException` rather than guessing. ```python parameters = list(inspect.signature(process_results).parameters.values()) if len(parameters) < 3: raise VectorSearchExecutionException("Unsupported redisvl process_results() signature.") third_parameter = parameters[2].name if third_parameter == "storage_type": return process_results(results, query, STORAGE_TYPE_MAP[collection_type]) if third_parameter == "schema": index = await AsyncSearchIndex.from_existing( name=collection_name, redis_client=redis_database, ) return process_results(results, query, index.schema) raise VectorSearchExecutionException( f"Unsupported redisvl process_results() parameter: {third_parameter}." ) ``` --- ## Bug 2 — `KeyError` in `RedisHashsetCollection._deserialize_store_models_to_dicts` ### Root cause The deserializer unconditionally decoded every vector field: ```python case "vector": dtype = DATATYPE_MAP_VECTOR[field.type_ or "default"] rec[field.name] = buffer_to_array(rec[field.name], dtype) # KeyError! ``` When `include_vectors=False` (the SDK default for search), Redis does not return vector fields in the result dict, so `rec[field.name]` raised `KeyError: 'vector'`. ### Fix Guard with an existence check before decoding: ```python case "vector": storage_name = field.storage_name or field.name if storage_name in rec: dtype = DATATYPE_MAP_VECTOR[field.type_ or "default"] rec[field.name] = buffer_to_array(rec[storage_name], dtype) else: rec[field.name] = None # vector not requested — leave as None ``` --- ## Impact Both bugs affected **every** Python user using the Redis connector for vector search: - Bug 1 blocked all search calls unconditionally. - Bug 2 blocked hashset search whenever `include_vectors=False` (the default). The connector was written against redisvl 0.4.x; the dependency pin allows installation of redisvl 0.15.x where the API changed. --- ## Testing - Unit tests in `python/tests/unit/connectors/memory/test_redis_store.py`: - `test_process_search_results_with_legacy_redisvl_api`: verifies legacy `storage_type` parameter dispatch. - `test_process_search_results_with_current_redisvl_api`: verifies `AsyncSearchIndex.from_existing` and `schema` dispatch. - `test_hash_deserialization_handles_omitted_vector`: verifies hashset deserialization when `vector` is omitted. - The fix is backward-compatible across supported redisvl versions without requiring a version pin change. - Existing upsert/get/delete integration tests are unaffected. Signed-off-by: Patrick Ribbsaeter <[email protected]>
1 parent 8aa1dc5 commit 6e5a4b7

2 files changed

Lines changed: 111 additions & 3 deletions

File tree

‎python/semantic_kernel/connectors/redis.py‎

Lines changed: 49 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
import ast
44
import asyncio
55
import contextlib
6+
import inspect
67
import json
78
import logging
89
import sys
@@ -17,6 +18,7 @@
1718
from redis.commands.search.field import Field as RedisField
1819
from redis.commands.search.field import NumericField, TagField, TextField, VectorField
1920
from redis.commands.search.index_definition import IndexDefinition, IndexType
21+
from redisvl.index import AsyncSearchIndex
2022
from redisvl.index.index import process_results
2123
from redisvl.query.filter import FilterExpression, Num, Tag, Text
2224
from redisvl.query.query import BaseQuery, VectorQuery
@@ -166,6 +168,32 @@ def _definition_to_redis_fields(
166168
return fields
167169

168170

171+
async def _process_search_results(
172+
results: Any,
173+
query: BaseQuery,
174+
collection_name: str,
175+
redis_database: Redis,
176+
collection_type: RedisCollectionTypes,
177+
) -> Any:
178+
"""Process RedisVL results across its old and current APIs."""
179+
parameters = list(inspect.signature(process_results).parameters.values())
180+
if len(parameters) < 3:
181+
raise VectorSearchExecutionException("Unsupported redisvl process_results() signature.")
182+
183+
third_parameter = parameters[2].name
184+
if third_parameter == "storage_type":
185+
return process_results(results, query, STORAGE_TYPE_MAP[collection_type])
186+
187+
if third_parameter == "schema":
188+
index = await AsyncSearchIndex.from_existing(
189+
name=collection_name,
190+
redis_client=redis_database,
191+
)
192+
return process_results(results, query, index.schema)
193+
194+
raise VectorSearchExecutionException(f"Unsupported redisvl process_results() parameter: {third_parameter}.")
195+
196+
169197
@release_candidate
170198
class RedisSettings(KernelBaseSettings):
171199
"""Redis model settings.
@@ -321,7 +349,18 @@ async def _inner_search(
321349
results = await self.redis_database.ft(self.collection_name).search( # type: ignore
322350
query=query.query, query_params=query.params
323351
)
324-
processed = process_results(results, query, STORAGE_TYPE_MAP[self.collection_type])
352+
try:
353+
processed = await _process_search_results(
354+
results,
355+
query,
356+
self.collection_name,
357+
self.redis_database,
358+
self.collection_type,
359+
)
360+
except VectorSearchExecutionException:
361+
raise
362+
except Exception as exc:
363+
raise VectorSearchExecutionException(f"An error occurred during the search: {exc}") from exc
325364
return KernelSearchResults(
326365
results=self._get_vector_search_results_from_results(desync_list(processed)),
327366
total_count=results.total,
@@ -616,8 +655,15 @@ def _deserialize_store_models_to_dicts(
616655
case FieldTypes.KEY:
617656
rec[field.name] = self._unget_redis_key(rec[field.name])
618657
case "vector":
619-
dtype = DATATYPE_MAP_VECTOR[field.type_ or "default"]
620-
rec[field.name] = buffer_to_array(rec[field.name], dtype)
658+
# When include_vectors=False (the default for search), the vector
659+
# field is not returned by Redis and will be absent from `rec`.
660+
# Guard against KeyError before attempting to decode the buffer.
661+
storage_name = field.storage_name or field.name
662+
if storage_name in rec:
663+
dtype = DATATYPE_MAP_VECTOR[field.type_ or "default"]
664+
rec[field.name] = buffer_to_array(rec[storage_name], dtype)
665+
else:
666+
rec[field.name] = None
621667
results.append(rec)
622668
return results
623669

‎python/tests/unit/connectors/memory/test_redis_store.py‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
from pytest import fixture, mark, raises
77
from redis.asyncio.client import Redis
88

9+
import semantic_kernel.connectors.redis as redis_module
910
from semantic_kernel.connectors.redis import (
1011
RedisCollectionTypes,
1112
RedisHashsetCollection,
@@ -306,3 +307,64 @@ async def test_create_index_manual(collection_hash, mock_ensure_collection_exist
306307
async def test_create_index_fail(collection_hash, mock_ensure_collection_exists):
307308
with raises(VectorStoreOperationException, match="Invalid index type supplied."):
308309
await collection_hash.ensure_collection_exists(index_definition="index_definition", fields="fields")
310+
311+
312+
async def test_process_search_results_with_legacy_redisvl_api():
313+
results = object()
314+
query = object()
315+
316+
def legacy_process_results(results_arg, query_arg, storage_type):
317+
assert results_arg is results
318+
assert query_arg is query
319+
assert storage_type is redis_module.StorageType.HASH
320+
return [{"id": "legacy"}]
321+
322+
with patch.object(redis_module, "process_results", new=legacy_process_results):
323+
processed = await redis_module._process_search_results(
324+
results,
325+
query,
326+
"test",
327+
object(),
328+
redis_module.RedisCollectionTypes.HASHSET,
329+
)
330+
331+
assert processed == [{"id": "legacy"}]
332+
333+
334+
async def test_process_search_results_with_current_redisvl_api():
335+
results = object()
336+
query = object()
337+
expected_schema = object()
338+
index = type("Index", (), {"schema": expected_schema})()
339+
redis_database = object()
340+
341+
def current_process_results(results_arg, query_arg, schema):
342+
assert results_arg is results
343+
assert query_arg is query
344+
assert schema is expected_schema
345+
return [{"id": "current"}]
346+
347+
with (
348+
patch.object(redis_module, "process_results", new=current_process_results),
349+
patch.object(
350+
redis_module.AsyncSearchIndex,
351+
"from_existing",
352+
new=AsyncMock(return_value=index),
353+
) as from_existing,
354+
):
355+
processed = await redis_module._process_search_results(
356+
results,
357+
query,
358+
"test",
359+
redis_database,
360+
redis_module.RedisCollectionTypes.HASHSET,
361+
)
362+
363+
from_existing.assert_awaited_once_with(name="test", redis_client=redis_database)
364+
assert processed == [{"id": "current"}]
365+
366+
367+
def test_hash_deserialization_handles_omitted_vector(collection_hash):
368+
records = collection_hash._deserialize_store_models_to_dicts([{"id": "id1", "content": "content"}])
369+
370+
assert records[0]["vector"] is None

0 commit comments

Comments
 (0)