Repository navigation
Python: use storage names for Redis search return fields and decode aliased vectors (#14753) - #14766
Open
MOHAMMED WASIM KHAN (wasim-builds) wants to merge 1 commit into
Conversation
…liased vectors (microsoft#14753) The Redis Hashset and JSON collections create their index attributes and store their values under storage_name or name, but the search query asked the server to RETURN the declared field name. A field with a storage_name was therefore missing from every search result, and the Hashset vector decode wrote the array under the declared name while the data model step reads the storage name, so a vector with a storage_name came back as raw bytes. Closes microsoft#14753 Co-authored-by: VANDRANKI <[email protected]>
MOHAMMED WASIM KHAN (wasim-builds)
requested a review
from a team
as a code owner
November 9, 2026 17:49
Muhammad Abdullah Butt (1AbdullahButt)
left a comment
Muhammad Abdullah Butt (1AbdullahButt)
left a comment
There was a problem hiding this comment.
Looks correct. Here's what I checked.
- Index attributes.
_definition_to_redis_fieldsbuilds every non-key field underfield.storage_name or field.name. For hashsets that's the attribute name. For JSON it's$.{storage_name}withas_name=storage_name.RETURNhas to name that attribute, which is what both_add_return_fieldsnow do. - Vector search. It already uses the storage name (
vector_field_name=vector_field.storage_name or vector_field.name,redis.py:378), so the vector query and the returned fields now agree. - Hashset vectors.
_deserialize_dict_to_data_modelreadsrecord.get(field.storage_name or field.name)for dict and dataclass records, and popsstorage_nametonamefor pydantic records. Decoding understorage_nameis the key that step reads. The extrarec[field.name]copy is harmless for both. - JSON records. These need no change on the way back: values come out under the alias, and the data-model step maps them.
git merge-treeagainstmainis clean.
I didn't run the tests.
Unrelated to this PR, but nearby: RedisJsonCollection._serialize_dicts_to_store_models assigns the vector field twice (the if field.field_type == "vector" line and the line after it do the same thing).
Reviewed with help from Claude Code.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Closes #14753
When a field in a Redis collection declares a
storage_name, the collection creates its index attributes and stores its values understorage_name or name. However,_add_return_fieldsinRedisHashsetCollectionandRedisJsonCollectionasked the server toRETURNthe declared field name. RediSearch only returns attributes named inRETURN, causing fields with astorage_nameto be missing from search results. Furthermore,RedisHashsetCollectionvector deserialization decoded the bytes intorec[field.name]while the subsequent data model step readsstorage_name, leaving vectors as raw bytes.Description
RedisHashsetCollection._add_return_fieldsandRedisJsonCollection._add_return_fields, addfield.storage_name or field.nameto the query's return fields instead offield.name.RedisHashsetCollection._deserialize_store_models_to_dicts, decode vectors in-place understorage_nameand also setrec[field.name].tests/unit/connectors/memory/test_redis_store.pycovering return fields construction, search against RETURN-limited results, and vector deserialization with aliased storage names.All 35 tests in
tests/unit/connectors/memory/test_redis_store.pypass cleanly.Co-authored with PRABHU KIRAN VANDRANKI (@VANDRANKI).
Contribution Checklist