Python: Bug: QdrantCollection fails to upsert dict and dataclass records when the key or vector field has a storage_name, and pydantic payloads ignore it #14754

Description

QdrantCollection cannot upsert a dict or dataclass record when the key field or the vector field declares a storage_name. It raises VectorStoreModelSerializationException: Error serializing records: 'vector' (or 'id').

The base class hands the connector a dict keyed by storage name for dict and dataclass records, but keyed by field name for pydantic models. _serialize_dicts_to_store_models in qdrant.py pops the key and the vector by the declared field name. That works for pydantic models and for fields without a storage_name. It fails with a KeyError for the other record types.

The pydantic case has a quieter problem. The payload is stored under the declared names. Filters use the storage names (_lambda_parser only accepts definition.storage_names), so a filter on a field with a storage_name finds nothing for records that were written from a pydantic model.

Reproduction

I ran this on current main (df1cb33), Python 3.13, with the local in-memory Qdrant (location=":memory:"), so no server is involved.

import asyncio
import warnings
from dataclasses import dataclass

warnings.filterwarnings("ignore")
from pydantic import BaseModel

from semantic_kernel.connectors.qdrant import QdrantCollection
from semantic_kernel.data.vector import VectorStoreCollectionDefinition, VectorStoreField


def definition(key_alias=None, vector_alias=None):
    return VectorStoreCollectionDefinition(
        fields=[
            VectorStoreField("key", name="id", type="int", storage_name=key_alias),
            VectorStoreField("data", name="text", type="str", storage_name="doc_text", is_indexed=True),
            VectorStoreField("vector", name="vector", dimensions=2, type="float", storage_name=vector_alias),
        ]
    )


@dataclass
class DC:
    id: int
    text: str
    vector: list[float] | None = None


class PM(BaseModel):
    id: int
    text: str
    vector: list[float] | None = None


def record(kind):
    values = {"id": 1, "text": "a", "vector": [1.0, 0.0]}
    return values if kind is dict else kind(**values)


async def upsert(kind, defn, label):
    async with QdrantCollection(
        record_type=kind, definition=defn, collection_name="t", location=":memory:"
    ) as collection:
        await collection.ensure_collection_exists()
        try:
            await collection.upsert(record(kind))
            print(f"{kind.__name__:5s} {label}: upsert ok")
        except Exception as ex:
            print(f"{kind.__name__:5s} {label}: {type(ex).__name__}: {ex}")


async def filtered(kind):
    async with QdrantCollection(
        record_type=kind, definition=definition(), collection_name="t", location=":memory:"
    ) as collection:
        await collection.ensure_collection_exists()
        await collection.upsert(record(kind))
        results = await collection.search(vector=[1.0, 0.0], filter=lambda x: x.doc_text == "a")
        print(f"{kind.__name__:5s} filter on doc_text: {len([r async for r in results.results])} result(s)")


async def main():
    for kind in (dict, DC, PM):
        await upsert(kind, definition(vector_alias="doc_vector"), "vector storage_name")
    for kind in (dict, DC, PM):
        await upsert(kind, definition(key_alias="point_id"), "key storage_name   ")
    for kind in (dict, DC, PM):
        await filtered(kind)


asyncio.run(main())

Output on main:

dict  vector storage_name: VectorStoreModelSerializationException: Error serializing records: 'vector'
DC    vector storage_name: VectorStoreModelSerializationException: Error serializing records: 'vector'
PM    vector storage_name: upsert ok
dict  key storage_name   : VectorStoreModelSerializationException: Error serializing records: 'id'
DC    key storage_name   : VectorStoreModelSerializationException: Error serializing records: 'id'
PM    key storage_name   : upsert ok
dict  filter on doc_text: 1 result(s)
DC    filter on doc_text: 1 result(s)
PM    filter on doc_text: 0 result(s)

The same collection and the same data give a different result for each record type. A storage_name on a data field alone works for dict and dataclass records, which is why this is easy to miss.

Cause

In python/semantic_kernel/connectors/qdrant.py, _serialize_dicts_to_store_models calls record.pop(self._key_field_name) and record.pop(field.name) for the vectors. _key_field_name is the declared name. The base class _serialize_data_model_to_dict builds {field.storage_name or field.name: ...} for dict and dataclass records and uses model_dump() (declared names) for pydantic models. The rest of the payload is passed on unchanged, so pydantic payloads keep the declared names.

InMemoryCollection._inner_upsert already accepts both shapes: it uses the storage name when it is in the record and the declared name otherwise. Qdrant does not.

Expected

Upsert, get and search behave the same for dict, dataclass and pydantic records, with or without storage_name. The payload uses the storage names, which is what filters and hybrid search use.

Possible fix

I have a branch ready: https://github.com/VANDRANKI/semantic-kernel/tree/fix/qdrant-storage-name-record-keys

• Before building the point, _serialize_dicts_to_store_models renames declared names to storage names in the record when the storage name is not already there. Then it pops the key and the vectors by storage name.
• _deserialize_store_models_to_dicts returns the key and an unnamed vector under their storage names. The data model step reads dict and dataclass records by storage name. I could not exercise this part on main, because the upsert already fails there. A test covers it with named_vectors=False.

It adds 6 tests to tests/unit/connectors/memory/test_qdrant_local.py (dict, dataclass and pydantic, each with named and unnamed vectors). They upsert, get, search with vectors and filter on a storage name, all on the local Qdrant. All 6 fail on main. The 11 tests in that file and the 22 in test_qdrant.py pass on the branch, and ruff check and format are clean.

I can't open a pull request from this account right now, so the compare link is main...VANDRANKI:semantic-kernel:fix/qdrant-storage-name-record-keys