MemoryRecord struct is not public visible. This blocks extending the storage system

I have a large number of text files already embedded, embedding vectors are stored in a single file (embedding.jsonz). I would like to add these files to the memory system. However It turns out I could not reuse SemanticTextMemory class with my own IMemoryStore implementation.
The reason is SemanticTextMemory expects the result from storage is MemoryRecord, which is an internal type. my storage system could not create this type inside my code.
suggestions:

Make MemoryRecord class public.
Re-design the memory system. Current memory system is heavly over-cooked. Too many nested generic interface. As storage system heavyly rely on data serialize/deserialize and there's 5 key elements total (collection,key,content,content-type,embedding). A simple and lightweight data structure would be much better.
Thanks

Clock Set Bird (@JohnMasen)

Clock Set Bird (@JohnMasen) thanks for raising the issue, I can see that navigating that part of the SDK can be challenging. I think it would be useful providing an example to cover this scenario. Abby Harrison (@awharrison-28) could we reproduce this scenario and verify that one can actually bring their own memory data, either embeddings, raw data or both?

suggested storage interface

public struct MemoryRecord
{
    public string? Collection;//null=default collection
    public string Key;
    public string Value;
    public string Value_ContentType; //default is text/plain (string), could be others, we can leave this for further use
    public readonlyspan<float>?  Embedding; //this coul be empty if retrieve from storage
}

Public Interface IMemoryStorage
{
    void InsertOrUpdate(MemoryRecord value);
    MemoryRecord Retrieve(string collection, string key);
    void Remove(string collection,string key);
    void Purge(string collection);
    void InsertOrUpdateBatch(iEnumertable<MemroyRecord> itemList); 
   IEnumertable<MemoryRecord> Query(readonlyspan<float> queryEmbedding, float relevance=0.75f, int top=1);
}

As every thing is stored as string, user may confused what if I want store a plan text and a content with referenced URL?
there're 2 solutions:

Store them in different collections, and use different deserialized to handle the data.
use Value_ContentType to identify. for example: "text/plan"=text content,"jsonObject/[mytype]"=a user type

The key is underline storage system should NOT take care of the exact content type (usage). Using nested generic interface as data causes storage system may not return the exact type like it was stored. Keep it simple, please.

Clock Set Bird (@JohnMasen)

Hi Clock Set Bird (@JohnMasen), thank you for your feedback. I have hit a similar challenges and have been working on the story for passing vector metadata between storage schemas.

Memory - Qdrant Vector Database Connector #210 : reference for how we currently expect IMemoryStore to be implemented. Note that the PR contains a change to IEmbeddingWithMetadata expecting a string representation of the metadata (dotnet/src/SemanticKernel/Memory).
your suggestion of public string Value_ContentType is also high priority. I agree that it is very important for users to provide custom metadata to their storage not necessarily enforced by the kernel. This will make your suggestion of IMemoryStore : IDataStore<IEmbeddingWithMetadata> -> IMemoryStore : IDataStore<MemoryRecord> much easier.

We also understand that the nested generics are noisy. It's certainly something we want to address in a cleaner fashion where possible.

TLDR; Lots of improvements to Memory are in work. I'll keep this thread updated with relevant PRs.

Glad to see such a quick response. Expecting the new version :)

Change visibility modifiers on MemoryRecord and associated collection…

Clock Set Bird (@JohnMasen)

Clock Set Bird (@JohnMasen) A number of the points you called out are addressed in #308. Note that MemoryRecord is already public as of #250. Adding a value string to MemoryRecord is coming soon among some other QoL changes to SemanticTextMemory.

Memory Refactor: Simplify Memory Interfaces, Remove Unnecessary Gener…

That's great! very appreacted for your guys hard work!

I found the latest nuget package (0.10.72.1-preview) is not using the lastest change. Will these changes be included in the next version?

Yes, these changes will be included in the next version. I believe we will be rolling it out today.

Clock Set Bird (@JohnMasen)

Clock Set Bird (@JohnMasen) PR is out that adds an optional valueString field to MemoryRecord. IMemoryStore get/search methods now optionally return the embedding to the user too (defaults to false).

Once those changes propagate to the latest nuget package, can we consider this issue closed?

Abby Harrison (@awharrison-28)

Abby Harrison (@awharrison-28) By upgrading to the lastest package, I confirm the fix is done, please feel free to close the issue.
Great job, guys!

Add customizable metadata field to MemoryRecord and memory retrieval …

Clock Set Bird (@JohnMasen)

Clock Set Bird (@JohnMasen) following up here, the next package release adds a field to MemoryRecordMetadata for custom metadata strings and Get/Search memories calls can optionally return the actual embedding (defaulting to not).

Change visibility modifiers on MemoryRecord and associated collection…

Memory Refactor: Simplify Memory Interfaces, Remove Unnecessary Gener…

Add customizable metadata field to MemoryRecord and memory retrieval …

Navigation Menu

Repository navigation

MemoryRecord struct is not public visible. This blocks extending the storage system #202

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

MemoryRecord struct is not public visible. This blocks extending the storage system #202

Description

Activity

dluc commented on Mar 29, 2023

JohnMasen commented on Mar 29, 2023

JohnMasen commented on Mar 29, 2023

awharrison-28 commented on Mar 29, 2023

JohnMasen commented on Mar 30, 2023

awharrison-28 commented on Apr 4, 2023

JohnMasen commented on Apr 6, 2023

JohnMasen commented on Apr 11, 2023

awharrison-28 commented on Apr 11, 2023

awharrison-28 commented on Apr 12, 2023

JohnMasen commented on Apr 13, 2023

awharrison-28 commented on Apr 13, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions