Skip to content

.Net MEVD: Strongly-typed property selectors - #10850

Merged
Adam Sitnik (adamsitnik) merged 6 commits into
microsoft:feature-vector-data-preb1from
adamsitnik:selectorAgain
Mar 7, 2025
Merged

Adam Sitnik (adamsitnik) merged 6 commits into
microsoft:feature-vector-data-preb1from
adamsitnik:selectorAgain

Conversation

@adamsitnik

Copy link
Copy Markdown
Member

With the amount of merge conflict I needed to solve, it was easier to start from scratch, so I am closing #10793.

The property selector can return a property:

var results = await fixture.Collection.VectorizedSearchAsync(
    new ReadOnlyMemory<float>([1, 2, 3]),
    new() { VectorProperty = r => r.SomeVectorProp });

But this time VectorStoreGenericDataModel is also supported for consts, variables and arguments:

var results = await fixture.Collection.VectorizedSearchAsync(
    new ReadOnlyMemory<float>([1, 2, 3]),
    new() { VectorProperty = r => r.Vectors[argument]});

fixes #10420

var actual = await sut.VectorizedSearchAsync(
new ReadOnlyMemory<float>([1, 1, 1, 1]),
new() { IncludeVectors = true, VectorPropertyName = "Vector" },
new() { IncludeVectors = true, VectorProperty = r => r.Vectors["Vector"] },

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an example of how it can be used with VectorStoreGenericDataModel<T>


var searchOptions = options ?? s_defaultKeywordVectorizedHybridSearchOptions;
var vectorProperty = this._propertyReader.GetVectorPropertyOrSingle(searchOptions.VectorPropertyName);
var vectorProperty = this._propertyReader.GetVectorPropertyOrSingle<TRecord>(new() { VectorProperty = searchOptions.VectorProperty });

@adamsitnik Adam Sitnik (adamsitnik) Mar 7, 2025 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we remove the [Obsolete] VectorPropertyName property, the GetVectorPropertyOrSingle is going to accept just Expression<Func<TRecord, object?>>? and there will be no need to instantiate the search option class here.

(I did not want to send a string and expression, as I would need to touch the obsolete property in every connector, add pragmas to disable the warning and overall it simply would require more changes now and in the future)


private static bool TryGetCapturedValue(Expression expression, out object? capturedValue)
{
if (expression is MemberExpression { Expression: ConstantExpression constant, Member: FieldInfo fieldInfo }

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I stole this idea from Shay Rojansky (@roji) and his filter translator

- replace AdditionalPropertyName with AdditionalProperty selector
@adamsitnik Adam Sitnik (adamsitnik) changed the title .Net MEVD: Strongly-typed vector property selector .Net MEVD: Strongly-typed property selectors Mar 7, 2025
return this.VectorProperties.FirstOrDefault(l => l.DataModelPropertyName.Equals(key, StringComparison.Ordinal))
?? throw new InvalidOperationException($"The {this._dataModelType.FullName} type does not have a vector property named '{key}'.");
}
// VectorPropertiesInfo is not available for VectorStoreGenericDataModel.

@westey-m westey (westey-m) Mar 7, 2025 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this is why we call VectorStoreRecordPropertyVerification.VerifyGenericDataModelDefinitionSupplied in the collection constructor. If the user uses the generic data model, they have to supply a definition, in which case VectorPropertiesInfo will be loaded in LoadPropertyInfoIfNeeded. That way, VectorPropertiesInfo is always either available from the type or from the definition.

Come to think of it though. After this release, it might make sense to do a small refactoring where we move the call to VerifyGenericDataModelDefinitionSupplied to the PropertyReader constructor. It should always be called after all, so doesn't make sense to allow a specific implementation to forget to invoke it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW the infos are not needed for the VectorStoreGenericDataModel (as the expression will never point to a single property, but rather the Data or Vectors dictionaries). I can make it nullable if you want.

@adamsitnik
Adam Sitnik (adamsitnik) changed the base branch from feature-vector-data-preb1 to main March 7, 2025 18:14
@adamsitnik
Adam Sitnik (adamsitnik) dismissed westey (westey-m)’s stale review March 7, 2025 18:14

The base branch was changed.

@adamsitnik
Adam Sitnik (adamsitnik) changed the base branch from main to feature-vector-data-preb1 March 7, 2025 18:14

This branch was previously deployed

1 inactive deployment
integration — a7b61f64 Deployed Mar 7, 2025 by adamsitnik
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants