Repository navigation
[MEVD] Implement telemetry in vector store connectors #10596
Description
Consider implementing telemetry in vector store connectors. At the moment, it should be possible to use the telemetry from underlying SDKs, but it's not clear if all the SDKs have the telemetry implemented. Another potential issue is that the telemetry from underlying SDKs can be different and not consistent. Having the same telemetry for all the connectors will ensure consistency in logs, metrics and traces for all supported vector stores. For connectors implemented outside of this repository, it will be possible to create a documentation with telemetry guidelines or publish a NuGet package with common telemetry methods so connector developers can re-use it.
- Code changes
- Implement Logging decorators
- Implement OpenTelemetry decorators
- Implement Delegating{Service} base classes.
- Investigate potential interface casting issues
-
Concepts/GettingStartedexamples- Add telemetry examples for
HybridSearchandVectorizableTextSearch
- Add telemetry examples for
- Learn documentation
- Updates to release pipeline
- Blog post
Reacted by Shay Rojansky
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
roji commentedon Feb 19, 2025
Thanks for opening this Dmytro Struk (@dmytrostruk), it's a really important aspect of what we're doing and I've had it in my notes for a while.
First, while (experimental) OpenTelemetry semantic conventions do exist for generative AI (link), these do not cover vector database access (though they may at some point). There's the general semantic conventions for database access (link), which may be applicable to vector databases as well. In other words, there doesn't seem to be a standardized way to do telemetry for vector database access yet - I'd recommend seeing what other frameworks (e.g. langchain) are doing in this area, and possibly align to them; otherwise we may end up defining our own, temporary schema here. Note that all this applies to both tracing (spans) and metrics (aggregated data).
Then there's the question of where the support actually lives... IMHO in the ideal world, a standard semantic convention exists, and the low-level vector DB SDKs implement it (rather than MEVD). Telemetry is a universal concern for anyone accessing the database, and should IMHO generally be implemented at the lowest point possible so that everyone benefits (in other words, I wouldn't want someone to have to use MEVD in order to get telemetry).
However, being more pragmatic, pushing in uniform telemetry support to all the low-level SDKs - in a timely manner - is probably not going to very easy. We do have the advantage of being able to build support into the MEVD connectors first (we own most of these at the moment), and then trying to see if we can push it down into the SDKs (at which point the MEVD-level support can be removed, otherwise we get double telemetry). We can take a case-by-case approach - where we know the SDK maintainer and can push telemetry support into there, that's ideal, and where not, we fall back to implementing in the MEVD connector.
A possibly useful thing is to implement specification/conformance tests (similar to what I've been adding to other things) which check that an MEVD connector emits proper, standardized telemetry (there are OTel testing packages which can help make this easy (I can help there).
Another approach is to implement telemetry via a wrapper/layer on top of our own abstractions, i.e. a TracingVectorStoreRecordCollection that wraps another IVectorStoreRecordCollection and adds tracing calls. AFAIK this is the model they they're going with in MEAI (a middleware-like wrapping approach). I'm not sure how easy/applicable this is to our abstraction (and how a user actually wires this in, considering vector stores are factories for collections), but it's definitely worth looking at what they're doing. We'd have to again keep in mind the fact that an SDK might implement telemetry itself, and avoid double telemetry.
I'll note that this is an "additive" aspect that we can always introduce later, without breaking. So strictly speaking, it's not absolutely vital that we finish this in time for the first release - we can do a second one with telemetry support at any point afterwards, allowing us to concentrate on stuff that must absolutely get done and not rushing this in half-baked. Having said that, this is indeed important, and if we have spare capacity, it may be a good idea to work on it.
/cc Stephen Toub (@stephentoub)
[-].Net: Investigate telemetry in vector store connectors[/-][+].Net: Implement telemetry in vector store connectors[/+][-].Net: Implement telemetry in vector store connectors[/-][+][MEVD] Implement telemetry in vector store connectors[/+].Net: [Feature Branch] Vector Store Logging (#10865)
roji commentedon Apr 8, 2025
Proposal PR for vector data semantic conventions by Enrico Zimuel (@ezimuel): open-telemetry/semantic-conventions#1231
github-actions commentedon Jul 17, 2026
This issue is stale because it has been open for 90 days with no activity.
github-actions commentedon Oct 21, 2026
This issue was closed because it has been inactive for 14 days since being marked as stale.