You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
SemanticCache vector index algorithm is hardcoded to FLAT; no way to configure HNSW #602
There is no constructor argument on SemanticCache to override this. FLAT is exact KNN — fine to the low millions of entries, latency-degrading beyond that. For large multi-tenant deployments or per-tenant player-facing caches (10M–100M entries), HNSW is required. Today the only workaround is to bypass SemanticCache entirely and build the index manually.
A simpler form taking just algorithm: Literal["flat", "hnsw"] = "flat" plus a default HNSW parameter preset would also be acceptable for a first cut.
Compatibility notes
The algorithm is fixed at index creation; existing FLAT caches cannot be hot-swapped to HNSW. Customers wanting to switch must build a new cache (new name), re-warm, and cut over — same migration shape as a schema change. This is acceptable as long as the option exists at construction time.
Notes
Surfaced while writing a scoped semantic caching architecture spec for customers running large-scale, multi-tenant deployments.
Activity