Skip to content

Commit a58e68f

Browse files
authored
feat(router): add async aroute and aroute_many to SemanticRouter (redis#784)
## Motivation Closes redis#783. `SemanticRouter` had no async query path, so routing from async apps (LangGraph nodes, FastAPI handlers, MCP servers) blocked the event loop twice per call: on the embedding and on the `FT.AGGREGATE` round trip. `SemanticCache` already has async lookups; the router had none. ## Changes ### `aroute()` and `aroute_many()` Async counterparts of `router(...)` and `route_many(...)`. They embed with `vectorizer.aembed` and run the same aggregate request on an `AsyncSearchIndex` over the same index, so they return exactly what the sync methods return. Everything after `vector` is keyword-only, so a positional `distance_threshold` (deprecated in `route_many`) can't be read as an aggregation method. - `_get_route_matches` is split into request building (`_route_matches_request`) and row processing (`_route_matches_from_result`), shared by the sync and async paths so the two cannot drift. `_build_aggregate_request` is untouched, so this does not collide with redis#766. - The async index is created on first use (same pattern as `SemanticCache`), derived from the sync `SearchIndex`: when the sync index owns its client and has a URL, the async index connects from that URL, connection kwargs and `lib_name`; otherwise the caller's client is converted with `RedisConnectionFactory.sync_to_async_redis`. `from_existing(redis_url=...)` builds an owned client from the URL, so it takes the URL branch too. - `adisconnect()` closes the async connection; `aindex` exposes the async index (None before first use). - Mutating methods (`add_route`, `remove_route`, ...) stay sync to keep the change small. ## Release Notes `SemanticRouter` gains `aroute()`, `aroute_many()`, `adisconnect()` and an `aindex` property. The async index is created on first use. Routers given an SSL, unix-socket, Sentinel or `RedisCluster` client can't convert it to async; pass `redis_url` to the router for async routing in those setups. Vectorizers without a native `aembed` (e.g. the default `HFTextVectorizer`) still embed synchronously; the Redis call is non-blocking either way. The async client is bound to the event loop that made the first call; call `adisconnect()` before switching loops. ## Reviewer Brief **Start here:** `redisvl/extensions/router/semantic.py`, `_get_async_index`. It decides how the async index reaches Redis, and every `aroute` / `aroute_many` call goes through it. **Where it could fail** - `redisvl/extensions/router/semantic.py:557`: the URL branch is taken only when the sync index owns its client *and* has a URL. Check that a router built with `redis_client=` plus `redis_url=` (both given) converts the client rather than connecting from the URL, since the client is what the sync path uses. - `redisvl/extensions/router/semantic.py:567`: the client-conversion branch raises on the first async call for SSL, unix-socket, Sentinel and Cluster clients (`sync_to_async_redis` limitation, pre-existing). The docstrings say to pass `redis_url` instead; nothing prevents the mismatch at construction. **Not verified:** cluster mode (async path unsupported for `RedisCluster` clients); non-blocking embedding with a native-async vectorizer (the tests use `HFTextVectorizer`, which embeds synchronously, so only the Redis call is exercised async); Redis < 7 error mapping (shared code, unreachable on the test container). **Beyond CI:** `test_async_routing_matches_sync` (`tests/integration/test_semantic_router.py:822`) fails with `AssertionError: the async path made a blocking call` when `aroute` is changed to call the sync `embed`, and passes on this branch; `test_aroute_from_existing_url_connects_from_url` passes with the async index derived from the sync one, so the earlier `from_existing` patch is no longer needed. **Since the last brief:** `8a70416` applies the review: the async index is derived from `self._index` (no router-side copies of URL/kwargs/`lib_name`, no `from_existing` change), keyword-only arguments after `vector`, the three docstring notes, and the six async tests replaced by one that blocks the sync paths. <sub>Reviewed `8a70416` · Claude Code (claude-fable-5-1) · lenses: correctness, tests. Ran: `pytest tests/integration/test_semantic_router.py` (42 passed, 1 skipped; Redis via the repo's testcontainers compose), `pytest tests/unit -k "router or create_index"` (50 passed), `make check-types` (120 files, no issues), `make format`.</sub> 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > New async API reuses the existing FT.AGGREGATE flow and SemanticCache-style connection setup; main caveats are documented (cluster client conversion, event-loop binding), not new security or data paths. > > **Overview** > Adds **non-blocking routing** for async apps via `aroute()` and `aroute_many()`, mirroring `router()` / `route_many()` by using `vectorizer.aembed` and a lazily created `AsyncSearchIndex` on the same Redis index. > > The sync lookup path is refactored so request building (`_route_matches_request`), Redis error handling (`_raise_aggregate_error`), and result parsing (`_route_matches_from_result`) are shared with `_aget_route_matches`, reducing drift between sync and async. **`adisconnect()`** and an **`aindex`** property manage the optional async client (URL-based when the router owns its connection, otherwise `sync_to_async_redis`). > > Integration tests assert async results match sync, that the async path does not call blocking `embed`/`aggregate`, and that routers loaded with **`from_existing(redis_url=...)`** wire async Redis from the URL. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8a70416. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent 1da1234 commit a58e68f

2 files changed

Lines changed: 253 additions & 30 deletions

File tree

‎redisvl/extensions/router/semantic.py‎

Lines changed: 201 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
from pathlib import Path
2-
from typing import Any
2+
from typing import Any, NoReturn
33

44
import redis.commands.search.reducers as reducers
55
import yaml
@@ -20,7 +20,7 @@
2020
RoutingConfig,
2121
SemanticRouterIndexSchema,
2222
)
23-
from redisvl.index import SearchIndex
23+
from redisvl.index import AsyncSearchIndex, SearchIndex
2424
from redisvl.query import FilterQuery, VectorRangeQuery
2525
from redisvl.query.filter import Tag
2626
from redisvl.redis.connection import RedisConnectionFactory, _split_from_existing_kwargs
@@ -52,6 +52,7 @@ class SemanticRouter(BaseModel):
5252
"""Configuration for routing behavior."""
5353

5454
_index: SearchIndex = PrivateAttr()
55+
_aindex: AsyncSearchIndex | None = PrivateAttr(default=None)
5556
# A private attribute, never a field: it describes this instance's
5657
# relationship to the index, and must not reach the stored route config.
5758
_create_index: bool = PrivateAttr(default=True)
@@ -459,6 +460,24 @@ def _get_route_matches(
459460
max_k: int = 1,
460461
) -> list[RouteMatch]:
461462
"""Get route response from vector db"""
463+
aggregate_request, params = self._route_matches_request(
464+
vector, aggregation_method, max_k
465+
)
466+
try:
467+
aggregation_result: AggregateResult = self._index.aggregate(
468+
aggregate_request, params
469+
)
470+
except ResponseError as e:
471+
self._raise_aggregate_error(e)
472+
return self._route_matches_from_result(aggregation_result)
473+
474+
def _route_matches_request(
475+
self,
476+
vector: list[float],
477+
aggregation_method: DistanceAggregationMethod,
478+
max_k: int,
479+
) -> tuple[AggregateRequest, dict[str, Any]]:
480+
"""Build the aggregate request and query params for a routing lookup."""
462481

463482
if not self.routes:
464483
# Otherwise max() below raises "max() arg is an empty sequence",
@@ -483,18 +502,19 @@ def _get_route_matches(
483502
aggregate_request = self._build_aggregate_request(
484503
vector_range_query, aggregation_method, max_k=max_k
485504
)
505+
return aggregate_request, vector_range_query.params
486506

487-
try:
488-
aggregation_result: AggregateResult = self._index.aggregate(
489-
aggregate_request, vector_range_query.params
507+
@staticmethod
508+
def _raise_aggregate_error(e: ResponseError) -> NoReturn:
509+
if "VSS is not yet supported on FT.AGGREGATE" in str(e):
510+
raise RuntimeError(
511+
"Semantic routing is only available on Redis version 7.x.x or greater"
490512
)
491-
except ResponseError as e:
492-
if "VSS is not yet supported on FT.AGGREGATE" in str(e):
493-
raise RuntimeError(
494-
"Semantic routing is only available on Redis version 7.x.x or greater"
495-
)
496-
raise e
513+
raise e
497514

515+
def _route_matches_from_result(
516+
self, aggregation_result: AggregateResult
517+
) -> list[RouteMatch]:
498518
# process aggregation results into route matches, dropping any row whose
499519
# field payload came back missing (the Redis 8.8+ expiry race)
500520
processed = [
@@ -509,16 +529,66 @@ def _get_route_matches(
509529
)
510530
return matches
511531

512-
def _classify_route(
532+
async def _aget_route_matches(
513533
self,
514534
vector: list[float],
515535
aggregation_method: DistanceAggregationMethod,
516-
) -> RouteMatch:
517-
"""Classify to a single route using a vector."""
536+
max_k: int = 1,
537+
) -> list[RouteMatch]:
538+
"""Get route response from vector db without blocking the event loop."""
539+
aggregate_request, params = self._route_matches_request(
540+
vector, aggregation_method, max_k
541+
)
542+
aindex = self._get_async_index()
543+
try:
544+
aggregation_result: AggregateResult = await aindex.aggregate(
545+
aggregate_request, params
546+
)
547+
except ResponseError as e:
548+
self._raise_aggregate_error(e)
549+
return self._route_matches_from_result(aggregation_result)
550+
551+
def _get_async_index(self) -> AsyncSearchIndex:
552+
"""Lazily construct the async search index over the same Redis index."""
553+
if self._aindex is None:
554+
index = self._index
555+
if index._owns_redis_client and index._redis_url:
556+
self._aindex = AsyncSearchIndex(
557+
schema=index.schema,
558+
redis_url=index._redis_url,
559+
connection_kwargs=index._connection_kwargs or None,
560+
lib_name=index._lib_name,
561+
)
562+
else:
563+
self._aindex = AsyncSearchIndex(
564+
schema=index.schema,
565+
redis_client=RedisConnectionFactory.sync_to_async_redis(
566+
index._redis_client # type: ignore[arg-type]
567+
),
568+
owns_client=True,
569+
)
570+
return self._aindex
518571

519-
# take max route as distance threshold
520-
route_matches = self._get_route_matches(vector, aggregation_method)
572+
@property
573+
def aindex(self) -> AsyncSearchIndex | None:
574+
"""The AsyncSearchIndex used by the async routing methods, once created.
575+
576+
Returns:
577+
AsyncSearchIndex: The async search index, or None before the first
578+
async call.
579+
"""
580+
return self._aindex
521581

582+
async def adisconnect(self) -> None:
583+
"""Close the async Redis connection opened by the async routing methods."""
584+
if self._aindex is not None:
585+
await self._aindex.disconnect()
586+
self._aindex = None
587+
588+
@staticmethod
589+
def _top_route_match(
590+
route_matches: list[RouteMatch], router_name: str
591+
) -> RouteMatch:
522592
if not route_matches:
523593
return RouteMatch()
524594

@@ -529,9 +599,34 @@ def _classify_route(
529599
return top_route_match
530600
else:
531601
raise ValueError(
532-
f"{top_route_match.name} not a supported route for the {self.name} semantic router."
602+
f"{top_route_match.name} not a supported route for the {router_name} semantic router."
533603
)
534604

605+
@staticmethod
606+
def _top_route_matches(
607+
route_matches: list[RouteMatch], router_name: str
608+
) -> list[RouteMatch]:
609+
top_route_matches: list[RouteMatch] = []
610+
for route_match in route_matches:
611+
if route_match.name is not None:
612+
top_route_matches.append(route_match)
613+
else:
614+
raise ValueError(
615+
f"{route_match.name} not a supported route for the {router_name} semantic router."
616+
)
617+
return top_route_matches
618+
619+
def _classify_route(
620+
self,
621+
vector: list[float],
622+
aggregation_method: DistanceAggregationMethod,
623+
) -> RouteMatch:
624+
"""Classify to a single route using a vector."""
625+
626+
# take max route as distance threshold
627+
route_matches = self._get_route_matches(vector, aggregation_method)
628+
return self._top_route_match(route_matches, self.name)
629+
535630
def _classify_multi_route(
536631
self,
537632
vector: list[float],
@@ -541,19 +636,7 @@ def _classify_multi_route(
541636
"""Classify to multiple routes, up to max_k (int), using a vector."""
542637

543638
route_matches = self._get_route_matches(vector, aggregation_method, max_k=max_k)
544-
545-
# process route matches
546-
top_route_matches: list[RouteMatch] = []
547-
if route_matches:
548-
for route_match in route_matches:
549-
if route_match.name is not None:
550-
top_route_matches.append(route_match)
551-
else:
552-
raise ValueError(
553-
f"{route_match.name} not a supported route for the {self.name} semantic router."
554-
)
555-
556-
return top_route_matches
639+
return self._top_route_matches(route_matches, self.name)
557640

558641
@deprecated_argument("distance_threshold")
559642
def __call__(
@@ -625,6 +708,94 @@ def route_many(
625708

626709
return top_route_matches
627710

711+
async def aroute(
712+
self,
713+
statement: str | None = None,
714+
vector: list[float] | None = None,
715+
*,
716+
aggregation_method: DistanceAggregationMethod | None = None,
717+
) -> RouteMatch:
718+
"""Async version of calling the router: match a statement or vector to
719+
its best route without blocking the event loop.
720+
721+
Embedding uses the vectorizer's ``aembed`` and the lookup runs on an
722+
:class:`~redisvl.index.AsyncSearchIndex` over the same Redis index, so
723+
it returns the same match as ``router(statement)``.
724+
725+
Pass ``redis_url`` rather than a client to the router if you need
726+
async routing over SSL, a unix socket, Sentinel or Redis Cluster: those
727+
clients cannot be converted to their async form. With a vectorizer that
728+
has no native ``aembed``, such as the default ``HFTextVectorizer``,
729+
embedding still blocks the loop; only the Redis call is non-blocking.
730+
The async client is bound to the event loop that made the first call,
731+
so call :meth:`adisconnect` before switching loops.
732+
733+
Args:
734+
statement (Optional[str]): The input statement to be queried.
735+
vector (Optional[List[float]]): The input vector to be queried.
736+
aggregation_method (Optional[DistanceAggregationMethod]): The aggregation method used for vector distances.
737+
738+
Returns:
739+
RouteMatch: The matching route.
740+
741+
.. code-block:: python
742+
743+
match = await router.aroute("hi there")
744+
"""
745+
if not vector:
746+
if not statement:
747+
raise ValueError("Must provide a vector or statement to the router")
748+
vector = await self.vectorizer.aembed(statement) # type: ignore
749+
750+
aggregation_method = (
751+
aggregation_method or self.routing_config.aggregation_method
752+
)
753+
754+
route_matches = await self._aget_route_matches(
755+
vector, aggregation_method # type: ignore
756+
)
757+
return self._top_route_match(route_matches, self.name)
758+
759+
async def aroute_many(
760+
self,
761+
statement: str | None = None,
762+
vector: list[float] | None = None,
763+
*,
764+
max_k: int | None = None,
765+
aggregation_method: DistanceAggregationMethod | None = None,
766+
) -> list[RouteMatch]:
767+
"""Async version of :meth:`route_many`: return up to ``max_k`` matching
768+
routes without blocking the event loop. See :meth:`aroute` for the
769+
connection and vectorizer caveats.
770+
771+
Args:
772+
statement (Optional[str]): The input statement to be queried.
773+
vector (Optional[List[float]]): The input vector to be queried.
774+
max_k (Optional[int]): The maximum number of top matches to return.
775+
aggregation_method (Optional[DistanceAggregationMethod]): The aggregation method used for vector distances.
776+
777+
Returns:
778+
List[RouteMatch]: The matching routes and their details.
779+
780+
.. code-block:: python
781+
782+
matches = await router.aroute_many("hi there", max_k=2)
783+
"""
784+
if not vector:
785+
if not statement:
786+
raise ValueError("Must provide a vector or statement to the router")
787+
vector = await self.vectorizer.aembed(statement) # type: ignore
788+
789+
max_k = max_k or self.routing_config.max_k
790+
aggregation_method = (
791+
aggregation_method or self.routing_config.aggregation_method
792+
)
793+
794+
route_matches = await self._aget_route_matches(
795+
vector, aggregation_method, max_k=max_k # type: ignore
796+
)
797+
return self._top_route_matches(route_matches, self.name)
798+
628799
def add_route(self, route: Route) -> str:
629800
"""Add a new route to the SemanticRouter.
630801

‎tests/integration/test_semantic_router.py‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -850,3 +850,55 @@ def test_delete_route_references(semantic_router):
850850
# the way back in, so from_existing could not reload the config.
851851
router_dict = semantic_router.to_dict()
852852
assert [r["name"] for r in router_dict["routes"]] == ["farewell"]
853+
854+
855+
async def test_async_routing_matches_sync(semantic_router, monkeypatch):
856+
skip_if_redis_version_below(semantic_router._index.client, "7.0.0")
857+
# Loose enough that both routes match, so the max_k default is observable.
858+
semantic_router.update_route_thresholds({"greeting": 2.0, "farewell": 2.0})
859+
vector = semantic_router.vectorizer.embed("goodbye")
860+
inputs = [{"statement": s} for s in ("hello", "unknown_phrase")]
861+
inputs.append({"vector": vector})
862+
expected = [(semantic_router(**i), semantic_router.route_many(**i)) for i in inputs]
863+
864+
def blocking(*args, **kwargs):
865+
raise AssertionError("the async path made a blocking call")
866+
867+
# The async path must embed with aembed and query the async index.
868+
monkeypatch.setattr(type(semantic_router.vectorizer), "embed", blocking)
869+
monkeypatch.setattr(semantic_router._index, "aggregate", blocking)
870+
try:
871+
actual = [
872+
(await semantic_router.aroute(**i), await semantic_router.aroute_many(**i))
873+
for i in inputs
874+
]
875+
with pytest.raises(ValueError, match="Must provide a vector or statement"):
876+
await semantic_router.aroute()
877+
finally:
878+
await semantic_router.adisconnect()
879+
assert actual == expected
880+
assert semantic_router.aindex is None
881+
882+
883+
async def test_aroute_from_existing_url_connects_from_url(
884+
redis_url, routes, hf_vectorizer, redis_test_name
885+
):
886+
name = redis_test_name("test_router_async_from_existing")
887+
router = SemanticRouter(
888+
name=name,
889+
routes=routes,
890+
redis_url=redis_url,
891+
overwrite=True,
892+
vectorizer=hf_vectorizer,
893+
)
894+
loaded = SemanticRouter.from_existing(name, redis_url=redis_url)
895+
try:
896+
skip_if_no_redis_search(router._index.client)
897+
skip_if_redis_version_below(router._index.client, "7.0.0")
898+
assert (await loaded.aroute("hello")).name == "greeting"
899+
# Built from the caller's URL, not converted from the client that
900+
# from_existing created, so URL-only setups (unix, sentinel) work too.
901+
assert loaded.aindex._redis_url == redis_url
902+
finally:
903+
await loaded.adisconnect()
904+
router.delete()

0 commit comments

Comments
 (0)