Skip to content

feat(router): add async aroute and aroute_many to SemanticRouter - #784

Merged
vishal-bala merged 2 commits into
redis:mainfrom
Yarmoluk:feat/router-async-query
Oct 30, 2026
Merged

vishal-bala merged 2 commits into
redis:mainfrom
Yarmoluk:feat/router-async-query

Conversation

@Yarmoluk

@Yarmoluk Yarmoluk commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Closes #783.

SemanticRouter had no async query path, so routing from async apps (LangGraph nodes, FastAPI handlers, MCP servers) blocked the event loop twice per call: on the embedding and on the FT.AGGREGATE round trip. SemanticCache already has async lookups; the router had none.

Changes

aroute() and aroute_many()

Async counterparts of router(...) and route_many(...). They embed with vectorizer.aembed and run the same aggregate request on an AsyncSearchIndex over the same index, so they return exactly what the sync methods return. Everything after vector is keyword-only, so a positional distance_threshold (deprecated in route_many) can't be read as an aggregation method.

  • _get_route_matches is split into request building (_route_matches_request) and row processing (_route_matches_from_result), shared by the sync and async paths so the two cannot drift. _build_aggregate_request is untouched, so this does not collide with fix(router): apply per-route thresholds before max_k truncation #766.
  • The async index is created on first use (same pattern as SemanticCache), derived from the sync SearchIndex: when the sync index owns its client and has a URL, the async index connects from that URL, connection kwargs and lib_name; otherwise the caller's client is converted with RedisConnectionFactory.sync_to_async_redis. from_existing(redis_url=...) builds an owned client from the URL, so it takes the URL branch too.
  • adisconnect() closes the async connection; aindex exposes the async index (None before first use).
  • Mutating methods (add_route, remove_route, ...) stay sync to keep the change small.

Release Notes

SemanticRouter gains aroute(), aroute_many(), adisconnect() and an aindex property. The async index is created on first use. Routers given an SSL, unix-socket, Sentinel or RedisCluster client can't convert it to async; pass redis_url to the router for async routing in those setups. Vectorizers without a native aembed (e.g. the default HFTextVectorizer) still embed synchronously; the Redis call is non-blocking either way. The async client is bound to the event loop that made the first call; call adisconnect() before switching loops.

Reviewer Brief

Start here: redisvl/extensions/router/semantic.py, _get_async_index. It decides how the async index reaches Redis, and every aroute / aroute_many call goes through it.

Where it could fail

  • redisvl/extensions/router/semantic.py:557: the URL branch is taken only when the sync index owns its client and has a URL. Check that a router built with redis_client= plus redis_url= (both given) converts the client rather than connecting from the URL, since the client is what the sync path uses.
  • redisvl/extensions/router/semantic.py:567: the client-conversion branch raises on the first async call for SSL, unix-socket, Sentinel and Cluster clients (sync_to_async_redis limitation, pre-existing). The docstrings say to pass redis_url instead; nothing prevents the mismatch at construction.

Not verified: cluster mode (async path unsupported for RedisCluster clients); non-blocking embedding with a native-async vectorizer (the tests use HFTextVectorizer, which embeds synchronously, so only the Redis call is exercised async); Redis < 7 error mapping (shared code, unreachable on the test container).

Beyond CI: test_async_routing_matches_sync (tests/integration/test_semantic_router.py:822) fails with AssertionError: the async path made a blocking call when aroute is changed to call the sync embed, and passes on this branch; test_aroute_from_existing_url_connects_from_url passes with the async index derived from the sync one, so the earlier from_existing patch is no longer needed.

Since the last brief: 8a70416 applies the review: the async index is derived from self._index (no router-side copies of URL/kwargs/lib_name, no from_existing change), keyword-only arguments after vector, the three docstring notes, and the six async tests replaced by one that blocks the sync paths.

Reviewed 8a70416 · Claude Code (claude-fable-5-1) · lenses: correctness, tests. Ran: pytest tests/integration/test_semantic_router.py (42 passed, 1 skipped; Redis via the repo's testcontainers compose), pytest tests/unit -k "router or create_index" (50 passed), make check-types (120 files, no issues), make format.

🤖 Generated with Claude Code


Note

Low Risk
New async API reuses the existing FT.AGGREGATE flow and SemanticCache-style connection setup; main caveats are documented (cluster client conversion, event-loop binding), not new security or data paths.

Overview
Adds non-blocking routing for async apps via aroute() and aroute_many(), mirroring router() / route_many() by using vectorizer.aembed and a lazily created AsyncSearchIndex on the same Redis index.

The sync lookup path is refactored so request building (_route_matches_request), Redis error handling (_raise_aggregate_error), and result parsing (_route_matches_from_result) are shared with _aget_route_matches, reducing drift between sync and async. adisconnect() and an aindex property manage the optional async client (URL-based when the router owns its connection, otherwise sync_to_async_redis).

Integration tests assert async results match sync, that the async path does not call blocking embed/aggregate, and that routers loaded with from_existing(redis_url=...) wire async Redis from the URL.

Reviewed by Cursor Bugbot for commit 8a70416. Bugbot is set up for automated code reviews on this repo. Configure here.

@vishal-bala vishal-bala self-assigned this Oct 27, 2026
@vishal-bala
vishal-bala self-requested a review October 27, 2026 14:36
@vishal-bala vishal-bala added the auto:minor Increment the minor version when merged label Oct 27, 2026

@vishal-bala vishal-bala left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! That's a useful addition, glad to have it 😄
Just a few minor comments and we should be good to go

Comment thread redisvl/extensions/router/semantic.py Resolved
Comment thread redisvl/extensions/router/semantic.py Resolved
Comment thread redisvl/extensions/router/semantic.py Resolved
Routing from async code (LangGraph nodes, FastAPI handlers, MCP servers)
blocked the event loop twice per call: once to embed, once on the
FT.AGGREGATE round trip. SemanticCache already has async lookups; the
router had none.

aroute() and aroute_many() embed with the vectorizer's aembed and run the
same aggregate request on a lazily created AsyncSearchIndex over the same
index, so they return what router() and route_many() return. Request
building and row processing are shared with the sync path. adisconnect()
closes the async connection. Mutating methods stay sync.
Address review on redis#784:
- _get_async_index reads the URL, connection kwargs, lib_name and client
  ownership from self._index instead of keeping copies on the router,
  which also makes the from_existing patch unnecessary.
- aroute/aroute_many take everything after vector keyword-only, so a
  positional distance_threshold can't be read as an aggregation method.
- Docstrings note that SSL, unix-socket, Sentinel and Cluster clients
  should pass redis_url for async routing, that vectorizers without a
  native aembed still block on embedding, and that adisconnect() must be
  called before switching event loops.
- Replace the six async tests with one that blocks the sync embed and
  sync aggregate paths, so it fails if aroute takes either.

@vishal-bala vishal-bala left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! We'll get this out with the next minor release 🎉

@vishal-bala
vishal-bala merged commit a58e68f into redis:main Oct 30, 2026
55 checks passed
@applied-ai-release-bot

Copy link
Copy Markdown

🚀 PR was released in v0.30.0 🚀

@applied-ai-release-bot applied-ai-release-bot Bot added the released This issue/pull request has been released. label Nov 2, 2026
Yarmoluk added a commit to Yarmoluk/redis-vl-python that referenced this pull request Nov 2, 2026
… to their own defaults

The sync pool injects maintenance-notification keywords (the pool handler,
maint_notifications_config and the orig_* copies of host and timeouts) for
its own connections. Older async classes don't accept them, so on
redis-py 7.4 with RESP3, and on 8.0 for every client, the conversion raised
naming settings the user never touched. Those keywords describe the sync
pool, not the user's settings, and the async pool computes its own, so they
are never carried over; maint_notifications_config is dropped unless the
user enabled notifications explicitly.

A remaining sync-only keyword now counts as unset when it still holds the
sync class's own declared default, found by inspect.signature, rather than
when it is None or False. That stops 0 being treated as unset and covers
keywords whose default is not falsy.

Also trims the SemanticRouter.aroute caveat to Sentinel and Cluster now
that redis#784 is in, notes that a custom SSLConnection subclass's behaviour is
not carried over, and folds the tests into two parametrised ones with
RESP3 and unix-URL cases.
@Yarmoluk
Yarmoluk deleted the feat/router-async-query branch November 2, 2026 19:50
vishal-bala pushed a commit that referenced this pull request Nov 3, 2026
Closes #789.

## Motivation

`RedisConnectionFactory.sync_to_async_redis` passed the sync pool's
`connection_kwargs` straight into `AsyncConnectionPool` and picked the
TLS class with an exact-type check. A client built with `Redis(ssl=True,
...)`, `Redis(unix_socket_path=...)` or `Sentinel.master_for(...)`
therefore raised `TypeError` on its first async command
(`SemanticCache.acheck`, and `SemanticRouter.aroute` once #784 lands),
and a subclass of `SSLConnection` got a plaintext async connection with
the password still set.

## Changes

- The async connection class is chosen with `issubclass`: TLS
connections (and subclasses) map to `AsyncSSLConnection`, unix-socket
connections to the async `UnixDomainSocketConnection`, everything else
to `AsyncConnection`.
- `_async_connection_kwargs` keeps only the keywords the chosen async
class accepts, found by walking its `__init__` chain
(`_accepted_init_kwargs`), so the filter follows whatever redis-py
version is installed rather than a hard-coded key list. A sync-only
keyword at its default (`None` or `False`) is dropped; one that is set,
such as `ssl_validate_ocsp=True`, raises a `ValueError` naming it,
because the async client cannot honour it and dropping it would silently
weaken TLS verification.
- Sentinel-managed pools are rejected up front with a clear
`ValueError`, as `RedisCluster` already was.
- Seven unit tests in `tests/unit/test_sync_to_async_redis.py`, none
needing a server; five fail on `main`.

## Release Notes

`SemanticCache`, `AsyncSearchIndex` and any other async path that
converts a caller's sync client now work with clients configured with
TLS keywords (`Redis(ssl=True, ...)`), unix sockets, and custom
`SSLConnection` subclasses. Sentinel-managed clients, `RedisCluster`
clients, and clients with a sync-only setting enabled (OCSP validation)
raise `ValueError` at conversion time with a pointer to `redis_url`;
previously they failed with `TypeError` on the first command.

## Reviewer Brief

**Start here:** `redisvl/redis/connection.py`,
`_async_connection_kwargs`. It decides which of the sync pool's settings
reach the async connection and which abort the conversion;
`sync_to_async_redis` only picks the class and calls it.

**Where it could fail**
- `redisvl/redis/connection.py`, `_SYNC_POOL_PLUMBING`: these four
keywords are dropped by name because the sync pool computes them from
the live settings; if a future redis-py adds another pool-computed
keyword, every conversion on a version whose async class rejects it
would raise again. Check against redis-py's
`ConnectionPool._update_connection_kwargs`.
- `redisvl/redis/connection.py`, `_is_unset`:
`maint_notifications_config` is dropped unless `enabled is True`, so a
user who set `enabled="auto"` on purpose gets an async client without
notifications on 7.4/8.0; on 8.1 it passes through.
- `redisvl/redis/connection.py`, `_init_parameters`: stops at the first
`__init__` without `**kwargs`; for the sync side that is the user's
`connection_class`, so a subclass with an explicit-keyword `__init__`
contributes only its own defaults.

**Not verified:** cluster mode (unchanged path, raises as before); a
live TLS or unix-socket server (the tests stop at `make_connection()`,
where every reported failure occurred); the sync `Retry` pass-through at
the end of the function is pre-existing and left for a follow-up.

**Beyond CI:** the ten tests pass on redis-py 7.4.1, 8.0.1 and 8.1.0;
the `resp3` case fails on 7.4.1 and 8.0.1 without the fix, and the
`tls-subclass` case fails on 8.0.1 when either the plumbing filter or
the `maint_notifications_config` rule is removed.

**Since the last brief:** 7c856ed rebases on 0.30.0, drops the
pool-computed keywords, compares sync-only keywords to their own
defaults, trims the `aroute` caveat, and folds the tests into two
parametrised ones.

<sub>Reviewed `7c856ed` · Claude Code (claude-fable-5-1) · lenses:
correctness, security, design, tests, docs</sub>

## Notes

- A set sync-only keyword raises rather than warns. OCSP validation is a
security setting; converting it into a client that cannot validate OCSP,
with only a log line (which #786 shows can be invisible), seemed the
wrong default.
- Sentinel conversion is rejected rather than rebuilt through
`AsyncSentinel`, as the issue suggests. The pool does carry
`sentinel_manager` and `service_name`, so a later change could rebuild
it; that is a separate feature.
- Once this merges, the SSL/unix-socket caveat in #784's
`SemanticRouter` docstring reduces to Sentinel and cluster; I'll trim it
there.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes how sync clients are translated into async pools across
index/cache/router async entry points, including TLS and socket
settings; incorrect filtering could misconfigure connections, though
explicit errors protect sync-only security options.
> 
> **Overview**
> Fixes **`RedisConnectionFactory.sync_to_async_redis`** so async paths
(`SemanticRouter.aroute`, `AsyncSearchIndex`, semantic caches) can reuse
a caller’s sync client when it uses **TLS**, **TLS subclasses**, or
**unix sockets**, instead of failing on the first command with
**`TypeError`** or building a plaintext async pool.
> 
> Conversion now picks the async connection class with **`issubclass`**
(TLS → `AsyncSSLConnection`, unix → `AsyncUnixDomainSocketConnection`),
filters pool kwargs through **`_async_connection_kwargs`**
(version-aware via **`_init_parameters`**, strips RESP3 pool plumbing,
drops sync-only defaults), and **raises `ValueError`** for
**Sentinel-managed** pools and for user-set sync-only TLS options (e.g.
**`ssl_validate_ocsp`**) rather than silently weakening security.
**`SemanticRouter.aroute`** docs now say only **Sentinel** and
**Cluster** need **`redis_url`** for async routing.
> 
> Adds **`tests/unit/test_sync_to_async_redis.py`** covering successful
mappings and failure modes without a live Redis server.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
7c856ed. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
vishal-bala pushed a commit that referenced this pull request Nov 6, 2026
Branch: `feat/message-history-async` (fork `Yarmoluk/redis-vl-python`),
rebased onto `main` at `506b961` (includes #792 and #785).
Commits: `82f91e4` feat, `93657d7` docs, `c1b5d75` test. Label:
`auto:minor`.

## Summary

Follow-up to #784, which gave `SemanticRouter` `aroute` / `aroute_many`.
`MessageHistory` had the same gap: an agent running on asyncio blocked
its event loop on every history read or write. This PR adds the async
twins -- `aget_recent`, `astore`, `aadd_messages`, `aadd_message`,
`acount`, `adrop`, `aclear`, `adisconnect` and an `aindex` property --
using the same pattern #784 landed:

- The `AsyncSearchIndex` is built lazily on the first async call, from
the sync index's connection settings. A history built from `redis_url`
gets its own async connection from that URL and `connection_kwargs`; a
history given a sync `redis_client` gets an async client converted from
that client's connection settings by
`RedisConnectionFactory.sync_to_async_redis`, owned by the async index,
so `adisconnect` closes only the converted client and the caller's sync
client is never touched or rebuilt.
- Every async method goes through the `AsyncSearchIndex`; none reaches
into a Redis client directly (`adrop` deletes with
`aindex.drop_keys(aindex.key(id))`).
- Query building (`_recent_query`, `_count_query`) and message
construction (`_chat_messages`, `_exchange`) move into helpers shared by
the sync and async paths, so the two cannot drift.
- The docstrings say what is and is not supported, in the router's
wording: since #792 a sync client over TLS or a unix socket converts;
Sentinel and Redis Cluster clients do not (pass `redis_url` instead);
the async client is bound to the loop that made the first call, so call
`adisconnect` before switching loops.

No sync behaviour changes. `SemanticMessageHistory` is not covered here
(see Notes).

## Reviewer Brief

**Start here:** `redisvl/extensions/message_history/message_history.py`,
`_get_async_index()` (line 418). Every async method goes through it, and
it decides whether the async index connects from the URL the sync index
owns or from a client converted from the caller's sync client; it is the
same code as `SemanticRouter._get_async_index`.

**Where it could fail**
- `message_history.py:432`: a caller-supplied Sentinel or Cluster
client, or one with a sync-only setting such as
`ssl_validate_ocsp=True`, raises `ValueError` from `sync_to_async_redis`
at the first async call; the `aget_recent` docstring names Sentinel and
Cluster but, like the router's, not the sync-only-setting case.
- `message_history.py:154`: `adrop` removes the key with `UNLINK` (via
`drop_keys`) where sync `drop` uses `DEL`; both are `@keyspace @write`,
but an ACL that grants `+del` by name and not `+unlink` would allow
`drop` and deny `adrop`
([UNLINK](https://redis.io/docs/latest/commands/unlink/),
[DEL](https://redis.io/docs/latest/commands/del/)).
- `message_history.py:449`: the async client belongs to the loop that
made the first call; a second `asyncio.run()` on the same history
without `adisconnect` in between fails, the same contract
`SemanticRouter` has shipped since #784.

**Not verified:** Redis Cluster and Sentinel (no cluster tests run
locally or in CI); the TLS and unix-socket conversion paths were not
exercised end to end through `MessageHistory` (they are covered by
#792's own tests); `connection_kwargs` and `lib_name` forwarding on the
URL path is not asserted by a test.

**Beyond CI:** each of these mutations makes
`test_async_message_history_matches_sync` fail: `astore` ignoring
`session_tag`, `adrop` going through the sync client, `adrop(id)`
ignoring `id`, the async index rebuilt per call, and `adisconnect` not
closing the converted client. The two new integration tests pass locally
against `redis:8.4` via testcontainers.

**Since the last brief:** rebased onto `506b961` (#792 now converts TLS
and unix-socket clients, so the caveat narrows to Sentinel and Cluster);
`adrop` deletes through `aindex.drop_keys` instead of the async client;
the sync-equivalence test now blocks the sync client's `execute_command`
and `pipeline`, seeds a second session with `astore`, and pins
`tool_call_id`/metadata, `adrop(id)`, async-index reuse and
`adisconnect` closing the converted pool.

<sub>Reviewed `c1b5d75` · Claude Code (claude-opus-5-5) · lenses:
correctness, security, design, tests, docs, Redis behaviour</sub>

## Test plan

Run on the rebased branch at `c1b5d75`, macOS, `uv`, Docker running
(testcontainers starts `redis:8.4`):

```bash
make format
# All done! ✨ 🍰 ✨
# 272 files left unchanged.

rm -rf .mypy_cache && make check-types
# Success: no issues found in 122 source files

uv run pytest tests/integration/test_message_history.py -k async_message_history -q -p no:cacheprovider
# 2 passed, 33 deselected

uv run pytest -n auto tests/integration/test_message_history.py tests/unit -q -p no:cacheprovider
# 3 failed, 1500 passed, 9 skipped, 53 errors
```

The 3 failures and 53 errors are the identical set on `main` at
`506b961`: every one is `ModuleNotFoundError: sentence_transformers`
(the HuggingFace vectorizer is not installed locally), in
`SemanticMessageHistory` tests this PR does not touch.

`test_async_message_history_matches_sync` takes its expected values from
the sync methods, then patches `execute_command` and `pipeline` on the
sync index's Redis client to raise, so any async method that made a
blocking call would fail it. `test_async_message_history_with_redis_url`
checks that a URL-built history builds its async index from that URL
rather than converting the client.

## Notes

- `SemanticMessageHistory` gets no async methods in this PR. Its
`get_relevant` embeds through the vectorizer, which brings in the
`aembed` caveats #784 documented for the router; it is a separate change
with its own brief.
- No `adelete`: dropping the index is setup/teardown rather than a
per-turn call, and #784 drew the same line for the router. No async form
of the `messages` property either, since a property cannot be awaited;
`aget_recent` with a large `top_k` is the async read.
- Loop binding follows #784's `adisconnect` contract rather than the
rebuild-on-new-loop #785 added for Cohere's async client. The async
index owns a connection pool, and rebuilding it silently on a loop
change would leak the previous pool; an explicit `adisconnect` keeps the
two extensions consistent and the close visible.
- `_get_async_index`, `aindex` and `adisconnect` match
`SemanticRouter`'s line for line; they are kept as two copies rather
than a shared helper, since extracting one would mean changing the
merged router code in this PR.
- `count`/`acount` keep the sync method's unannotated signature rather
than adding `-> int` to one twin only.

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> New async Redis connection lifecycle (lazy index, loop binding, client
conversion) could misbehave for Sentinel/Cluster/unix-socket setups;
sync API and data paths are unchanged.
> 
> **Overview**
> Adds **async counterparts** to `MessageHistory` read/write/clear/drop
operations (`aget_recent`, `astore`, `aadd_messages`, `aadd_message`,
`acount`, `adrop`, `aclear`, plus `adisconnect` and an `aindex`
property) so asyncio agents can use history without blocking the event
loop.
> 
> Async work goes through a **lazily created** `AsyncSearchIndex` that
reuses the sync index’s schema and connection settings: URL-owned
clients open a separate async connection from `redis_url`;
caller-supplied sync clients get a derived async client via
`RedisConnectionFactory.sync_to_async_redis`, closed only by
`adisconnect`. Docstrings document loop binding and that
Sentinel/Cluster sync clients need `redis_url` instead.
> 
> Sync behavior is preserved by **extracting shared helpers**
(`_recent_query`, `_recent_result`, `_count_query`, `_exchange`,
`_chat_messages`) so sync and async paths share query and message
construction. Two integration tests assert parity with sync (including a
monkeypatch that fails if async calls hit the sync Redis client) and
cover the `redis_url` async-index path.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
c1b5d75. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:minor Increment the minor version when merged released This issue/pull request has been released.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add async routing methods to SemanticRouter (aroute / aroute_many)

2 participants