Repository navigation
fix(router): apply per-route thresholds before max_k truncation - #766
Merged
Merged
Conversation
The FT.AGGREGATE plan ran SORTBY @distance MAX max_k before FILTER, so a closer route that missed its own distance_threshold could crowd out a farther route that met its threshold. route() then returned an empty RouteMatch and route_many() returned fewer matches than it should. Call .filter() before .sort_by() so the plan is GROUPBY -> REDUCE -> FILTER -> SORTBY MAX.
Address review feedback: the test's premise relies on avg aggregation (under min, greeting would sit at ~0 and pass its threshold), so pin it rather than depend on the default. Exercise max_k via the routing config in route_many(), and say in the builder comment that FT.AGGREGATE itself runs steps in order.
vishal-bala
marked this pull request as ready for review
October 15, 2026 08:39
This was referenced Oct 26, 2026
…fix/router-filter-before-sortby # Conflicts: # tests/integration/test_semantic_router.py
limjoobin
approved these changes
Oct 28, 2026
limjoobin
left a comment
Contributor
There was a problem hiding this comment.
LGTM, this PR fixes the aggregation ordering issue.
|
🚀 PR was released in |
vishal-bala
pushed a commit
that referenced
this pull request
Oct 30, 2026
## Motivation Closes #783. `SemanticRouter` had no async query path, so routing from async apps (LangGraph nodes, FastAPI handlers, MCP servers) blocked the event loop twice per call: on the embedding and on the `FT.AGGREGATE` round trip. `SemanticCache` already has async lookups; the router had none. ## Changes ### `aroute()` and `aroute_many()` Async counterparts of `router(...)` and `route_many(...)`. They embed with `vectorizer.aembed` and run the same aggregate request on an `AsyncSearchIndex` over the same index, so they return exactly what the sync methods return. Everything after `vector` is keyword-only, so a positional `distance_threshold` (deprecated in `route_many`) can't be read as an aggregation method. - `_get_route_matches` is split into request building (`_route_matches_request`) and row processing (`_route_matches_from_result`), shared by the sync and async paths so the two cannot drift. `_build_aggregate_request` is untouched, so this does not collide with #766. - The async index is created on first use (same pattern as `SemanticCache`), derived from the sync `SearchIndex`: when the sync index owns its client and has a URL, the async index connects from that URL, connection kwargs and `lib_name`; otherwise the caller's client is converted with `RedisConnectionFactory.sync_to_async_redis`. `from_existing(redis_url=...)` builds an owned client from the URL, so it takes the URL branch too. - `adisconnect()` closes the async connection; `aindex` exposes the async index (None before first use). - Mutating methods (`add_route`, `remove_route`, ...) stay sync to keep the change small. ## Release Notes `SemanticRouter` gains `aroute()`, `aroute_many()`, `adisconnect()` and an `aindex` property. The async index is created on first use. Routers given an SSL, unix-socket, Sentinel or `RedisCluster` client can't convert it to async; pass `redis_url` to the router for async routing in those setups. Vectorizers without a native `aembed` (e.g. the default `HFTextVectorizer`) still embed synchronously; the Redis call is non-blocking either way. The async client is bound to the event loop that made the first call; call `adisconnect()` before switching loops. ## Reviewer Brief **Start here:** `redisvl/extensions/router/semantic.py`, `_get_async_index`. It decides how the async index reaches Redis, and every `aroute` / `aroute_many` call goes through it. **Where it could fail** - `redisvl/extensions/router/semantic.py:557`: the URL branch is taken only when the sync index owns its client *and* has a URL. Check that a router built with `redis_client=` plus `redis_url=` (both given) converts the client rather than connecting from the URL, since the client is what the sync path uses. - `redisvl/extensions/router/semantic.py:567`: the client-conversion branch raises on the first async call for SSL, unix-socket, Sentinel and Cluster clients (`sync_to_async_redis` limitation, pre-existing). The docstrings say to pass `redis_url` instead; nothing prevents the mismatch at construction. **Not verified:** cluster mode (async path unsupported for `RedisCluster` clients); non-blocking embedding with a native-async vectorizer (the tests use `HFTextVectorizer`, which embeds synchronously, so only the Redis call is exercised async); Redis < 7 error mapping (shared code, unreachable on the test container). **Beyond CI:** `test_async_routing_matches_sync` (`tests/integration/test_semantic_router.py:822`) fails with `AssertionError: the async path made a blocking call` when `aroute` is changed to call the sync `embed`, and passes on this branch; `test_aroute_from_existing_url_connects_from_url` passes with the async index derived from the sync one, so the earlier `from_existing` patch is no longer needed. **Since the last brief:** `8a70416` applies the review: the async index is derived from `self._index` (no router-side copies of URL/kwargs/`lib_name`, no `from_existing` change), keyword-only arguments after `vector`, the three docstring notes, and the six async tests replaced by one that blocks the sync paths. <sub>Reviewed `8a70416` · Claude Code (claude-fable-5-1) · lenses: correctness, tests. Ran: `pytest tests/integration/test_semantic_router.py` (42 passed, 1 skipped; Redis via the repo's testcontainers compose), `pytest tests/unit -k "router or create_index"` (50 passed), `make check-types` (120 files, no issues), `make format`.</sub> 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > New async API reuses the existing FT.AGGREGATE flow and SemanticCache-style connection setup; main caveats are documented (cluster client conversion, event-loop binding), not new security or data paths. > > **Overview** > Adds **non-blocking routing** for async apps via `aroute()` and `aroute_many()`, mirroring `router()` / `route_many()` by using `vectorizer.aembed` and a lazily created `AsyncSearchIndex` on the same Redis index. > > The sync lookup path is refactored so request building (`_route_matches_request`), Redis error handling (`_raise_aggregate_error`), and result parsing (`_route_matches_from_result`) are shared with `_aget_route_matches`, reducing drift between sync and async. **`adisconnect()`** and an **`aindex`** property manage the optional async client (URL-based when the router owns its connection, otherwise `sync_to_async_redis`). > > Integration tests assert async results match sync, that the async path does not call blocking `embed`/`aggregate`, and that routers loaded with **`from_existing(redis_url=...)`** wire async Redis from the URL. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8a70416. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SemanticRouter._build_aggregate_requestbuilt its FT.AGGREGATE as GROUPBY → REDUCE → SORTBY @distance MAX max_k → FILTER. FT.AGGREGATE runs its steps in the order given, and redis-py emits them in call order, so the router cut the result down to the max_k closest routes before it checked each route's owndistance_threshold. When a close route missed its threshold, it still took a slot, and a farther route that met its threshold never reached the FILTER.route()(and calling the router directly) always aggregates with max_k=1, whateverRoutingConfig.max_ksays. So the most common call was always exposed: whenever the closest route missed its own threshold,route()returned an emptyRouteMatch, even if another route qualified.route_many()could likewise return fewer matches than it should.For example, with the test suite's
all-mpnet-base-v2vectorizer, avg aggregation and max_k=1, the query "hello" sits about 0.07 from greeting (threshold 0.01) and about 0.60 from farewell (threshold 0.8). The old plan kept only greeting, the FILTER then dropped it, and nothing came back.The fix calls
.filter()before.sort_by().build_args()now showsGROUPBY … REDUCE … FILTER … SORTBY 2 @distance ASC MAX 1. RedisVL has no async router, so nothing else needed changing.Tests
test_threshold_filter_applies_before_max_k_truncationcovers bothroute()androute_many(), withmax_ktaken from the routing config. It fails onmain(assert None == 'farewell') and passes with the fix.tests/integration/test_semantic_router.pyandtests/unit/test_semantic_router_queries.pypass locally against Redis 8.2 (43 passed, 1 skipped).Release Notes
Fixed
SemanticRouter.route()androute_many()sometimes missing a route that met itsdistance_threshold. Each route's threshold is now applied before results are truncated tomax_k, so a closer route that fails its own threshold no longer displaces a farther one that passes. Previouslyroute()could return an emptyRouteMatchandroute_many()could return fewer matches than expected.Note
Medium Risk
Changes Redis aggregation step order for all semantic routing queries; behavior fix is targeted but affects core matching for
route()androute_many().Overview
SemanticRouter now builds
FT.AGGREGATEso per-routedistance_thresholdfiltering runs beforeSORTBY … MAX max_k, fixing cases where the closest route failed its threshold but still consumed amax_kslot.Previously
route()could return an empty match androute_many()could omit qualifying routes when a nearer route was over its threshold._build_aggregate_requestchains.filter()before.sort_by()(with an inline comment on redis-py step order).A new integration test asserts that for
"hello", farewell wins over greeting when greeting’s threshold is too tight and farewell’s is not, for bothroute()androute_many().Reviewed by Cursor Bugbot for commit 95c000d. Bugbot is set up for automated code reviews on this repo. Configure here.