Skip to content

fix(router): apply per-route thresholds before max_k truncation - #766

Merged
vishal-bala merged 3 commits into
mainfrom
fix/router-filter-before-sortby
Oct 30, 2026
Merged

vishal-bala merged 3 commits into
mainfrom
fix/router-filter-before-sortby

Conversation

@vishal-bala

@vishal-bala vishal-bala commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

SemanticRouter._build_aggregate_request built its FT.AGGREGATE as GROUPBY → REDUCE → SORTBY @distance MAX max_k → FILTER. FT.AGGREGATE runs its steps in the order given, and redis-py emits them in call order, so the router cut the result down to the max_k closest routes before it checked each route's own distance_threshold. When a close route missed its threshold, it still took a slot, and a farther route that met its threshold never reached the FILTER.

route() (and calling the router directly) always aggregates with max_k=1, whatever RoutingConfig.max_k says. So the most common call was always exposed: whenever the closest route missed its own threshold, route() returned an empty RouteMatch, even if another route qualified. route_many() could likewise return fewer matches than it should.

For example, with the test suite's all-mpnet-base-v2 vectorizer, avg aggregation and max_k=1, the query "hello" sits about 0.07 from greeting (threshold 0.01) and about 0.60 from farewell (threshold 0.8). The old plan kept only greeting, the FILTER then dropped it, and nothing came back.

The fix calls .filter() before .sort_by(). build_args() now shows GROUPBY … REDUCE … FILTER … SORTBY 2 @distance ASC MAX 1. RedisVL has no async router, so nothing else needed changing.

Tests

  • New integration test test_threshold_filter_applies_before_max_k_truncation covers both route() and route_many(), with max_k taken from the routing config. It fails on main (assert None == 'farewell') and passes with the fix.
  • tests/integration/test_semantic_router.py and tests/unit/test_semantic_router_queries.py pass locally against Redis 8.2 (43 passed, 1 skipped).

Release Notes

Fixed SemanticRouter.route() and route_many() sometimes missing a route that met its distance_threshold. Each route's threshold is now applied before results are truncated to max_k, so a closer route that fails its own threshold no longer displaces a farther one that passes. Previously route() could return an empty RouteMatch and route_many() could return fewer matches than expected.


Note

Medium Risk
Changes Redis aggregation step order for all semantic routing queries; behavior fix is targeted but affects core matching for route() and route_many().

Overview
SemanticRouter now builds FT.AGGREGATE so per-route distance_threshold filtering runs before SORTBY … MAX max_k, fixing cases where the closest route failed its threshold but still consumed a max_k slot.

Previously route() could return an empty match and route_many() could omit qualifying routes when a nearer route was over its threshold. _build_aggregate_request chains .filter() before .sort_by() (with an inline comment on redis-py step order).

A new integration test asserts that for "hello", farewell wins over greeting when greeting’s threshold is too tight and farewell’s is not, for both route() and route_many().

Reviewed by Cursor Bugbot for commit 95c000d. Bugbot is set up for automated code reviews on this repo. Configure here.

The FT.AGGREGATE plan ran SORTBY @distance MAX max_k before FILTER, so a
closer route that missed its own distance_threshold could crowd out a
farther route that met its threshold. route() then returned an empty
RouteMatch and route_many() returned fewer matches than it should.

Call .filter() before .sort_by() so the plan is GROUPBY -> REDUCE ->
FILTER -> SORTBY MAX.
Address review feedback: the test's premise relies on avg aggregation
(under min, greeting would sit at ~0 and pass its threshold), so pin it
rather than depend on the default. Exercise max_k via the routing config
in route_many(), and say in the builder comment that FT.AGGREGATE itself
runs steps in order.
@vishal-bala
vishal-bala marked this pull request as ready for review October 15, 2026 08:39
@vishal-bala vishal-bala added the auto:patch Increment the patch version when merged label Oct 15, 2026
…fix/router-filter-before-sortby

# Conflicts:
#	tests/integration/test_semantic_router.py

@limjoobin limjoobin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this PR fixes the aggregation ordering issue.

@vishal-bala
vishal-bala merged commit a54c8a7 into main Oct 30, 2026
58 checks passed
@vishal-bala
vishal-bala deleted the fix/router-filter-before-sortby branch October 30, 2026 09:01
@applied-ai-release-bot

Copy link
Copy Markdown

🚀 PR was released in v0.29.0 🚀

@applied-ai-release-bot applied-ai-release-bot Bot added the released This issue/pull request has been released. label Oct 30, 2026
vishal-bala pushed a commit that referenced this pull request Oct 30, 2026
## Motivation

Closes #783.

`SemanticRouter` had no async query path, so routing from async apps
(LangGraph nodes, FastAPI handlers, MCP servers) blocked the event loop
twice per call: on the embedding and on the `FT.AGGREGATE` round trip.
`SemanticCache` already has async lookups; the router had none.

## Changes

### `aroute()` and `aroute_many()`

Async counterparts of `router(...)` and `route_many(...)`. They embed
with `vectorizer.aembed` and run the same aggregate request on an
`AsyncSearchIndex` over the same index, so they return exactly what the
sync methods return. Everything after `vector` is keyword-only, so a
positional `distance_threshold` (deprecated in `route_many`) can't be
read as an aggregation method.

- `_get_route_matches` is split into request building
(`_route_matches_request`) and row processing
(`_route_matches_from_result`), shared by the sync and async paths so
the two cannot drift. `_build_aggregate_request` is untouched, so this
does not collide with #766.
- The async index is created on first use (same pattern as
`SemanticCache`), derived from the sync `SearchIndex`: when the sync
index owns its client and has a URL, the async index connects from that
URL, connection kwargs and `lib_name`; otherwise the caller's client is
converted with `RedisConnectionFactory.sync_to_async_redis`.
`from_existing(redis_url=...)` builds an owned client from the URL, so
it takes the URL branch too.
- `adisconnect()` closes the async connection; `aindex` exposes the
async index (None before first use).
- Mutating methods (`add_route`, `remove_route`, ...) stay sync to keep
the change small.

## Release Notes

`SemanticRouter` gains `aroute()`, `aroute_many()`, `adisconnect()` and
an `aindex` property. The async index is created on first use. Routers
given an SSL, unix-socket, Sentinel or `RedisCluster` client can't
convert it to async; pass `redis_url` to the router for async routing in
those setups. Vectorizers without a native `aembed` (e.g. the default
`HFTextVectorizer`) still embed synchronously; the Redis call is
non-blocking either way. The async client is bound to the event loop
that made the first call; call `adisconnect()` before switching loops.

## Reviewer Brief

**Start here:** `redisvl/extensions/router/semantic.py`,
`_get_async_index`. It decides how the async index reaches Redis, and
every `aroute` / `aroute_many` call goes through it.

**Where it could fail**
- `redisvl/extensions/router/semantic.py:557`: the URL branch is taken
only when the sync index owns its client *and* has a URL. Check that a
router built with `redis_client=` plus `redis_url=` (both given)
converts the client rather than connecting from the URL, since the
client is what the sync path uses.
- `redisvl/extensions/router/semantic.py:567`: the client-conversion
branch raises on the first async call for SSL, unix-socket, Sentinel and
Cluster clients (`sync_to_async_redis` limitation, pre-existing). The
docstrings say to pass `redis_url` instead; nothing prevents the
mismatch at construction.

**Not verified:** cluster mode (async path unsupported for
`RedisCluster` clients); non-blocking embedding with a native-async
vectorizer (the tests use `HFTextVectorizer`, which embeds
synchronously, so only the Redis call is exercised async); Redis < 7
error mapping (shared code, unreachable on the test container).

**Beyond CI:** `test_async_routing_matches_sync`
(`tests/integration/test_semantic_router.py:822`) fails with
`AssertionError: the async path made a blocking call` when `aroute` is
changed to call the sync `embed`, and passes on this branch;
`test_aroute_from_existing_url_connects_from_url` passes with the async
index derived from the sync one, so the earlier `from_existing` patch is
no longer needed.

**Since the last brief:** `8a70416` applies the review: the async index
is derived from `self._index` (no router-side copies of
URL/kwargs/`lib_name`, no `from_existing` change), keyword-only
arguments after `vector`, the three docstring notes, and the six async
tests replaced by one that blocks the sync paths.

<sub>Reviewed `8a70416` · Claude Code (claude-fable-5-1) · lenses:
correctness, tests. Ran: `pytest
tests/integration/test_semantic_router.py` (42 passed, 1 skipped; Redis
via the repo's testcontainers compose), `pytest tests/unit -k "router or
create_index"` (50 passed), `make check-types` (120 files, no issues),
`make format`.</sub>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> New async API reuses the existing FT.AGGREGATE flow and
SemanticCache-style connection setup; main caveats are documented
(cluster client conversion, event-loop binding), not new security or
data paths.
> 
> **Overview**
> Adds **non-blocking routing** for async apps via `aroute()` and
`aroute_many()`, mirroring `router()` / `route_many()` by using
`vectorizer.aembed` and a lazily created `AsyncSearchIndex` on the same
Redis index.
> 
> The sync lookup path is refactored so request building
(`_route_matches_request`), Redis error handling
(`_raise_aggregate_error`), and result parsing
(`_route_matches_from_result`) are shared with `_aget_route_matches`,
reducing drift between sync and async. **`adisconnect()`** and an
**`aindex`** property manage the optional async client (URL-based when
the router owns its connection, otherwise `sync_to_async_redis`).
> 
> Integration tests assert async results match sync, that the async path
does not call blocking `embed`/`aggregate`, and that routers loaded with
**`from_existing(redis_url=...)`** wire async Redis from the URL.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
8a70416. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:patch Increment the patch version when merged released This issue/pull request has been released.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants