Skip to content

Commit 59d36e1

Browse files
authored
fix(connection): convert TLS and unix-socket clients to async (#792)
Closes #789. ## Motivation `RedisConnectionFactory.sync_to_async_redis` passed the sync pool's `connection_kwargs` straight into `AsyncConnectionPool` and picked the TLS class with an exact-type check. A client built with `Redis(ssl=True, ...)`, `Redis(unix_socket_path=...)` or `Sentinel.master_for(...)` therefore raised `TypeError` on its first async command (`SemanticCache.acheck`, and `SemanticRouter.aroute` once #784 lands), and a subclass of `SSLConnection` got a plaintext async connection with the password still set. ## Changes - The async connection class is chosen with `issubclass`: TLS connections (and subclasses) map to `AsyncSSLConnection`, unix-socket connections to the async `UnixDomainSocketConnection`, everything else to `AsyncConnection`. - `_async_connection_kwargs` keeps only the keywords the chosen async class accepts, found by walking its `__init__` chain (`_accepted_init_kwargs`), so the filter follows whatever redis-py version is installed rather than a hard-coded key list. A sync-only keyword at its default (`None` or `False`) is dropped; one that is set, such as `ssl_validate_ocsp=True`, raises a `ValueError` naming it, because the async client cannot honour it and dropping it would silently weaken TLS verification. - Sentinel-managed pools are rejected up front with a clear `ValueError`, as `RedisCluster` already was. - Seven unit tests in `tests/unit/test_sync_to_async_redis.py`, none needing a server; five fail on `main`. ## Release Notes `SemanticCache`, `AsyncSearchIndex` and any other async path that converts a caller's sync client now work with clients configured with TLS keywords (`Redis(ssl=True, ...)`), unix sockets, and custom `SSLConnection` subclasses. Sentinel-managed clients, `RedisCluster` clients, and clients with a sync-only setting enabled (OCSP validation) raise `ValueError` at conversion time with a pointer to `redis_url`; previously they failed with `TypeError` on the first command. ## Reviewer Brief **Start here:** `redisvl/redis/connection.py`, `_async_connection_kwargs`. It decides which of the sync pool's settings reach the async connection and which abort the conversion; `sync_to_async_redis` only picks the class and calls it. **Where it could fail** - `redisvl/redis/connection.py`, `_SYNC_POOL_PLUMBING`: these four keywords are dropped by name because the sync pool computes them from the live settings; if a future redis-py adds another pool-computed keyword, every conversion on a version whose async class rejects it would raise again. Check against redis-py's `ConnectionPool._update_connection_kwargs`. - `redisvl/redis/connection.py`, `_is_unset`: `maint_notifications_config` is dropped unless `enabled is True`, so a user who set `enabled="auto"` on purpose gets an async client without notifications on 7.4/8.0; on 8.1 it passes through. - `redisvl/redis/connection.py`, `_init_parameters`: stops at the first `__init__` without `**kwargs`; for the sync side that is the user's `connection_class`, so a subclass with an explicit-keyword `__init__` contributes only its own defaults. **Not verified:** cluster mode (unchanged path, raises as before); a live TLS or unix-socket server (the tests stop at `make_connection()`, where every reported failure occurred); the sync `Retry` pass-through at the end of the function is pre-existing and left for a follow-up. **Beyond CI:** the ten tests pass on redis-py 7.4.1, 8.0.1 and 8.1.0; the `resp3` case fails on 7.4.1 and 8.0.1 without the fix, and the `tls-subclass` case fails on 8.0.1 when either the plumbing filter or the `maint_notifications_config` rule is removed. **Since the last brief:** 7c856ed rebases on 0.30.0, drops the pool-computed keywords, compares sync-only keywords to their own defaults, trims the `aroute` caveat, and folds the tests into two parametrised ones. <sub>Reviewed `7c856ed` · Claude Code (claude-fable-5-1) · lenses: correctness, security, design, tests, docs</sub> ## Notes - A set sync-only keyword raises rather than warns. OCSP validation is a security setting; converting it into a client that cannot validate OCSP, with only a log line (which #786 shows can be invisible), seemed the wrong default. - Sentinel conversion is rejected rather than rebuilt through `AsyncSentinel`, as the issue suggests. The pool does carry `sentinel_manager` and `service_name`, so a later change could rebuild it; that is a separate feature. - Once this merges, the SSL/unix-socket caveat in #784's `SemanticRouter` docstring reduces to Sentinel and cluster; I'll trim it there. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes how sync clients are translated into async pools across index/cache/router async entry points, including TLS and socket settings; incorrect filtering could misconfigure connections, though explicit errors protect sync-only security options. > > **Overview** > Fixes **`RedisConnectionFactory.sync_to_async_redis`** so async paths (`SemanticRouter.aroute`, `AsyncSearchIndex`, semantic caches) can reuse a caller’s sync client when it uses **TLS**, **TLS subclasses**, or **unix sockets**, instead of failing on the first command with **`TypeError`** or building a plaintext async pool. > > Conversion now picks the async connection class with **`issubclass`** (TLS → `AsyncSSLConnection`, unix → `AsyncUnixDomainSocketConnection`), filters pool kwargs through **`_async_connection_kwargs`** (version-aware via **`_init_parameters`**, strips RESP3 pool plumbing, drops sync-only defaults), and **raises `ValueError`** for **Sentinel-managed** pools and for user-set sync-only TLS options (e.g. **`ssl_validate_ocsp`**) rather than silently weakening security. **`SemanticRouter.aroute`** docs now say only **Sentinel** and **Cluster** need **`redis_url`** for async routing. > > Adds **`tests/unit/test_sync_to_async_redis.py`** covering successful mappings and failure modes without a live Redis server. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 7c856ed. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent f9f2638 commit 59d36e1

3 files changed

Lines changed: 251 additions & 11 deletions

File tree

‎redisvl/extensions/router/semantic.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -723,8 +723,8 @@ async def aroute(
723723
it returns the same match as ``router(statement)``.
724724
725725
Pass ``redis_url`` rather than a client to the router if you need
726-
async routing over SSL, a unix socket, Sentinel or Redis Cluster: those
727-
clients cannot be converted to their async form. With a vectorizer that
726+
async routing over Sentinel or Redis Cluster: those clients cannot be
727+
converted to their async form. With a vectorizer that
728728
has no native ``aembed``, such as the default ``HFTextVectorizer``,
729729
embedding still blocks the loop; only the Redis call is non-blocking.
730730
The async client is bound to the event loop that made the first call,

‎redisvl/redis/connection.py‎

Lines changed: 132 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import inspect
12
import os
23
from collections.abc import Mapping
34
from typing import Any, TypeVar, overload
@@ -11,10 +12,13 @@
1112
from redis.asyncio.connection import AbstractConnection as AsyncAbstractConnection
1213
from redis.asyncio.connection import Connection as AsyncConnection
1314
from redis.asyncio.connection import SSLConnection as AsyncSSLConnection
15+
from redis.asyncio.connection import (
16+
UnixDomainSocketConnection as AsyncUnixDomainSocketConnection,
17+
)
1418
from redis.asyncio.sentinel import Sentinel as AsyncSentinel
15-
from redis.connection import SSLConnection
19+
from redis.connection import SSLConnection, UnixDomainSocketConnection
1620
from redis.exceptions import ResponseError
17-
from redis.sentinel import Sentinel
21+
from redis.sentinel import Sentinel, SentinelConnectionPool
1822

1923
from redisvl import __version__
2024
from redisvl.redis.constants import (
@@ -302,6 +306,99 @@ async def _aidentify_client(
302306
logger.debug(f"CLIENT SETINFO was not applied, continuing without it: {e}")
303307

304308

309+
def _init_parameters(cls: type) -> dict[str, inspect.Parameter]:
310+
"""Return the keyword parameters ``cls(...)`` accepts, walking its MRO.
311+
312+
redis-py's connection classes each declare their own keywords and pass the
313+
rest up with ``**kwargs``, so the full set is the union of every
314+
``__init__`` in the chain up to the first one without ``**kwargs``. That
315+
stop is deliberate: a subclass whose ``__init__`` takes no ``**kwargs``
316+
can't be given its parents' keywords, so they aren't accepted either.
317+
"""
318+
params: dict[str, inspect.Parameter] = {}
319+
for klass in cls.__mro__:
320+
init = klass.__dict__.get("__init__")
321+
if init is None:
322+
continue
323+
signature = inspect.signature(init).parameters
324+
for name, param in signature.items():
325+
if name == "self" or name in params:
326+
continue
327+
if param.kind in (param.POSITIONAL_OR_KEYWORD, param.KEYWORD_ONLY):
328+
params[name] = param
329+
if not any(p.kind is p.VAR_KEYWORD for p in signature.values()):
330+
break
331+
return params
332+
333+
334+
# Keywords the sync pool computes for its own connections (redis-py >= 7.4
335+
# fills them in for maintenance notifications). They describe the sync pool,
336+
# not the user's settings, and the async pool computes its own, so they are
337+
# never carried over.
338+
_SYNC_POOL_PLUMBING = frozenset(
339+
{
340+
"maint_notifications_pool_handler",
341+
"orig_host_address",
342+
"orig_socket_timeout",
343+
"orig_socket_connect_timeout",
344+
}
345+
)
346+
347+
348+
def _is_unset(
349+
name: str, value: Any, sync_defaults: dict[str, inspect.Parameter]
350+
) -> bool:
351+
"""Whether ``value`` is what the sync connection class would have used anyway.
352+
353+
The comparison is against the sync class's own declared default, not
354+
against ``None``/``False``, so ``0`` is a setting and a keyword whose
355+
default is ``True`` is unset only when it is still ``True``.
356+
"""
357+
if name == "maint_notifications_config":
358+
# The sync pool injects a config in "auto" mode on its own; only a
359+
# user who turned notifications on explicitly would lose anything.
360+
return getattr(value, "enabled", None) is not True
361+
param = sync_defaults.get(name)
362+
if param is None or param.default is param.empty:
363+
return value is None
364+
default = param.default
365+
return value is default or (type(value) is type(default) and value == default)
366+
367+
368+
def _async_connection_kwargs(
369+
connection_kwargs: Mapping[str, Any],
370+
connection_class: type[AsyncAbstractConnection],
371+
sync_connection_class: type,
372+
) -> dict[str, Any]:
373+
"""Translate a sync pool's ``connection_kwargs`` for an async connection class.
374+
375+
Keywords the async class does not accept are dropped when they are unset,
376+
meaning they still hold the sync class's own default, or when they are
377+
pool plumbing the async pool computes for itself. One that the user set,
378+
such as ``ssl_validate_ocsp=True``, is a setting the async client cannot
379+
honour, so it raises instead of being silently lost.
380+
"""
381+
accepted = _init_parameters(connection_class)
382+
sync_defaults = _init_parameters(sync_connection_class)
383+
kwargs: dict[str, Any] = {}
384+
lost: list[str] = []
385+
for name, value in connection_kwargs.items():
386+
if name in _SYNC_POOL_PLUMBING:
387+
continue
388+
if name in accepted:
389+
kwargs[name] = value
390+
elif not _is_unset(name, value, sync_defaults):
391+
lost.append(name)
392+
if lost:
393+
raise ValueError(
394+
"Cannot convert this Redis client to async: the async "
395+
f"{connection_class.__name__} does not support "
396+
f"{', '.join(sorted(lost))}. Pass redis_url and connection_kwargs "
397+
"instead so the async client is built directly."
398+
)
399+
return kwargs
400+
401+
305402
def _normalize_index_info_mapping(value: Any) -> dict[str, Any]:
306403
"""Normalize a mapping-shaped or alternating-list FT.INFO section."""
307404
if isinstance(value, dict):
@@ -876,7 +973,21 @@ def get_async_redis_cluster_connection(
876973
def sync_to_async_redis(
877974
redis_client: SyncRedisClient,
878975
) -> AsyncRedisClient:
879-
"""Convert a synchronous Redis client to an asynchronous one."""
976+
"""Convert a synchronous Redis client to an asynchronous one.
977+
978+
The async client is built from the sync client's connection settings:
979+
TLS and unix-socket connections map to redis-py's stock async
980+
connection classes, so the behaviour of a custom ``SSLConnection``
981+
subclass is not carried over. Keywords that only the sync classes
982+
accept are dropped when unset.
983+
984+
Raises:
985+
ValueError: For a ``RedisCluster`` client, a Sentinel-managed
986+
client (from ``Sentinel.master_for`` or ``slave_for``), or a
987+
client with a sync-only setting the async client cannot honour
988+
(such as OCSP validation). Build the async client from a URL
989+
in those cases.
990+
"""
880991
if isinstance(redis_client, RedisCluster):
881992
raise ValueError(
882993
"RedisCluster is not supported for sync-to-async conversion."
@@ -885,17 +996,29 @@ def sync_to_async_redis(
885996
# At this point, redis_client is guaranteed to be Redis type
886997
assert isinstance(redis_client, Redis) # Type narrowing for MyPy
887998

999+
pool = redis_client.connection_pool
1000+
if isinstance(pool, SentinelConnectionPool):
1001+
raise ValueError(
1002+
"Sentinel-managed clients are not supported for sync-to-async "
1003+
"conversion. Build the async client from the sentinel URL instead."
1004+
)
1005+
8881006
# pick the right connection class
889-
connection_class: type[AsyncAbstractConnection] = (
890-
AsyncSSLConnection
891-
if redis_client.connection_pool.connection_class == SSLConnection
892-
else AsyncConnection
893-
)
1007+
connection_class: type[AsyncAbstractConnection]
1008+
if issubclass(pool.connection_class, SSLConnection):
1009+
connection_class = AsyncSSLConnection
1010+
elif issubclass(pool.connection_class, UnixDomainSocketConnection):
1011+
connection_class = AsyncUnixDomainSocketConnection
1012+
else:
1013+
connection_class = AsyncConnection
1014+
8941015
# make async client
8951016
return AsyncRedis.from_pool(
8961017
AsyncConnectionPool(
8971018
connection_class=connection_class,
898-
**redis_client.connection_pool.connection_kwargs,
1019+
**_async_connection_kwargs(
1020+
pool.connection_kwargs, connection_class, pool.connection_class
1021+
),
8991022
)
9001023
)
9011024

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
"""Unit tests for RedisConnectionFactory.sync_to_async_redis.
2+
3+
No server is needed: every failure the conversion can produce happens when the
4+
async pool builds its first connection, so ``make_connection()`` is enough.
5+
"""
6+
7+
import pytest
8+
from redis import Redis
9+
from redis.asyncio.connection import Connection as AsyncConnection
10+
from redis.asyncio.connection import SSLConnection as AsyncSSLConnection
11+
from redis.asyncio.connection import (
12+
UnixDomainSocketConnection as AsyncUnixDomainSocketConnection,
13+
)
14+
from redis.connection import ConnectionPool, SSLConnection
15+
from redis.sentinel import Sentinel
16+
17+
from redisvl.redis.connection import (
18+
RedisConnectionFactory,
19+
_async_connection_kwargs,
20+
_init_parameters,
21+
)
22+
23+
24+
class _TLS(SSLConnection):
25+
pass
26+
27+
28+
@pytest.mark.parametrize(
29+
"client, expected, settings",
30+
[
31+
(
32+
Redis(host="h", port=6380, db=3, decode_responses=True),
33+
AsyncConnection,
34+
{"host": "h", "port": 6380, "db": 3, "decode_responses": True},
35+
),
36+
# RESP3 makes the sync pool inject maintenance-notification plumbing
37+
# that older async classes don't accept (redis-py 7.4 and 8.0).
38+
(
39+
Redis(host="h", protocol=3),
40+
AsyncConnection,
41+
{"host": "h", "protocol": 3},
42+
),
43+
(
44+
Redis(host="h", ssl=True, password="pw", ssl_check_hostname=True),
45+
AsyncSSLConnection,
46+
{"host": "h", "password": "pw", "ssl_check_hostname": True},
47+
),
48+
(
49+
Redis(connection_pool=ConnectionPool(connection_class=_TLS, host="h")),
50+
AsyncSSLConnection,
51+
{"host": "h"},
52+
),
53+
(
54+
Redis(unix_socket_path="/tmp/redis.sock", db=2),
55+
AsyncUnixDomainSocketConnection,
56+
{"path": "/tmp/redis.sock", "db": 2},
57+
),
58+
# The URL form sets connection_class rather than unix_socket_path.
59+
(
60+
Redis.from_url("unix:///tmp/redis.sock"),
61+
AsyncUnixDomainSocketConnection,
62+
{"path": "/tmp/redis.sock"},
63+
),
64+
],
65+
ids=["plain", "resp3", "tls", "tls-subclass", "unix", "unix-url"],
66+
)
67+
def test_converts_to_the_matching_async_connection(client, expected, settings):
68+
pool = RedisConnectionFactory.sync_to_async_redis(client).connection_pool
69+
70+
assert type(pool.make_connection()) is expected
71+
assert settings.items() <= pool.connection_kwargs.items()
72+
73+
74+
@pytest.mark.parametrize(
75+
"client, match",
76+
[
77+
(Sentinel([("localhost", 26379)]).master_for("mymaster"), "Sentinel-managed"),
78+
(Redis(host="h", ssl=True, ssl_validate_ocsp=True), "ssl_validate_ocsp"),
79+
],
80+
ids=["sentinel", "sync-only-setting"],
81+
)
82+
def test_unconvertible_client_raises(client, match):
83+
with pytest.raises(ValueError, match=match):
84+
RedisConnectionFactory.sync_to_async_redis(client)
85+
86+
87+
def test_sync_only_keyword_is_unset_only_at_its_own_default():
88+
# ssl_validate_ocsp_stapled defaults to False on the sync class; 0 is a
89+
# value the user chose, not the default, so it is not silently dropped.
90+
assert "ssl_validate_ocsp_stapled" not in _init_parameters(AsyncSSLConnection)
91+
92+
kwargs = _async_connection_kwargs(
93+
{"host": "h", "ssl_validate_ocsp_stapled": False},
94+
AsyncSSLConnection,
95+
SSLConnection,
96+
)
97+
assert kwargs == {"host": "h"}
98+
99+
with pytest.raises(ValueError, match="ssl_validate_ocsp_stapled"):
100+
_async_connection_kwargs(
101+
{"host": "h", "ssl_validate_ocsp_stapled": 0},
102+
AsyncSSLConnection,
103+
SSLConnection,
104+
)
105+
106+
107+
def test_explicitly_enabled_maintenance_notifications_are_not_dropped():
108+
maint = pytest.importorskip("redis.maint_notifications")
109+
config = maint.MaintNotificationsConfig(enabled=True)
110+
client = Redis(host="h", protocol=3, maint_notifications_config=config)
111+
112+
if "maint_notifications_config" in _init_parameters(AsyncConnection):
113+
pool = RedisConnectionFactory.sync_to_async_redis(client).connection_pool
114+
assert pool.connection_kwargs["maint_notifications_config"] is config
115+
else:
116+
with pytest.raises(ValueError, match="maint_notifications_config"):
117+
RedisConnectionFactory.sync_to_async_redis(client)

0 commit comments

Comments
 (0)