Skip to content

feat(message_history): add async methods to MessageHistory - #816

Merged
vishal-bala merged 3 commits into
redis:mainfrom
Yarmoluk:feat/message-history-async
Nov 6, 2026
Merged

vishal-bala merged 3 commits into
redis:mainfrom
Yarmoluk:feat/message-history-async

Conversation

@Yarmoluk

@Yarmoluk Yarmoluk commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Branch: feat/message-history-async (fork Yarmoluk/redis-vl-python), rebased onto main at 506b961 (includes #792 and #785).
Commits: 82f91e4 feat, 93657d7 docs, c1b5d75 test. Label: auto:minor.

Summary

Follow-up to #784, which gave SemanticRouter aroute / aroute_many. MessageHistory had the same gap: an agent running on asyncio blocked its event loop on every history read or write. This PR adds the async twins -- aget_recent, astore, aadd_messages, aadd_message, acount, adrop, aclear, adisconnect and an aindex property -- using the same pattern #784 landed:

  • The AsyncSearchIndex is built lazily on the first async call, from the sync index's connection settings. A history built from redis_url gets its own async connection from that URL and connection_kwargs; a history given a sync redis_client gets an async client converted from that client's connection settings by RedisConnectionFactory.sync_to_async_redis, owned by the async index, so adisconnect closes only the converted client and the caller's sync client is never touched or rebuilt.
  • Every async method goes through the AsyncSearchIndex; none reaches into a Redis client directly (adrop deletes with aindex.drop_keys(aindex.key(id))).
  • Query building (_recent_query, _count_query) and message construction (_chat_messages, _exchange) move into helpers shared by the sync and async paths, so the two cannot drift.
  • The docstrings say what is and is not supported, in the router's wording: since fix(connection): convert TLS and unix-socket clients to async #792 a sync client over TLS or a unix socket converts; Sentinel and Redis Cluster clients do not (pass redis_url instead); the async client is bound to the loop that made the first call, so call adisconnect before switching loops.

No sync behaviour changes. SemanticMessageHistory is not covered here (see Notes).

Reviewer Brief

Start here: redisvl/extensions/message_history/message_history.py, _get_async_index() (line 418). Every async method goes through it, and it decides whether the async index connects from the URL the sync index owns or from a client converted from the caller's sync client; it is the same code as SemanticRouter._get_async_index.

Where it could fail

  • message_history.py:432: a caller-supplied Sentinel or Cluster client, or one with a sync-only setting such as ssl_validate_ocsp=True, raises ValueError from sync_to_async_redis at the first async call; the aget_recent docstring names Sentinel and Cluster but, like the router's, not the sync-only-setting case.
  • message_history.py:154: adrop removes the key with UNLINK (via drop_keys) where sync drop uses DEL; both are @keyspace @write, but an ACL that grants +del by name and not +unlink would allow drop and deny adrop (UNLINK, DEL).
  • message_history.py:449: the async client belongs to the loop that made the first call; a second asyncio.run() on the same history without adisconnect in between fails, the same contract SemanticRouter has shipped since feat(router): add async aroute and aroute_many to SemanticRouter #784.

Not verified: Redis Cluster and Sentinel (no cluster tests run locally or in CI); the TLS and unix-socket conversion paths were not exercised end to end through MessageHistory (they are covered by #792's own tests); connection_kwargs and lib_name forwarding on the URL path is not asserted by a test.

Beyond CI: each of these mutations makes test_async_message_history_matches_sync fail: astore ignoring session_tag, adrop going through the sync client, adrop(id) ignoring id, the async index rebuilt per call, and adisconnect not closing the converted client. The two new integration tests pass locally against redis:8.4 via testcontainers.

Since the last brief: rebased onto 506b961 (#792 now converts TLS and unix-socket clients, so the caveat narrows to Sentinel and Cluster); adrop deletes through aindex.drop_keys instead of the async client; the sync-equivalence test now blocks the sync client's execute_command and pipeline, seeds a second session with astore, and pins tool_call_id/metadata, adrop(id), async-index reuse and adisconnect closing the converted pool.

Reviewed c1b5d75 · Claude Code (claude-opus-5-5) · lenses: correctness, security, design, tests, docs, Redis behaviour

Test plan

Run on the rebased branch at c1b5d75, macOS, uv, Docker running (testcontainers starts redis:8.4):

make format
# All done! ✨ 🍰 ✨
# 272 files left unchanged.

rm -rf .mypy_cache && make check-types
# Success: no issues found in 122 source files

uv run pytest tests/integration/test_message_history.py -k async_message_history -q -p no:cacheprovider
# 2 passed, 33 deselected

uv run pytest -n auto tests/integration/test_message_history.py tests/unit -q -p no:cacheprovider
# 3 failed, 1500 passed, 9 skipped, 53 errors

The 3 failures and 53 errors are the identical set on main at 506b961: every one is ModuleNotFoundError: sentence_transformers (the HuggingFace vectorizer is not installed locally), in SemanticMessageHistory tests this PR does not touch.

test_async_message_history_matches_sync takes its expected values from the sync methods, then patches execute_command and pipeline on the sync index's Redis client to raise, so any async method that made a blocking call would fail it. test_async_message_history_with_redis_url checks that a URL-built history builds its async index from that URL rather than converting the client.

Notes


Note

Medium Risk
New async Redis connection lifecycle (lazy index, loop binding, client conversion) could misbehave for Sentinel/Cluster/unix-socket setups; sync API and data paths are unchanged.

Overview
Adds async counterparts to MessageHistory read/write/clear/drop operations (aget_recent, astore, aadd_messages, aadd_message, acount, adrop, aclear, plus adisconnect and an aindex property) so asyncio agents can use history without blocking the event loop.

Async work goes through a lazily created AsyncSearchIndex that reuses the sync index’s schema and connection settings: URL-owned clients open a separate async connection from redis_url; caller-supplied sync clients get a derived async client via RedisConnectionFactory.sync_to_async_redis, closed only by adisconnect. Docstrings document loop binding and that Sentinel/Cluster sync clients need redis_url instead.

Sync behavior is preserved by extracting shared helpers (_recent_query, _recent_result, _count_query, _exchange, _chat_messages) so sync and async paths share query and message construction. Two integration tests assert parity with sync (including a monkeypatch that fails if async calls hit the sync Redis client) and cover the redis_url async-index path.

Reviewed by Cursor Bugbot for commit c1b5d75. Bugbot is set up for automated code reviews on this repo. Configure here.

@vishal-bala vishal-bala added the auto:minor Increment the minor version when merged label Nov 5, 2026
MessageHistory had no async path, so reading or writing chat history from
an async agent loop blocked the event loop on every Redis call. Add
aget_recent, astore, aadd_messages, aadd_message, acount, adrop, aclear,
adisconnect and an aindex property. The async index is built on first use
from the sync SearchIndex's connection settings, the same way SemanticRouter
does it. Query building and message construction move into helpers shared
by the sync and async paths so the two cannot drift.
…eats

The caveats that matter before the first async call -- which clients
convert to async, loop binding, and adisconnect -- were only on
aget_recent, so a caller who reads astore or aadd_messages first never
saw them. Each short async docstring now points there.
Seed the other session through astore, block every sync round trip by
patching execute_command and pipeline on the sync client, and pin a tool
message's tool_call_id and metadata through an async write, adrop(id),
the reused async index and adisconnect closing the converted client.
adrop now goes through AsyncSearchIndex.drop_keys rather than the private
client, and the connection caveat drops SSL and unix sockets, which
convert since redis#792.
@Yarmoluk
Yarmoluk force-pushed the feat/message-history-async branch from 8fc3ce8 to c1b5d75 Compare November 5, 2026 11:54
@Yarmoluk

Yarmoluk commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. Rebased onto main and pushed c1b5d75:

  • The caveat now reads "over Sentinel or Redis Cluster", matching the router.
  • adrop uses aindex.drop_keys(aindex.key(id)).
  • The test seeds the other session with astore inside the try, and blocks sync calls by patching execute_command and pipeline on the sync client.
  • It now pins tool_call_id and metadata through aadd_message, adrop(id), the reused async index and adisconnect closing the converted client. Each of these assertions, and the astore session tag, fails the test when I break the corresponding code.

@vishal-bala
vishal-bala self-requested a review November 5, 2026 13:16
assert standard_history.aindex is None
# adisconnect closed the converted client's connections.
assert pool is not None
connections = pool._available_connections + list(pool._in_use_connections)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: _available_connections and _in_use_connections are private to redis-py's pool, so an internal rename there would break this assert without anything in redisvl changing. Not blocking, but if there's a public way to check the client was closed, it'd be sturdier.

@vishal-bala
vishal-bala merged commit 2a5d4b6 into redis:main Nov 6, 2026
65 checks passed
@applied-ai-release-bot

Copy link
Copy Markdown

🚀 PR was released in v0.32.0 🚀

@applied-ai-release-bot applied-ai-release-bot Bot added the released This issue/pull request has been released. label Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:minor Increment the minor version when merged released This issue/pull request has been released.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants