Skip to content

Commit 119c795

Browse files
Python: [BREAKING] Scope Redis history storage keys (#8236)
* Python: Scope Redis history storage keys Co-authored-by: Copilot App <[email protected]> * Python: Address Redis history review feedback Co-authored-by: Copilot App <[email protected]> --------- Co-authored-by: Copilot App <[email protected]>
1 parent 2497597 commit 119c795

6 files changed

Lines changed: 350 additions & 50 deletions

File tree

‎python/packages/redis/AGENTS.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,9 @@ Redis-based storage for agent threads and context.
44

55
## Main Classes
66

7-
- **`RedisHistoryProvider`** - Persistent chat history provider using Redis
7+
- **`RedisHistoryProvider`** - Persistent chat history provider using scoped Redis keys. Scoped mode requires an
8+
application ID and non-empty session ID, and can additionally isolate tenants and agents. Use explicit legacy mode
9+
only while deliberately migrating historical keys; scoped mode never accesses them automatically.
810
- **`RedisContextProvider`** - Context provider with Redis-backed retrieval
911
- **`RedisSettings`** - TypedDict connection settings for vector stores, resolved with core `load_settings` from
1012
explicit URL overrides, an optional .env file, or `REDIS_URL`. URLs use `SecretString` to mask credentials.
@@ -40,7 +42,9 @@ their own indexes/keys, never `FLUSHALL`. Coverage includes both formats and
4042
from agent_framework.redis import RedisContextProvider, RedisHistoryProvider
4143

4244
context_provider = RedisContextProvider(redis_url="redis://localhost:6379")
43-
history_provider = RedisHistoryProvider(redis_url="redis://localhost:6379")
45+
history_provider = RedisHistoryProvider(
46+
redis_url="redis://localhost:6379", application_id="my-app", agent_id="my-agent"
47+
)
4448
```
4549

4650
## Import Path

‎python/packages/redis/README.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,47 @@ arrays. Indexed strings cannot contain surrounding whitespace, NUL, or U+001F,
7979
and must fit Redis's 4096-byte TAG limit; these restrictions do not apply to
8080
unindexed payloads. Unsupported operations raise an error.
8181

82+
## Isolate conversation history
83+
84+
`RedisHistoryProvider` uses scoped keys by default. Supply a stable
85+
`application_id`; also supply `tenant_id` and `agent_id` whenever those
86+
boundaries exist in your application. The provider's `source_id` and each
87+
non-empty session ID are included automatically:
88+
89+
```python
90+
from agent_framework.redis import RedisHistoryProvider
91+
92+
history_provider = RedisHistoryProvider(
93+
redis_url="redis://localhost:6379",
94+
application_id="support-app",
95+
tenant_id="contoso",
96+
agent_id="triage-agent",
97+
)
98+
```
99+
100+
Scoped mode rejects missing application or session identifiers rather than
101+
placing unrelated conversations under a shared fallback key. Identifiers are
102+
encoded independently, so they do not need to be globally unique across
103+
tenants, applications, agents, and provider sources.
104+
105+
Releases that predate scoped keys used
106+
`{key_prefix}:{session_id or "default"}`. Existing deployments can temporarily
107+
retain that exact format by opting in explicitly:
108+
109+
```python
110+
legacy_history_provider = RedisHistoryProvider(
111+
redis_url="redis://localhost:6379",
112+
key_format="legacy",
113+
)
114+
```
115+
116+
Legacy mode does not accept scoped identifiers. Scoped mode never reads,
117+
rewrites, or deletes legacy keys. To migrate existing history, copy only the
118+
records belonging to a verified application, tenant, agent, provider source,
119+
and session into the corresponding scoped key using an application-owned
120+
migration process. After verifying the copied history, remove legacy keys
121+
separately according to the application's retention policy.
122+
82123
## Store and search documents
83124

84125
This example uses precomputed vectors, so no embedding service is needed.

‎python/packages/redis/agent_framework_redis/_history_provider.py‎

Lines changed: 92 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,14 @@
99
from __future__ import annotations
1010

1111
import json
12+
import warnings
1213
from collections.abc import Awaitable, Sequence
1314
from inspect import isawaitable
14-
from typing import Any, ClassVar, TypeVar, cast
15+
from typing import Any, ClassVar, Literal, TypeVar, cast
1516

1617
import redis.asyncio as redis
1718
from agent_framework import Message
19+
from agent_framework._filesystem import _storage_key_segment # pyright: ignore[reportPrivateUsage]
1820
from agent_framework._sessions import HistoryProvider, filter_new_messages
1921
from agent_framework._telemetry import mark_feature_used
2022
from redis.credentials import CredentialProvider
@@ -43,10 +45,17 @@ class RedisHistoryProvider(HistoryProvider):
4345
"""Redis-backed history provider using the new HistoryProvider hooks pattern.
4446
4547
Stores conversation history in Redis Lists, with each session isolated by a
46-
unique Redis key.
48+
key scoped to the application, optional tenant and agent, and provider source.
4749
"""
4850

4951
DEFAULT_SOURCE_ID: ClassVar[str] = "redis_memory"
52+
_ENCODED_KEY_PREFIX: ClassVar[str] = "~redis-key-prefix-"
53+
_ENCODED_TENANT_PREFIX: ClassVar[str] = "~redis-tenant-"
54+
_ENCODED_APPLICATION_PREFIX: ClassVar[str] = "~redis-application-"
55+
_ENCODED_AGENT_PREFIX: ClassVar[str] = "~redis-agent-"
56+
_ENCODED_SOURCE_PREFIX: ClassVar[str] = "~redis-source-"
57+
_ENCODED_SESSION_PREFIX: ClassVar[str] = "~redis-session-"
58+
_ABSENT_SCOPE_SEGMENT: ClassVar[str] = "~none"
5059

5160
def __init__(
5261
self,
@@ -59,6 +68,10 @@ def __init__(
5968
username: str | None = None,
6069
*,
6170
key_prefix: str = "chat_messages",
71+
tenant_id: str | None = None,
72+
application_id: str | None = None,
73+
agent_id: str | None = None,
74+
key_format: Literal["scoped", "legacy"] = "scoped",
6275
max_messages: int | None = None,
6376
load_messages: bool = True,
6477
store_outputs: bool = True,
@@ -78,7 +91,17 @@ def __init__(
7891
port: Redis port number. Defaults to 6380 (Azure Redis SSL port).
7992
ssl: Enable SSL/TLS connection. Defaults to True.
8093
username: Redis username.
81-
key_prefix: Prefix for Redis keys. Defaults to 'chat_messages'.
94+
key_prefix: Base prefix for Redis keys. Scoped mode appends independently encoded
95+
tenant, application, agent, provider source, and session segments.
96+
Defaults to 'chat_messages'.
97+
tenant_id: Optional tenant identifier used as an independent key boundary.
98+
application_id: Application identifier used as a required key boundary in scoped mode.
99+
agent_id: Optional agent identifier used as an independent key boundary.
100+
key_format: Redis key format. ``"scoped"`` isolates history by tenant,
101+
application, agent, provider source, and session. ``"legacy"``
102+
preserves the historical ``{key_prefix}:{session_id}`` format for
103+
explicit migration compatibility. Scoped identifiers cannot be
104+
supplied in legacy mode. Defaults to ``"scoped"``.
82105
max_messages: Maximum number of messages to retain per session.
83106
When exceeded, oldest messages are automatically trimmed.
84107
None means unlimited storage; 0 retains nothing, and no message
@@ -95,6 +118,7 @@ def __init__(
95118
ValueError: If both redis_url and credential_provider are provided.
96119
ValueError: If credential_provider is used without host parameter.
97120
ValueError: If max_messages is negative.
121+
ValueError: If key_format or its scoped identifiers are invalid.
98122
"""
99123
super().__init__(
100124
source_id,
@@ -113,8 +137,30 @@ def __init__(
113137
raise ValueError("host is required when using credential_provider")
114138
if max_messages is not None and max_messages < 0:
115139
raise ValueError("max_messages must be None (unlimited) or a non-negative integer")
140+
if key_format not in ("scoped", "legacy"):
141+
raise ValueError("key_format must be 'scoped' or 'legacy'")
142+
if key_format == "scoped":
143+
if not application_id:
144+
raise ValueError("application_id must be a non-empty string when key_format='scoped'")
145+
if tenant_id == "":
146+
raise ValueError("tenant_id must be non-empty when supplied")
147+
if agent_id == "":
148+
raise ValueError("agent_id must be non-empty when supplied")
149+
elif any(scope is not None for scope in (tenant_id, application_id, agent_id)):
150+
raise ValueError("tenant_id, application_id, and agent_id cannot be used with key_format='legacy'")
151+
if key_format == "legacy":
152+
warnings.warn(
153+
"key_format='legacy' is deprecated and will be removed in a future version. "
154+
"Migrate persisted history to scoped keys and use key_format='scoped'.",
155+
DeprecationWarning,
156+
stacklevel=2,
157+
)
116158

117159
self.key_prefix = key_prefix
160+
self.tenant_id = tenant_id
161+
self.application_id = application_id
162+
self.agent_id = agent_id
163+
self.key_format = key_format
118164
self.max_messages = max_messages
119165
self.redis_url = redis_url
120166

@@ -130,9 +176,47 @@ def __init__(
130176
else:
131177
self._redis_client = redis.from_url(redis_url, decode_responses=True) # type: ignore[no-untyped-call]
132178

179+
@classmethod
180+
def _optional_scope_segment(cls, value: str | None, *, encoded_prefix: str) -> str:
181+
"""Encode an optional isolation boundary without conflating absence with a caller value."""
182+
if value is None:
183+
return cls._ABSENT_SCOPE_SEGMENT
184+
return _storage_key_segment(value, encoded_prefix=encoded_prefix)
185+
133186
def _redis_key(self, session_id: str | None) -> str:
134-
"""Get the Redis key for a given session's messages."""
135-
return f"{self.key_prefix}:{session_id or 'default'}"
187+
"""Get a pipe-delimited scoped key or the historical colon-delimited legacy key."""
188+
if self.key_format == "legacy":
189+
return f"{self.key_prefix}:{session_id or 'default'}"
190+
if not session_id:
191+
raise ValueError("session_id must be a non-empty string when key_format='scoped'")
192+
193+
key_prefix_segment = _storage_key_segment(
194+
self.key_prefix,
195+
encoded_prefix=self._ENCODED_KEY_PREFIX,
196+
)
197+
tenant_segment = self._optional_scope_segment(
198+
self.tenant_id,
199+
encoded_prefix=self._ENCODED_TENANT_PREFIX,
200+
)
201+
agent_segment = self._optional_scope_segment(
202+
self.agent_id,
203+
encoded_prefix=self._ENCODED_AGENT_PREFIX,
204+
)
205+
application_segment = _storage_key_segment(
206+
cast("str", self.application_id),
207+
encoded_prefix=self._ENCODED_APPLICATION_PREFIX,
208+
)
209+
source_segment = _storage_key_segment(self.source_id, encoded_prefix=self._ENCODED_SOURCE_PREFIX)
210+
session_segment = _storage_key_segment(session_id, encoded_prefix=self._ENCODED_SESSION_PREFIX)
211+
return "|".join((
212+
key_prefix_segment,
213+
"v2",
214+
tenant_segment,
215+
application_segment,
216+
agent_segment,
217+
source_segment,
218+
session_segment,
219+
))
136220

137221
async def get_messages(
138222
self,
@@ -181,19 +265,17 @@ async def save_messages(
181265
**kwargs: Additional arguments (unused).
182266
"""
183267
mark_feature_used(FeatureIndex.REDIS)
268+
key = self._redis_key(session_id)
184269
if not messages:
185270
return
186271

187272
if self.max_messages == 0:
188273
# Retention is disabled. Trimming cannot express this - LTRIM key 0 -1 keeps
189274
# the whole list - so return before serializing: no payload reaches Redis, an
190-
# AOF or a replica. Stored history is deliberately left alone. _redis_key omits
191-
# source_id, so the list can belong to a co-located provider, and removing
192-
# stored history is what clear() is for.
275+
# AOF or a replica. Stored history is deliberately left alone; removing stored
276+
# history is what clear() is for.
193277
return
194278

195-
key = self._redis_key(session_id)
196-
197279
existing_messages = await self.get_messages(session_id, state=state, **kwargs)
198280
new_messages = filter_new_messages(existing_messages, messages)
199281

0 commit comments

Comments
 (0)