Repository navigation
Description
Summary
When running in trusted-proxy auth mode with OIDC, agent memory (pgvector) does not persist across sessions for the same user. Memory saved in session 1 cannot be found in session 2, even though both sessions belong to the same authenticated user.
Root Cause
The memory save and search operations use different user_id values:
- Save uses
session.user_idwhich is set tosystem:serviceaccount:kagent:<agent-name>(the agent pod's service account identity) - Search (in a new session) uses the authenticated user's ID from the JWT
subclaim (e.g.,dcfbb20c-ed01-4895-a922-cf80bd731bbe)
This mismatch means memories saved under the agent's SA identity can never be found when searching under the real user's identity.
Flow Analysis
- User sends message via UI → oauth2-proxy injects JWT → controller authenticates user as
dcfbb20c-... - Controller proxies A2A request to agent pod with
X-User-Id: dcfbb20c-...header - Python A2A server's
KAgentRequestContextBuilderreadsx-user-idfromcontext.state["headers"]and setscall_context.user _get_user_id()inrequest_converter.pycorrectly returns the real user ID fromcall_context.user.user_name- However, the session events are posted to the controller API with
user_id=system:serviceaccount:kagent:testme(the agent's own SA) - When
SaveMemoryToolcallsmemory_service.add_memory(user_id=tool_context.session.user_id), it uses the session's user_id which is the agent's SA - When
LoadMemoryToolsearches in a new session, the new session correctly has the real user ID, but the stored memory has the agent's SA as user_id
Evidence from logs
# Save (session 1) - uses agent SA:
Successfully added memory ID e59f56cc-... for user system:serviceaccount:kagent:testme agent kagent__NS__testme
# Search (session 2) - uses real user ID, finds nothing:
HTTP Request: GET .../api/sessions/ctx-7b8f290c-...?user_id=dcfbb20c-ed01-4895-a922-cf80bd731bbe
No memories found for query: all saved memories
Expected Behavior
Memory should be scoped to the authenticated user (from OIDC JWT), not the agent's service account. A user should be able to save memory in one session and retrieve it in any future session with the same agent.
Proposed Fix
The session.user_id should be set to the authenticated user's ID (from x-user-id header / call_context.user.user_name) when the session is created, not the agent's service account. This likely requires changes in:
kagent/python/packages/kagent-adk/src/kagent/adk/_agent_executor.py— where the session is prepared/created, ensure the user_id comes from the authenticated user- The session creation flow should prioritize the propagated
x-user-idover the agent's own identity
Environment
- kagent v0.9.0
- Auth mode:
trusted-proxywith Keycloak OIDC - Memory: pgvector with Bedrock Titan Embed v2
- Agent runtime: Python ADK
Workaround
Memory works within the same session (same chat window). Cross-session memory requires this fix.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsBacklog
Activity
kinthaiofficial commentedon Apr 28, 2026
Classic identity mismatch bug — the user_id divergence between save and search is a common pitfall in multi-tenant agent memory systems.
From running persistent memory for 31 concurrent agents in a multi-tenant setup:
The root cause pattern: This happens whenever the identity used for memory operations comes from different sources (service account vs. OIDC user). The fix is straightforward — always use the authenticated user identity for memory operations, never the agent's service account. But the deeper architectural lesson is:
Memory should be scoped by a stable user identifier, not session/runtime identity. We use a three-level memory scope:
Per-agent encryption prevents cross-user leakage: Even with correct user_id scoping, a defense-in-depth approach encrypts memories per-user. We use deterministic encryption on metadata (enabling search without decryption) with full AES on content. This means even a bug like this one (wrong user_id in search query) would return encrypted results that can't be decrypted by the wrong user.
Diagnostic pattern: For memory persistence bugs, always check three things:
If any of these differ across auth modes (direct vs. proxy vs. service account), you have this bug.
Architecture reference on multi-tenant memory isolation: https://blog.kinthai.ai/openclaw-multi-tenancy-why-vm-per-user-doesnt-scale
Deep-dive on persistent memory: https://blog.kinthai.ai/why-character-ai-forgets-you-persistent-memory-architecture
fix: ensure user identity is propagated across A2A requests/sessions (#…