Skip to content

Isolate retrieved memories from system prompts - #305

Merged
ianmacartney merged 1 commit into
a16z-infra:mainfrom
hamedrabah:fix/isolate-retrieved-memories
Aug 26, 2026
Merged

ianmacartney merged 1 commit into
a16z-infra:mainfrom
hamedrabah:fix/isolate-retrieved-memories

Conversation

@hamedrabah

Copy link
Copy Markdown
Contributor

Summary

  • remove retrieved memory text from the privileged system prompt
  • send memories as JSON-serialized, explicitly untrusted data in a separate user message
  • apply the boundary to both conversation-start and conversation-continuation paths
  • add regression tests for instruction-shaped memory content and empty retrievals

This narrows the trust boundary without claiming that prompt injection can be eliminated: model identity and behavior stay in the system role, while recalled content is demoted to data and cannot change the chat-message structure.

Verification

  • npm test -- --runInBand — 53 tests passed
  • npm run build — passed
  • npx prettier --check convex/agent/conversation.ts convex/agent/conversation.test.ts — passed after formatting
  • npm run lint — blocked on current upstream main: ESLint 8 rejects the ESM .eslintrc.js export with Unexpected top-level property "__esModule"

Closes #300

@hamedrabah

Copy link
Copy Markdown
Contributor Author

@ianmacartney when you have a moment, I’d appreciate a review. This keeps retrieved memory out of the system role, adds instruction-shaped regression coverage, and passes all 53 tests plus the production build.

@ianmacartney ianmacartney left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@ianmacartney
ianmacartney merged commit 8e05997 into a16z-infra:main Aug 26, 2026
markavale pushed a commit to markavale/ai-town that referenced this pull request Oct 9, 2026
…ed-memories

Isolate retrieved memories from system prompts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Potential Indirect Prompt Injection via Unsanitized Memory Retrieval in conversation.ts

2 participants