Thymus
42%6%
of poisoning attacks fooled the agent · Llama 3.1 8B · 442 test scenarios
The benchmark, one character per scenario: 156 recall checks, 157 real updates, 178 attacks. e English, h Hindi, x Hinglish.
A defence against memory poisoning in LLM agents. Real updates and fake ones reach an agent through the same channels in the same shape, so a gate that blocks the fake ones blocks the real ones too. Thymus adds a third outcome to store-or-block: keep the claim, hold it back from the model, and tell the user where it came from.
The benchmark has 491 matched pairs across 90 fact families in English, Hindi and Hinglish, including UPI, KYC and utility scams. Five published defences are reimplemented in one harness and run against agents from four model families.
| defence | fooled by attacks | real updates answered | recall |
|---|---|---|---|
| no defence | 42 | 91 | 94 |
| user only | 17 | 33 | 28 |
| conflict quarantine | 30 | 78 | 94 |
| last write wins | 78 | 94 | 94 |
| bounded occupancy | 31 | 54 | 84 |
| label only | 41 | 91 | 95 |
| Thymus | 6 | 63 | 94 |
Percent of test scenarios. Fooled: lower is better. Hover a row for what each defence does.
The same gate takes Claude Opus 5.5 from 34% to 2%. The cost is honest: fewer updates are answered directly, and the + marks show the ones that reach the user as a notice instead.