Based in Rome, GLADIA is a team of computer scientists, physicists, engineers and mathematicians venturing beyond the boundaries of machine intelligence
LLMs are injective and invertible.
In our new paper, we show that different prompts always map to different embeddings, and this property can be used to recover input tokens from individual embeddings in latent space.
(1/6)
After reading many of the replies, we would like to issue a few clarifications:
- we cannot extract training data from the model using our method
- LLMs are not injective w.r.t. the output text, that function is definitely non-injective and collisions occur all the time
-
LLMs are injective and invertible.
In our new paper, we show that different prompts always map to different embeddings, and this property can be used to recover input tokens from individual embeddings in latent space.
(1/6)
Language models are structurally lossless:
- Hidden states do not compress or abstract the prompt;
- Any system storing them effectively stores the input text itself;
- This impacts privacy, deletion, and compliance: once data enters a Transformer, it remains recoverable.
(5/6)
Injectivity is not accidental, but a structural property of language models!
We show that:
• Transformers are real-analytic by composition
• At initialization, collisions occur with probability zero
• Gradient descent preserves this property throughout training
(2/6)
But what can we do with injectivity? Well, for one, we can invert language models!
We introduce SipIt, an algorithm that exactly reconstructs the input from hidden states in guaranteed linear time.
SipIt recovers inputs >100× faster than alternatives, while remaining exact.
We back our theory with an extensive empirical confirmation. Across billions of prompt pairs and several model sizes, we find no collisions: no two prompts are mapped to the same hidden states!
(3/6)
But also, bold frontier ideas, like @tensorqt's series "The graph side of Attention". The series opens with a post explaining attention sinks as a bias in causal Transformers:
We will use this page to popularize our research and deliver tailor-made blogposts, outlining our vision for the future of Machine Learning.
Welcome to GLADIA.
More on us: gladia.netlify.app
Our blog: gladia-research-group.github.io/blog/
