Sabri Eyuboglu (@EyubogluSabri) / X

Pinned

Most times I use an agent it spends minutes rereading all my stuff – spending dollars and energy just to gather the context it needs to understand my work. But it won’t always be this way. We can train that understanding into the model – amortizing across tasks. We started

Introducing Engram: Scaling compute on your context

We’re Engram. We’re building AI that learns from you and deeply understands your work. Today’s AI models don’t understand what you do. Not really. Everything models know comes from their training –...

Do you ever wonder if your model - despite logging impressive accuracy - is still failing on an important but unknown slice of your dataset? We certainly do! Stoked to share recent work @iclr22 in which we develop & evaluate ~slice discovery methods~ (1/7)

Discovering the systematic errors made by machine learning models

When we put lots of text (eg a code repo) into LLM context, cost soars b/c of the KV cache’s size. What if we trained a smaller KV cache for our documents offline? Using a test-time training recipe we call self-study, we find that this can reduce cache memory on avg 39x

Curious whether sub-quadratic LMs like RWKV and Hyena will replace Transformers? We find that Transformers are still much better at associative recall (AR): a simple task known to be essential for in-context learning. hazyresearch.stanford.edu/blog/2023-12-1… github.com/HazyResearch/z…

Stoked to be sharing Based! We find that the simple combo of linear and sliding window attention can enable 24x higher throughput than Transformers. Had a ton of fun diving deep on the tradeoffs that govern these recurrent models! arxiv.org/abs/2402.18668 github.com/HazyResearch/b…

GIF

Simran Arora

@simran_s_arora

Mar 4, 2024

Excited to release Based, an architecture that combines two✌️ simple, familiar, attention-like primitives – short (size-64) sliding window attention and softmax-approximating linear attention – to enable high quality and efficient inference! 💨 🚀 joint w/ @EyubogluSabri,

(1/7) Our work, which explores how weak supervision and multi-task learning can together address some of the challenges that arise when applying ML to whole-body PET/CT, is out today in @NatureComms! Code: github.com/seyuboglu/weak… Paper: nature.com/articles/s4146…

GitHub - seyuboglu/weakly-supervised-petct: PyTorch implementation of a multi-task, weak supervis...

Replying to @EyubogluSabri

Blog: hazyresearch.stanford.edu/blog/2025-06-0… Github: github.com/HazyResearch/c…

Foundation models are amazing for human-in-the-loop tasks. But, we're also excited to apply them to batch computing tasks where we can't check every output and require high throughput. We wrote about some opportunities and research in the space:

hazyresearch.stanford.edu

Batch computing and the coming age of AI systems

Thanks @willccbb!! For those at ICML, I'm giving a talk on Cartridges at the ES-FoMo workshop on Saturday at 10:45 -- come through!! Excited to talk memory, test-time training, and continual learning!

will brown

@willccbb

Jul 17, 2025

cant stop thinking about this one insanely elegant, seems insanely powerful

All these on-device models are coming out (e.g. llama 3.2). But how can we actually make them useful for hard reasoning workloads (beyond iMessage summarization)? Our idea: give the on-device models your long context and let them communicate with frontier models in the cloud.

Dan Biderman

@dan_biderman

Feb 25, 2025

How can we use small LLMs to shift more AI workloads onto our laptops and phones? In our paper and open-source code, we pair on-device LLMs (@ollama) with frontier LLMs in the cloud (@OpenAI, @Together), to solve token-intensive workloads on your 💻 at 17.5% of the cloud cost

00:00

I'll be at #ICML in Vancouver next week -- looking forward to meeting new folks. Shoot me an email if you'll be there and want to chat!! These days, I'm particularly interested in LLM memory, personalization, and lifelong learning -- but excited to learn about anything!

We built something to help you wrangle machine learning datasets and model artifacts! Excited to share – check out our blogpost ⬇️⬇️⬇️ 💻GitHub: github.com/robustness-gym… 📄Blogpost: notion.so/Meerkat-DataPa… @krandiash @jundesai @HazyResearch

GitHub - HazyResearch/meerkat: Explore and understand your training and validation data.

Replying to @EyubogluSabri

Here’s the main idea: instead of creating a KV cache by running a single forward pass on the context, we train a smaller KV cache - which we call a cartridge - with gradient descent by back-propagating loss into the key and value vectors (equivalent to prefix tuning). Users

Replying to @EyubogluSabri

Serving different long contexts for many users is slow and expensive. This is largely due to enormous per-user KV caches. There are techniques that reduce this memory consumption by making architectural modifications (e.g. linear attention) or applying KV cache compression

3.4K