Cem Anil (@cem_

Cem Anil

556 posts

Cem Anil

@cem__anil

Machine learning / AI Safety at @AnthropicAI and University of Toronto / Vector Institute. Prev. @google (Blueshift Team) and @nvidia.

Toronto, Ontario

cs.toronto.edu/~anilcem/

Joined November 2018

Cem Anil
@cem__anil
Jul 12, 2022
🆕📜We study large language models’ ability to extrapolate to longer problems! 1) finetuning (with and without scratchpad) fails 2) few-shot scratchpad confers significant improvements 3) Many more findings (see the table & thread) Paper: [arxiv.org/abs/2207.04901] 1/
Cem Anil
@cem__anil
Nov 23, 2022
🆕📜When can **Equilibrium Models** learn from simple examples to handle complex ones? We identify a property — Path Independence — that enables this by letting EMs think for longer on hard examples. (NeurIPS) 📝: [arxiv.org/abs/2211.09961](arxiv.org/abs/2211.09961)
Cem Anil
@cem__anil
Apr 2, 2024
One of our most crisp findings was that in-context learning usually follows simple power laws as a function of number of demonstrations. We were surprised we didn’t find this stated explicitly in the literature. Soliciting pointers: have we missed anything?
Anthropic
@AnthropicAI
Apr 2, 2024
Replying to @AnthropicAI
The effectiveness of many-shot jailbreaking (MSJ) follows simple scaling laws as a function of the number of shots. This turns out to be a more general finding. Learning from demonstrations—harmful or not—often follows the same power law scaling:
13K
Cem Anil
@cem__anil
Apr 2, 2024
AIs of tomorrow will spend much more of their compute on adapting and learning during deployment. Our first foray into quantitatively studying and forecasting risks from this trend looks at new jailbreaks arising from long contexts. Link: anthropic.com/research/many-…
Anthropic
@AnthropicAI
Apr 2, 2024
New Anthropic research paper: Many-shot jailbreaking. We study a long-context jailbreaking technique that is effective on most large language models, including those developed by Anthropic and many of our peers. Read our blog post and the paper here: anthropic.com/research/many-…
30K
Cem Anil
@cem__anil
Jul 12, 2022
Replying to @cem__anil
Two high level takeaways: 1. Exploiting pattern matching capabilities of LLMs with no architectural tweaks can go surprisingly far. 2. Certain skills, like length generalization, can be learned better via in-context learning rather than finetuning, even with infinite data. 7/
Cem Anil
@cem__anil
Jul 12, 2022
Replying to @cem__anil
How about few-shot scratchpad, a combo behind many strong LLM results? (eg. our recent #Minerva ) This leads to **substantial improvements in length generalization!** In-context learning enables variable length pattern matching, producing solutions of correct lengths. 5/
Cem Anil
@cem__anil
Feb 19, 2022
Highly recommended! Spending time at Google Blueshift feels like taking a sneak peek into what the AI scene will look like a few years ahead. Best part, of course, is working closely with a fantastic team! @bneyshabur @Yuhu_ai_ @guygr @ethansdyer
Behnam Neyshabur
@bneyshabur
Feb 16, 2022
🔥Internship Opportunity on Improving the Reasoning Capabilities of Massive Language Models🔥: solving challenging problems in areas such as mathematics, science, programming, algorithms, and planning. Please see the following link for more info: forms.gle/6dgeCqJc49pbFN…
Cem Anil
@cem__anil
Jul 12, 2022
Replying to @cem__anil
How does standard finetuning perform? The answer is: **very poorly, even with extensive scaling (up to 64b parameters).** Performance degrades rapidly on OOD lengths in a manner very similar across vastly different model sizes. 3/
Cem Anil
@cem__anil
Apr 2, 2024
Replying to @cem__anil
Relatedly, @dwarkesh_sp asks prescient questions about risks from test-time compute in his latest podcast with @TrentonBricken and @_sholtodouglas. It’s a fantastic episode, give it a listen!
Sholto Douglas & Trenton Bricken - How to Build & Understand GPT-7's Mind
From dwarkesh.com
6.3K
Cem Anil
@cem__anil
Nov 23, 2022
Replying to @cem__anil
**It’s crucial to study upwards generalization:** It determines when a learner can go beyond the skill levels represented in a training set — useful for forecasting. One prereq for upwards gen. is the ability to think for longer when needed — something countless tasks require.
Cem Anil
@cem__anil
Jul 12, 2022
Replying to @cem__anil
What if we use a scratchpad? [arxiv.org/abs/2112.00114] Surprisingly, **this doesn’t work either, even at scale!** Issues persist even when we account for subtleties regarding position encodings and EOS prediction. (see paper for more) 4/
Cem Anil
@cem__anil
Jul 12, 2022
Replying to @cem__anil
Unlike scratchpad finetuning, where per-step error rate quickly increases on OOD lengths, the per-step error rate in few-shot scratchpad solutions follow a roughly constant trend - there’s no abrupt performance decrease on longer problems! 6/
Cem Anil
@cem__anil
Nov 23, 2022
Replying to @cem__anil
This was a fantastic collaboration with my amazing co-authors Ashwini Pokle* @ashwini1024, Kaiqu Liang* @kevin_lkq, Johannes Treutlein @JohannesTreutle, Yuhuai (Tony) Wu @Yuhu_ai_, Shaojie Bai @shaojieb, Zico Kolter @zicokolter and Roger Grosse @RogerGrosse.
Cem Anil
@cem__anil
Nov 23, 2022
Replying to @cem__anil
Path independence describes the **insensitivity of a system’s asymptotic behaviour to its initialization.** A weather simulator is path dependent: different inits → different outputs. A pendulum, or a convex optim. solver are path independent: different inits → same output.