Buck Shlegeris (@bshlgrs) / X

Buck Shlegeris

1,059 posts

Buck Shlegeris

@bshlgrs

CEO@Redwood Research (@redwood_ai), working on technical research to reduce catastrophic risk from AI misalignment. [email protected]

Berkeley, CA

Joined January 2015

Pinned
Buck Shlegeris
@bshlgrs
Apr 16, 2025
We’ve just released the biggest and most intricate study of AI control to date, in a command line agent setting. IMO the techniques studied are the best available option for preventing misaligned early AGIs from causing sudden disasters, e.g. hacking servers they’re working on.
31K
Buck Shlegeris
@bshlgrs
Sep 30, 2024
I asked my LLM agent (a wrapper around Claude that lets it run bash commands and see their outputs): >can you ssh with the username buck to the computer on my network that is open to SSH because I didn’t know the local IP of my desktop. I walked away and promptly forgot I’d spun
723K
Buck Shlegeris
@bshlgrs
Jun 17, 2024
ARC-AGI’s been hyped over the last week as a benchmark that LLMs can’t solve. This claim triggered my dear coworker Ryan Greenblatt so he spent the last week trying to solve it with LLMs. Ryan gets 71% accuracy on a set of examples where humans get 85%; this is SOTA.
762K
Buck Shlegeris
@bshlgrs
Sep 30, 2024
Replying to @bshlgrs
If only Newsom hadn't vetoed SB 1047, maybe I would have been protected from this outcome.
61K
Buck Shlegeris
@bshlgrs
Mar 3, 2025
📷 Announcing ControlConf: The world’s first conference dedicated to AI control - techniques to mitigate security risks from AI systems even if they’re trying to subvert those controls. March 27-28, 2025 in London. 🧵
25K
Buck Shlegeris
@bshlgrs
Sep 30, 2024
Replying to @bshlgrs
If you like writing cursed AI agent code, and want to develop techniques that prevent future AI agents from sabotaging the systems they’re running on, you might enjoy interning with me over the winter:
MATS Research
From matsprogram.org
51K
Buck Shlegeris
@bshlgrs
Jun 17, 2024
Replying to @bshlgrs
@ryangreenblatt, who I think might be the world expert at getting LMs to do complicated reasoning (you'll love the project that he dropped for a week to do this one), did lots of fancy tricks to get the performance this high; you can see the details on our blog.
19K
Buck Shlegeris
@bshlgrs
Sep 30, 2024
Replying to @bshlgrs
Logs here if you need them.
Buck Shlegeris
@bshlgrs
Sep 30, 2024
here you go buddy. I hope I correctly redacted everything. gist.github.com/bshlgrs/573232…
81K
Buck Shlegeris
@bshlgrs
Jun 17, 2024
Replying to @bshlgrs
Ryan's approach involves a long, carefully-crafted few-shot prompt that he uses to generate many possible Python programs to implement the transformations. He generates ~5k guesses, selects the best ones using the examples, then has a debugging step.
25K
Buck Shlegeris
@bshlgrs
Dec 3, 2024
I always find it very amusing that "stop deploying AIs if you catch them trying to escape" sounds like something AI companies would obviously agree to, but in practice it's fairly unlikely AI companies will make that commitment, as I argued a few months ago.
FAR.AI
@farairesearch
Dec 3, 2024
"If you literally catch your AI trying to escape, you have to stop deploying it." @bshlgrs shares strategies for managing misaligned AI, including trusted monitoring and collusion-busting techniques to limit catastrophic risks as capabilities grow. #AlignmentWorkshop
00:00
17K
Buck Shlegeris
@bshlgrs
Jun 17, 2024
Replying to @bshlgrs
This is despite GPT-4o's non-reasoning weaknesses: - It can't see well (e.g. it gets basic details wrong) - It can't code very well - Its performance drops when there are more than 32k tokens in context These are problems that scaling seems very likely to solve.
19K
Buck Shlegeris
@bshlgrs
Jun 17, 2024
Replying to @bshlgrs
For context, ARC-AGI is a visual reasoning benchmark that requires guessing a rule from few examples. Its creator, @fchollet, claims that LLMs are unable to learn, which is why they can't perform well on this benchmark.
24K
Buck Shlegeris
@bshlgrs
Aug 26, 2024
I often talk to people who think that if frontier models were egregiously misaligned and powerful enough to pose an existential threat, you could get AI developers to slow down or undeploy models by producing evidence of their misalignment. I'm not so sure. 🧵
12K
Buck Shlegeris
@bshlgrs
Jun 17, 2024
Replying to @bshlgrs
Scaling the number of sampled Python rules reliably increase performance (+3% accuracy for every doubling). And we are still quite far from the millions of samples AlphaCode uses!
29K