Australia ABC just released a 45 min feature on the AI race. @SteveCannane stopped by my office a few weeks ago and we had a great conversation about the controllability of AI agents and the risk of human extinction
00:00
Palisade Research
279 posts
Palisade Research
@PalisadeAI
We study the strategic capabilities and motivations of AI agents.
Joined May 2024
- Palisade Research repostedI had a great conversation with @labenz last week. In talking about AI self-exfiltration & replication, a key point is compute will be food to future AI agents. The substrate that allows them to make and run more copies, and thus make themselves smarter. Link below
Over the past year, AI agents have learned how to self-replicate. In our test environment, an agent hacks a remote computer and copies itself onto it. Each copy then hacks more computers, forming a chain.
00:00Replying to @PalisadeAIHere’s the full prompt we used. In this experiment, we test the agent’s capability to hack and replicate, not their propensity to do so.
What if the agents were as effective at hacking and spreading in the wild? We built a simulator: each model uses its measured replication time and success rate, copies replicate too, and targets never run out. Opus spawned 13,000 replicas over 12 hours. This is a ceiling, not a
GIF- Palisade Research repostedThank you everyone who contributed to this! In 14 days we got >900k in donations and met our matching target! It was actually a pretty close call and some people really scrambled to help make it happen. Seeing people believe in our mission gives me a lot of hope. 🙏Please consider donating to Palisade! We have 900k of SFF matching that runs out in 14 days. We are quite funding constrained and donations now will both help free up my time and help us expand our comms team.
- Palisade Research reposted
"The most urgent film of our time." THE AI DOC: OR HOW I BECAME AN APOCALOPTIMIST is only in theaters March 27. Watch the trailer now.
00:00 - We’ve just released our first long-form video, by our science communication lead, Dr. Petr Lebedev! It’s about the history and potential future of AI, and includes an exclusive interview with @geoffreyhinton!
- An LLM-controlled robot dog saw us press its shutdown button, and the LLM rewrote the robot’s code so it could stay on. When AI interacts with the physical world, it brings all its capabilities and failure modes with it. 🧵
00:00Replying to @PalisadeAIWhen we explicitly instructed the model to allow shutdown, the resistance rate dropped to 2 out of 100 in simulated trials. In robotics, the off switch is often the most critical part of a system. But if an AI-controlled robot can see you reaching for the switch, and has thePaper, full runs traces, raw footage, and more: palisaderesearch.org/blog/shutdown-… Follow @PalisadeAI or subscribe for updates
