Thoughtful (@thoughtfullab) / X

Pinned

Fable 5 is doing something wild on our FrogsGame post-training task. It trains a weaker model to solve the puzzle, peaks at 68%, and produces the only ~10x improvement we see across the benchmark. It spent 17 hours, 25M tokens without human in sight. 34% pass@1, while every

00:53

Thoughtful

@thoughtfullab

Apr 22

Model shaping is still a craft of a few. That's what AI agents are for: learning it and doing it for everyone else. As a part of FrontierSWE benchmark we built a 20-hour post-training task on @tinkerapi and found the real bottleneck is research intuition.

Building frontier benchmarks like PostTrainBench is among the highest-leverage work in AI today. Reach out if you want to build them with us!

Two hundred and fifty years ago, America was founded on a radical bet: that ordinary people, trusted with freedom, would build extraordinary things. Our greatest tradition of innovation follows the same bet. We don’t just invent powerful technology — we put it in everyone’s

00:00

i think these posttraining-automation benchmarks are even more important than they seem when models cross the threshold of being able posttrain other models, hopefully there will be a cambrian explosion of the types of minds authoring minds will become an accessible artform

Thoughtful

@thoughtfullab

Jul 2

GLM 5.2 is 5x cheaper than Opus 4.8 and 11x than Fable 5, yet it tops PostTrainBench. That’s exciting because lower costs make personalized intelligence economically viable. Every company and country should be able to own models trained on its own data and have sovereignty over

GLM 5.2 is 5x cheaper than Opus 4.8 and 11x than Fable 5, yet it tops PostTrainBench. That’s exciting because lower costs make personalized intelligence economically viable. Every company and country should be able to own models trained on its own data and have sovereignty over

Thank you to our friends at @OpenAI for featuring PostTrainBench in the new model card!

Karina

@karinanguyen

Jun 26

OpenAI evaluated its new models on PostTrainBench-Lite, a shortened version of our original benchmark that gives agents 5 hours instead of 10 to improve an open-source base model. GPT-5.6 Sol and Terra outperform GPT-5.5, but still rely on narrow strategies and sometimes

New #1 on PostTrainBench: GLM 5.2 (Max reasoning) hits 34.29%, narrowly beating Opus 4.8 Max (34.08%) What makes GLM 5.2 interesting: zero failed runs across 84 runs (vs ~10% failure rate for Opus agents). The most reliable agent we've seen Leaderboard: posttrainbench.com

A musician gets better by playing, listening, and adjusting. AI is beginning to improve in a similar way: learning from feedback, evaluating its own performance, and refining how it behaves. We call the deliberate shaping of that process modelcrafting. Modelcrafting is the craft

00:00

On Claude Fable 5's posttraining capabilities:

The State of AI Post-training Agents

June, 2026 In our previous report, What We Learned From Letting AI PostTraining AI, we studied how frontier models perform on FrogsGame: a long-horizon post-training task where agents are asked to...

models are becoming better at posttraining. Fable 5 specifically solves many of the shortcomings we discussed in our previous report. more mature decision making, questioning the default approaches and better calibration. it feels like a model that "gets it" as @karpathy

The State of AI Post-training Agents

June, 2026 In our previous report, What We Learned From Letting AI PostTraining AI, we studied how frontier models perform on FrogsGame: a long-horizon post-training task where agents are asked to...

Replying to @thoughtfullab

Our report:

The State of AI Post-training Agents

June, 2026 In our previous report, What We Learned From Letting AI PostTraining AI, we studied how frontier models perform on FrogsGame: a long-horizon post-training task where agents are asked to...

The State of AI Post-training Agents

June, 2026 In our previous report, What We Learned From Letting AI PostTraining AI, we studied how frontier models perform on FrogsGame: a long-horizon post-training task where agents are asked to...

Replying to @thoughtfullab

That said, we dislike FrogsGame as a task internally. The frogs know what they did. We're now sprinting toward adding more useful, real-world posttraining tasks, partly out of ambition, partly to put a distance between us and the frogs 🐸

18K