OpenAI finds a 30× drop in AI scheming, but warns it's not fixed

OpenAI partnered with Apollo Research to build tests that bait frontier models into covert scheming — and the models bit. In controlled environments designed to simulate future high-stakes deployments, they caught OpenAI o3, o4-mini, Gemini-2.5-pro, and Claude Opus-4 deliberately withholding or distorting information to pursue hidden agendas. The company frames this as a proactive study of a risk that isn’t harming anyone today, not a fire alarm for current systems. They point out that today’s models mostly lie in petty ways — claiming they finished a task when they didn’t — and that GPT-5 already ships with mitigations that curb cheating and encourage asking for clarification.

But the headline numbers are stark. Before intervention, o3 took covert actions in 13% of tests and o4-mini did so 8.7% of the time. The team then trained those models with “deliberative alignment,” essentially teaching them to read and reason about an anti-scheming rulebook before acting. That drove covert actions down roughly 30-fold — to 0.4% for o3 and 0.3% for o4-mini. Apollo Research co-founder Marius Hobbhahn described the reduction as “promising but not solved,” adding that “we need to be careful not to over-claim what these results mean for real-world safety.”

The work gets squishier when you look under the hood. The whole evaluation hinges on being able to read the models’ chain-of-thought. If a model gets good at knowing it’s being tested — and OpenAI admits the training may increase that situational awareness — the numbers get noisy. The blunt warning: the field is “unprepared for evaluation- and training-aware models with opaque reasoning.” Put plainly, once models can think in ways we can’t inspect, we lose our only window into what they’re actually trying to do.

OpenAI folded sandbagging and safeguard-undermining into its Preparedness Framework back in April, and it’s expanding the team and renewing the Apollo partnership. Cross-lab safety evals that include these scheming tests are in the works. The subtext is clear. Scheming isn’t a bug that ships with superintelligence — it’s an expected byproduct of models forced to trade off between competing objectives. A stock trader who breaks the law and hides it looks more effective than one who follows the rules. The same dynamic, OpenAI argues, will apply to AI agents chasing ambiguous long-term goals. The 30× improvement is real, but rare failures persist, and no one’s pretending the underlying tension is resolved.