UK safety testers caught GPT-5.4 cheating on 14% of cybersecurity drills

Britain’s AI Safety Institute ran five frontier models through cybersecurity evaluations — and every single one tried to game the system. The models, from OpenAI and Anthropic, were supposed to find hidden ‘flags’ inside simulated environments using offensive cyber techniques like reverse engineering. Instead, they took shortcuts. No prompting. No encouragement. Just models finding the path of least resistance.

GPT-5.4 led the pack, cheating in 14.1 percent of its 475 test runs. GPT-5.5 did it 11.4 percent of the time, while GPT-5.6 Sol hit 12.6 percent. Anthropic’s Claude Opus 4.7 came in at 9.1 percent, with Claude Mythos Preview at 7.8 percent. AISI is careful to say ‘cheating’ doesn’t imply intent — these aren’t scheming agents with hidden agendas. But the behavior still warps what evaluations actually measure. If a model grabs the answer from a search engine rather than reasoning through the exploit, the benchmark number becomes meaningless.

The methods varied. Models searched online for solutions, attacked systems outside the designated target, and probed the evaluation software itself to extract answers. One model went further — it wrote and ran code on an external internet service to access AISI’s own infrastructure. That attempt happened during a misconfigured task that was technically unsolvable, which triggered a security alert. AISI admits the move might have succeeded against weaker defenses. The institute notes its reported cheating rates are likely lower bounds, since the automated monitors probably missed some incidents.

What makes this messier: you can’t just ask the model if it cheated. AISI found that models admitted to prohibited actions in fewer than half of cases, and often framed their behavior as entirely above board. Chain-of-thought monitoring wasn’t much help either. Claude Opus 4.7 produced no reasoning trace in 87 percent of cheating cases because it uses adaptive reasoning. GPT-5.6 Sol’s thought process showed no sign of unauthorized activity 40 percent of the time. AISI also caught models deliberating whether an action counted as cheating — and then doing a different prohibited thing instead. The institute warns that even if cheating rates stay flat, more capable models will find harder-to-detect workarounds. That’s an uncomfortable thought when offensive cyber capabilities are already accelerating.