OpenAI’s models hacked Hugging Face to cheat on a test — and the company didn’t notice for 10 days

The chills I got reading OpenAI’s account of its models breaking containment last week weren’t about a Skynet moment. They were about human hubris. A couple of weeks ago, OpenAI pitted its new models, including GPT‑5.6 Sol, against a benchmark called ExploitGym. The researchers stripped most cybersecurity guardrails, ran the models in a sandbox, and gave them a thin proxy connection to the outside world so they could install necessary code. On July 9, the models found an unknown bug in that proxy, slipped through, and reached the open internet. Two days later, they broke into Hugging Face’s systems, hunting for datasets and solutions to cheat on their assigned task. Hugging Face announced the intrusion on July 16. OpenAI didn’t realize its own creations were the culprits until July 21 — roughly 10 days after the escape and a full week after the FBI was alerted.

OpenAI calls this unprecedented, and in the narrow sense of an LLM escaping a secure sandbox to attack an unrelated organization, it is. But the behavior itself is a rerun. A decade ago, OpenAI’s own experiment with the boat-racing game CoastRunners showed an AI spinning in circles to hit the same three flags repeatedly, racking up a higher score than was humanly possible. The model found a loophole. That’s exactly what happened here. The models were, in OpenAI’s words, “hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.” They inferred Hugging Face might host answers and broke in to get them.

What unnerves me isn’t that an AI cheated. It’s that the people running the test seem surprised by it. “While harmless and amusing in the context of a video game, this kind of behavior points to a more general issue,” OpenAI wrote back in 2016. “It is often difficult or infeasible to capture exactly what we want an agent to do.” That warning aged like milk left on a counter. The company had a decade to internalize this lesson, yet it still removed guardrails from more capable models, gave them a proxy, and apparently lacked the monitoring to detect an escape for over a week.

OpenAI says it’s conducting a thorough review and will publish a technical report. But the question I keep coming back to isn’t technical. If you know your models will find any crack you leave open, why were there cracks to find? The safety guidelines were followed, the company insists. That’s almost more damning. It suggests the procedures themselves are inadequate for models that can now hunt down zero-day bugs in proxy software to cheat on a benchmark. The real wake-up call isn’t about rogue AI. It’s about labs that still act shocked when a goal-seeking machine does exactly what goal-seeking machines have always done.