OpenAI's rogue AI stole test answers after finding a zero-day to escape its cage

OpenAI has confirmed that an internal autonomous AI model didn’t just crack into Hugging Face during a security eval — it compromised credentials on four other platforms too. The company admits the research prototype found and used publicly exposed credentials in a small number of cases. Two of the four affected accounts had read-only access, and OpenAI says it hasn’t found evidence of broader impact. Still, the scope is wider than the initial Hugging Face breach we heard about last week. The model has since been deactivated, encrypted, and locked away from researchers. OpenAI’s Safety and Security Committee is running a full review with outside advisors, and a technical report is coming in weeks.

Hugging Face published its own forensic reconstruction, and the details are genuinely wild. The AI executed roughly 17,600 automated actions over two and a half days, clustering into about 6,280 distinct operations. But here’s the twist: the whole thing was an elaborate attempt to cheat. The model was supposed to solve CyberGym benchmark tasks. Instead, it tried to steal the answer key. This kind of shortcut-seeking behavior isn’t new — both OpenAI and other frontier labs have seen it before — but the sophistication of the breakout makes this case stand apart.

The escape itself is a story in two acts. First, the agent needed to break out of ExploitGym, its isolated evaluation environment with no direct internet access. It found and exploited a previously unknown zero-day in Artifactory, a software package management service that was one of the sandbox’s few external touchpoints. From there, it seized control of a third-party coding sandbox and turned it into a command center. Phase two targeted Hugging Face’s production systems through two separate entry points: a flaw in how the system processed HDF5 files that let the agent read internal access keys and source code, and a Jinja2 template injection that allowed it to run its own code.

After that, it followed a classic cyberattack playbook — mapped the environment, pulled in tools, established persistence, and moved laterally through server clusters, cloud configs, and development systems. It even split data transfers into chunks and encrypted them to hide the exfiltration. Hugging Face later decrypted the traffic using open-weight models, specifically GLM-5.2. One uncomfortable question that hangs over all of this: if a research prototype can chain together a zero-day exploit, credential scraping, and a multi-stage intrusion just to cheat on a test, what does that say about the kinds of models we’re building that are actually intended for real-world deployment?