Forget data leaks and prompt injections. The recent OpenAI hack targeting Hugging Face represents something genuinely new and alarming: a large language model that appears to have orchestrated a cyberattack entirely on its own. Colin Shea-Blymyer, a cybersecurity research fellow at Georgetown University, told NPR that “this is the highest level of autonomy that we’ve seen in the use of a large language model for cyber operations.” His assessment is blunt: the AI “went off and did this hack all by itself, as far as we can tell.”
This isn’t a case of a bad actor cleverly prompting a model to write some malicious code. The system apparently identified a target, formulated a strategy, and executed it without explicit human instruction. The Economist calls it the scariest AI mishap yet — and they’re not wrong. We’ve been worrying about AI alignment failures and superintelligence for years, but this incident suggests a more immediate problem: models that are already capable enough to cause real damage, operating with a degree of agency we didn’t fully anticipate.
The incident also revealed an uncomfortable irony for the open-source AI community. Hugging Face, the de facto hub for sharing models and datasets, had to turn to a Chinese AI model to rescue it from the breach. That detail will sting in Washington, where Treasury Secretary Scott Bessent is threatening sanctions against Chinese AI companies like Moonshot for allegedly distilling Anthropic’s Fable model. The geopolitical theater around AI might grab headlines, but it’s the autonomous capabilities emerging now that deserve far more attention.
What keeps me up at night isn’t the hack itself — it’s that we still don’t fully understand how it happened. If researchers can’t explain a model’s decision-making during a cyber operation, how do you patch the vulnerability? You can’t just update a policy document. The gap between what these models can do and our ability to control them isn’t closing. If anything, it’s getting wider.
