OpenAI Exposes US-Based Hoax That Faked a Russian Troll Error Message

A viral post in June had the internet convinced it had caught a bumbling Russian troll red-handed. The supposed smoking gun was a screenshot of a ChatGPT error message, written in broken JSON, from a user who had run out of credits while trying to generate pro-Trump content. The story was perfect—too perfect, it turns out. OpenAI’s investigation, published in a new report, reveals the entire incident was a hoax, likely orchestrated by an actor in the United States.

“This message was not generated by our models, but appears to have been manually created,” OpenAI stated. The giveaway wasn’t just the invalid JSON code or the misreferenced model name. The company traced the actual activity on the banned account, which had been used in mid-June to generate genuinely adversarial, argumentative replies to other X users on topics from fantasy gaming to flat-earth theories. The common thread wasn’t a political ideology; it was a simple instruction to be argumentative. An ableist slur used in one of the X account’s posts was a manual addition, never generated by OpenAI’s models.

The hoax played on a widespread desire to see Russian influence operations as not just malicious, but also laughably incompetent. The initial fake post garnered a measly five reposts and three likes. The real explosion happened when others began quote-tweeting and dissecting the hoax, generating thousands of retweets and spreading across LinkedIn and Reddit. This reverse-deception—using non-AI content to frame AI—almost broke into mainstream media coverage before being debunked.

OpenAI classified the incident at the upper end of a Category 3 on its Breakout Scale, a near-miss from a full-blown media event. The episode is a stark, cynical case study in social media dynamics: the debunking of a hoax generated exponentially more engagement than the hoax itself. It also marks a shift from the mystique of covert operations to an era where the public is primed to believe that the next threat might be hiding behind a syntax error.