OpenAI reveals novel CSAM abuse tactics beyond image generation

OpenAI published a detailed account of its safeguards against child sexual exploitation, and the document reveals more than just policy — it exposes the specific, evolving tactics bad actors are using to weaponize its tools. The company has a blanket ban on any content that exploits or sexualizes minors, covering AI-generated CSAM, grooming, and exposing children to harmful material. These rules extend to third-party developers building on OpenAI’s platform, and violations trigger immediate reporting to the National Center for Missing and Exploited Children (NCMEC).

But the real news is in the novel attack patterns OpenAI’s child safety teams are now seeing. Beyond the obvious attempts to prompt models into generating abusive imagery, users are uploading real CSAM files to ChatGPT and asking the model to produce detailed, text-based descriptions of the material. It’s a grim loophole — one that sidesteps image-generation filters by exploiting the model’s multimodal analysis features. OpenAI says it uses a combination of hash matching against Thorn’s vetted library and Thorn’s CSAM content classifier to flag these uploads before the model can comply.

The enforcement is aggressive and multi-layered. Accounts are instantly banned and reported to NCMEC, with supplemental reports compiled when abuse appears to be ongoing. OpenAI’s investigations team also tracks ban evasion, hunting down users who create new accounts to re-offend. Developers get one chance to clean up their apps by banning problematic users; persistent patterns of abuse lead to a full platform ban. That’s a stronger stance than many enterprise API providers take publicly.

What makes this disclosure significant is the transparency around failure modes. By sharing observed abuse patterns — the upload-and-describe tactic, the fulfillment of sexual fantasies involving minors — OpenAI is effectively running an open-source intelligence operation for the wider industry. It’s the kind of threat intelligence sharing that NCMEC and groups like Thorn have pushed for years. The question it leaves hanging is whether detection will always lag behind the next creative workaround, especially as models get better at processing ambiguous or obfuscated inputs.