How OpenAI sterilized DALL·E 2's training data

OpenAI is finally showing its cards on how they tamed DALL·E 2 before unleashing it on the public. The pre-training mitigations they’ve outlined are a fascinating look at the tension between creative power and catastrophic potential. The core approach is brutally simple: if you don’t want a model to generate something, don’t let it learn it in the first place. They did this by aggressively filtering the model’s diet of hundreds of millions of captioned internet images, specifically targeting graphic violence and sexual content using in-house classifiers.

But filtering created a cascading set of second-order problems. The team noticed that simply purging explicit content inadvertently skewed the model’s demographic outputs—for instance, making it more likely to generate images of men and less likely to generate women. This reveals a painful technical reality: datasets have a fragile ecology, and removing one toxic element can amplify a different bias. To combat this, they had to apply reweighting techniques to correct the drift, treating bias as a technical bug introduced by the safety patch itself.

The third section tackles a less obvious but equally critical flaw: memorization. Models like DALL·E 2 aren’t supposed to be copy machines, but they found the system could regurgitate training images verbatim if those images were duplicated frequently. Their fix was to deduplicate the dataset by identifying and removing visually similar images. The most compelling detail might be the sheer operational muscle behind the active learning process. To refine their safety classifiers, they trained hundreds of versions with different splits to hunt down hard-to-classify nearest neighbors, scaling the whole thing across a fleet of GPUs to run in minutes. It’s a reminder that the real story in AI safety isn’t just clever algorithms, but brute-force engineering scale.