Consistency models hit 3.5× boost by ditching a teacher

The knock on consistency models has always been the same: they’re fast, sure, but to get decent images you need a diffusion model to hold their hand. Yang Song and Prafulla Dhariwal just flipped that script. Their new paper details a set of training tweaks that let consistency models learn directly from data — no teacher required — and the results are a genuine leap forward.

Here’s the core insight that changes things. The authors dug into the theory and found that using an Exponential Moving Average (EMA) on the teacher model was actually hurting performance, a flaw previous work had missed. Once they killed the EMA and fixed the teacher in place, everything started working better. They didn’t stop there. They swapped out perceptual metrics like LPIPS — which can bias evaluation in weird ways — for Pseudo-Huber losses borrowed from robust statistics. They also introduced a lognormal noise schedule and a scheme that doubles discretization steps as training progresses. Combined with more careful hyperparameter tuning, the impact is hard to ignore.

The numbers tell the story. On CIFAR-10, their consistency training approach hit an FID of 2.51 in a single sampling step — a 3.5× improvement over prior methods. ImageNet 64×64 saw an even bigger jump, landing at 3.25 FID, which is 4× better than what came before. Two-step sampling pushed those scores even lower: 2.24 on CIFAR-10 and 2.77 on ImageNet 64×64. Those results don’t just beat previous consistency training attempts, they surpass models that relied on distillation from pre-trained diffusion models.

What’s particularly notable is that this closes the gap with other state-of-the-art generative models without sacrificing the one-step speed that makes consistency models appealing in the first place. The distillation ceiling — being forever capped by the quality of your diffusion teacher — is gone. Dhariwal and Song, both known for foundational work in this space, have essentially shown that consistency models can stand on their own. The question now is whether these techniques scale to the kind of high-resolution, diverse datasets where diffusion models currently dominate. If they do, the practical upside for real-time generation workloads is significant.