Why judging generative AI by its samples is a dangerous game

The impressive images and audio clips pouring out of modern generative models have a seductive quality. They look real. They sound real. But a new paper from researchers at the University of Toronto and MIT argues that eyeballing samples is a deeply flawed way to measure how good these models actually are. The core problem is something AI researchers have been grappling with for years: for many of the most powerful decoder-based models—a category that includes VAEs, GANs, and generative moment matching networks—calculating the log-likelihood, a standard statistical measure of model fit, is mathematically intractable. You can’t just compute how well the model understands the data it’s supposed to represent.

The researchers propose a practical fix. They’ve adapted a technique called Annealed Importance Sampling (AIS) to reliably estimate log-likelihoods specifically for these decoder-based models, and they validated their numbers against a more computationally expensive gold-standard method called bidirectional Monte Carlo. To make sure others can actually use it, they’ve released the evaluation code publicly. This isn’t just an academic exercise in better bookkeeping. The team used their new tool to put several popular models under a microscope and uncovered some uncomfortable truths about what gets lost when we judge a model purely by how pretty its outputs are.

What they found should give the field pause. Their analysis revealed that existing log-likelihood estimators, which many researchers have relied on, can be wildly off the mark. More critically, the paper quantifies the degree to which these powerful generators overfit their training data and, crucially, how severely they miss important modes of the data distribution. This is the classic ‘mode collapse’ problem made measurable: a model might spit out a thousand flawless pictures of a dog, but if it can’t produce a cat to save its life, its understanding of the broader world of animals is broken, no matter how good the dogs look.

A core tension in generative AI becomes visible when you stop looking at the highlights and start reading the full statistical report. The work, led by Yuhuai Wu and colleagues, suggests the field has been navigating with a faulty compass, giving high-fives for realistic samples while a model’s deeper understanding lags far behind. The open-source evaluation code is a direct challenge to that complacency, providing a rigorous tool that could reshape which modeling approaches are considered genuinely state-of-the-art and which are just very good at faking it.