The speech synthesis field has a measurement problem. Word error rate, the standard objective metric, tells you almost nothing about whether a voice sounds natural. Mean opinion score studies are usually too small to distinguish between models of similar quality. Hugging Face’s answer is TTS Arena, a crowdsourced leaderboard that borrows its playbook directly from LMSys’s Chatbot Arena, which has already racked up over 300,000 human rankings.
The setup is straightforward. You type in text, two models synthesize it, you listen to both samples, and you vote for whichever sounds more natural. Model names stay hidden until after you vote, a deliberate choice to blunt bias and gaming. The launch lineup includes several open-source heavyweights alongside proprietary models, giving developers a direct read on how the open ecosystem stacks up against closed alternatives.
Rankings use an Elo-style system, the same approach that powers chess ratings and the Chatbot Arena leaderboard. Results will be public and update automatically as votes come in, though the board starts empty and models only appear once they’ve accumulated enough data to justify a position.
For anyone who’s watched the Chatbot Arena become the de facto reference point for LLM quality, this is a logical extension. What it actually delivers depends on volume: an Elo leaderboard is only as trustworthy as the number of votes behind it. With enough participation, though, TTS Arena could give speech researchers something they’ve never really had — a fast, public, and reasonably reliable signal for which voices actually sound good to humans. The team credits Clémentine Fourrier, Lucian Pouget, Yoach Lacombe, Main Horse, and the broader Hugging Face crew for the build. One note: the page now directs users to vote on TTS Arena V2 instead, and flags Zyphra’s Zonos release as a quality Apache 2.0 option worth watching.
