OpenAI’s Voice Engine Needs Just 15 Seconds to Clone a Voice

OpenAI has pulled back the curtain on Voice Engine, a model it first built in late 2022 that can clone a person’s voice from just a 15-second audio sample. The company isn’t rushing to launch it broadly—and for good reason. In a blog post detailing results from a small-scale preview, OpenAI explicitly flagged the model’s potential for misuse, especially during a major global election year, and is instead using a set of trusted partners to explore the upside while it figures out the guardrails.

The applications already in testing are genuinely striking, not just another parlor trick. Age of Learning, an edtech company, is pairing Voice Engine with GPT-4 to generate real-time, personalized voice responses for kids, opening up more expressive and representative reading assistance. On the translation front, HeyGen is using the model to dub videos into multiple languages while preserving the speaker’s vocal identity and even their native accent—so a French speaker’s English translation still sounds distinctly French.

Healthcare and accessibility use cases are where the model’s speed really shines. Because it needs so little source audio, doctors at Lifespan’s Norman Prince Neurosciences Institute managed to restore a young patient’s voice after she lost fluent speech to a vascular brain tumor, using nothing more than a clip from an old school project video. Dimagi is also leveraging the tech to give community health workers interactive feedback in their primary languages in remote settings, while the AAC app Livox uses it to craft unique, non-robotic voices for non-verbal individuals.

OpenAI is framing all of this as a conversation starter rather than a product launch. They’re engaging with policymakers globally and have built safety measures like watermarking and proactive monitoring into the system. The company says it will use the feedback from these early deployments to decide if and how to deploy the technology at scale. Until then, the preset voices in ChatGPT and the text-to-speech API are the only public taste of what Voice Engine can do.