Descript, an AI-native video editor, has overhauled its translation pipeline using OpenAI’s reasoning models to fix a persistent dubbing problem: dubbed audio often sounded unnatural because different languages need different amounts of time to express the same idea. German, for instance, is typically “longer” than English, forcing Descript to artificially speed up or slow down translated speech, resulting in chipmunk-like or sluggish audio. By optimizing for both semantic fidelity and duration adherence during generation—not after—Descript improved duration adherence by 13 to 43 percentage points (depending on language) and saw a 15% increase in dubbed exports within the first 30 days. Previously, the company used Whisper for transcription and GPT models in its co-editor, but the new approach directly addresses the pacing issue that was the top user complaint. This enables batch translation and lip-syncing for entire content libraries, making high-quality localization faster and cheaper than traditional methods that required language experts for every step. Why it matters: This approach could democratize global video distribution for creators and businesses, eliminating a major technical barrier to scaling multilingual content.
How Descript uses AI for multilingual video dubbing
Descript, an AI-native video editor, has overhauled its translation pipeline using OpenAI's reasoning models to fix a persistent dubbing problem: dubbed audio often sounded unnatural because different languages need diff…
Summarized from
OpenAI Blog AI-summarized
Was this helpful?
Discussion
Join the conversation
Comments are powered by GitHub Discussions. You need a GitHub account to participate.
Go deeper
The sharpest AI news, curated daily.
One email each morning. Summarized, sourced, and skimmable in under three minutes.
The sharpest AI news, curated daily.
One email each morning. Summarized, sourced, and skimmable in under three minutes.