PolyAI’s New Model Listens to Raw Audio, Skips the Transcript and Boosts Containment 11%

PolyAI just shipped a dialog model that doesn’t bother with transcripts. Dialog-RSN-1 takes raw audio directly, deciding turn-taking, function calls, and responses without ever converting speech to text on the critical path. It’s already running live production calls, and the early numbers are worth a look — +11% relative containment at a restaurant group and a 37% latency drop at an insurer.

This is audio-aware on the input side only. The team kept text-to-speech separate, which means customers don’t lose control over the output voice. The architecture probes the model on demand rather than streaming continuously, a design choice that avoids tying up a GPU for an entire call. The very first token the model spits out is the turn-taking decision: EMPTY, ONGOING, or COMPLETE. It’s an elegant way to let cheap acoustic cues decide when to ask the model, while the model itself makes the actual conversational call with full context.

PolyAI post-trained open-weight multimodal models with supervised and reinforcement finetuning on in-house data. They evaluated Gemma, GPT-OSS, Qwen, and Mistral, ultimately targeting the 8B dense to 30B sparse range to keep latency under 300ms on A100s. The latency tricks are worth noting — they prefill the attention cache while the user is still speaking, use an append-only prompt template to minimize cache invalidation, and route each caller to the same GPU.

Don’t expect to download this one. There are no open weights and no public API. Dialog-RSN-1 is English-only at launch and delivered exclusively through PolyAI’s platform. For non-English or rich web chat, the company still points customers to its Raven 3.5 model. Existing enterprise customers can enable it today; new ones can request early access. The company plans a technical report and a paper on Dialog-Eval, the internal benchmark it intends to open-source.