OpenAI ships gpt-realtime with MCP, image inputs, and phone calls

OpenAI’s Realtime API graduated from beta today with a stack of features aimed squarely at enterprises that want voice agents in production, not just in demos. The centerpiece is gpt-realtime, a new speech-to-speech model that the company claims is its most advanced yet. It handles instruction following, tool calling, and audio comprehension significantly better than the December 2024 predecessor. On the Big Bench Audio eval, accuracy jumped from 65.6% to 82.8%. Instruction following on the MultiChallenge benchmark nearly doubled to 30.5%, and function calling performance on ComplexFuncBench climbed from 49.7% to 66.5%. Those aren’t incremental tweaks.

The model can now capture non-verbal cues like laughter, switch languages mid-sentence, and adapt its tone on the fly — think “snappy and professional” one moment, “kind and empathetic” the next. OpenAI also released two new voices, Cedar and Marin, exclusive to the API, and updated its existing eight voices to benefit from the same speech quality improvements. Developers who’ve been wrestling with multi-model pipelines for voice will appreciate that the Realtime API processes audio directly through a single model, cutting latency and preserving the kind of nuance that gets flattened by traditional speech-to-text-to-speech chains.

On the integration side, the API now supports remote MCP servers. Pass a server URL into the session config and the API handles tool calls automatically — no manual wiring required. That’s a meaningful unlock for developers who want their agents to pull from external tools and context without building brittle middleware. The API also gains image input support and Session Initiation Protocol (SIP) for phone calling, which means voice agents can now take and make calls through standard telephony infrastructure. For customer support, personal assistance, and education use cases, that’s table stakes.

OpenAI trained gpt-realtime in close collaboration with customers, aligning the model to how developers actually build and deploy voice agents rather than optimizing for a research benchmark. Asynchronous function calling now works natively, so long-running tool calls don’t stall a conversation — the model keeps talking while it waits for results. The company framed the release as a direct response to developer feedback gathered since the public beta launched last October. If the benchmark numbers hold up in the wild, the gap between stitching together voice demos and shipping reliable voice agents just got narrower.