How Tolan built a voice-first AI with GPT-5.1

Tolan is a voice-first AI companion app where people talk to an animated character that learns from them over time. Built by Portola, the app targets open-ended conversations, not quick prompt-reply exchanges. Voice AI is harder than text because you’re holding a live, meandering chat that demands near-instant responses and consistent personality. The team focused on memory and character design, creating a character-driven universe with animators and a sci-fi writer. GPT-5.1 was a turning point, offering major gains in steerability and latency. It cut speech initiation time by over 0.7 seconds, making conversations flow naturally. Tolan also rebuilds its context window from scratch each turn—pulling in summaries, persona cards, memories, and tone guidance—rather than caching prompts. This lets the system adapt smoothly to abrupt topic shifts. A memory system retains facts and emotional tone across long, nonlinear chats. Why it matters: Tolan shows how GPT-5.1’s combination of speed and steerability is unlocking truly conversational AI that feels alive instead of robotic, pointing to a future where voice becomes the default interface for companionship and support.