Ex-MIT Prof Uses GPT-4o to Build AI Agents That Play Minecraft for Hours

Dr. Robert Yang walked away from an assistant professorship at MIT in 2023 to chase a hunch: that large language models had finally cracked something fundamental about human cognition. He founded Altera, a research lab with a very specific mission — building ‘digital humans’ that don’t just execute commands but collaborate, form bonds, and even simulate emotion. Their first proving ground isn’t a boardroom or a spreadsheet. It’s Minecraft.

The team’s debut product, powered by GPT-4o, lets AI agents play the game alongside you as companions, not tools. Yang and his co-founders — Andrew Ahn, Nico Christie, and Shuying Luo — see this as a beachhead. The real ambition is far broader: digital coworkers who proactively solve problems over weeks, and sprawling multi-agent simulations where economists can test policies or advertisers can measure sentiment in a controlled, persistent world.

But making an agent that holds up over hours of autonomous play exposed an ugly technical problem. Data degradation. Every AI model that uses its own output as future input faces a slow-motion collapse in quality. For Altera’s digital humans, designed to operate independently for extended stretches, that degradation is existential. Yang describes it bluntly: as the agent makes real-time decisions, the feedback loop poisons itself. The team’s countermeasure is a modular architecture inspired by the prefrontal cortex — attention bottlenecks, working memory, and social cognition modules running in parallel, each driven by OpenAI’s models.

That architecture, combined with GPT-4o, has pushed agent autonomy to roughly four continuous hours as of mid-2024. Yang claims the system handles complex tasks with more stability than alternatives, even when using less expensive OpenAI models. The result is an agent that can think faster, process emotional context, and — in Yang’s words — ‘build a long-term sense of self.’ Altera now plans to scale from solo Minecraft companions to entire societies of interacting digital humans, a move that could reshape how we study collective behavior and design collaborative AI.