Frontier LLMs fumble classic 80s text games, fail to climb back down cliffs

If you think today’s frontier models are on the verge of AGI, watching them get hopelessly lost in a 1980s text adventure might change your mind. Researchers behind the new TextQuests benchmark dropped a suite of 25 classic Infocom games—including Zork I and Wishbringer—in front of the best LLMs, and the results are a sobering reality check.

The test is brutally straightforward. Each model gets up to 500 steps to solve puzzles that demand multi-step planning over a context window that can balloon past 100,000 tokens. No external tools. No memory hacks. Just the model’s own reasoning, with two setups: one blind and one with access to the game’s official hint guide. Progress is measured by a series of labeled checkpoints, and the benchmark also tracks harmful in-game actions.

Long-context reasoning collapsed in spectacularly human-like ways. As the history grew, models started hallucinating past interactions, insisting they’d already grabbed an item that remained untouched or trapping themselves in navigation loops. The paper notes a particularly damning failure: most LLMs couldn’t reverse a sequence of moves to climb back down a cliff in Wishbringer, even though the directions were sitting right there in the context history. Spatial reasoning and building a mental map remain a fundamental weakness. Efficiency is another sore spot. More test-time compute generally helps, but the returns diminish fast—many basic navigation steps don’t need a thousand reasoning tokens.

What stings isn’t just the failure rate. It’s that these are games designed for humans playing on a Commodore 64. The benchmark shows that sustained, self-directed exploration over a growing context is still an enormous gulf between current LLM performance and anything resembling robust autonomous reasoning. The leaderboard is open for submissions, but early results suggest that no one should hold their breath for a model that can conquer the infamous Zork I maze without getting utterly lost.