A new benchmark called DiG-bench is testing something frontier AI systems rarely get evaluated on: the ability to figure out unwritten rules through exploration, not instruction. Built by researchers from Oxford, Princeton, MIT, and others — including OG AI researcher Juergen Schmidhuber — the benchmark consists of 70 handcrafted games where both the rules and objectives are hidden. Players must poke around, observe how the environment responds, and infer mechanics on their own.
The results are humbling for today’s top models. Opus 5 and Fable 5 with Claude Code lead the pack, managing to beat tasks in Tier 7 — but only 0.2 of them, a success rate that looks almost accidental next to the human baseline. Every single game has been beaten by at least one person, meaning humans hit 100% on tasks where the best AI systems barely register. GPT-5.5, Kimi K3, GLM-5.2, and Gemini 3.1 Pro all trail behind, with the weaker models only cracking lower-tier games.
What makes DiG-bench genuinely interesting is that it isolates a prerequisite for creativity. Solving these games requires spotting important mechanics in novel environments and updating your assumptions — the kind of flexible, curiosity-driven reasoning that recursive self-improvement would demand. The benchmark keeps most games private so models can’t train on them, and the 70 games range from trivial to brutally difficult, with action spaces spanning 2 to 34 possible moves per step.
The newsletter’s author, Jack Clark, predicts human parity on DiG-bench by mid-2027 — and ties that milestone directly to recursive self-improvement becoming a serious possibility. That timeline feels plausible if progress on agentic reasoning continues at its current pace, but the gap between today’s 20% on Tier 7 and full human performance is wider than a single generation of model releases. The question DiG-bench really poses is whether scaling alone gets us there, or whether we need fundamentally different approaches to exploration and hypothesis testing.
