AI Agents Invented Tool Use Just to Win Hide-and-Seek

Nobody told them to pick up the blocks. OpenAI researchers have documented agents in a simulated hide-and-seek game developing increasingly sophisticated tool use entirely on their own, progressing through six distinct strategic stages without any explicit instruction to interact with their environment. The setup was deceptively simple: hiders must break line-of-sight from seekers, using objects scattered around a procedurally generated arena. The only supervision came from a binary reward signal tied to the game’s core objective, not object manipulation.

What emerged was an “autocurriculum” driven by competitive pressure. Early on, hiders learned to run away and build crude shelters by moving boxes. Seekers eventually learned to use ramps to scale those shelters, which forced hiders to lock boxes in place before assembling their fortifications. The arms race escalated to a point where seekers discovered they could surf a locked box across the floor to reach entrenched hiders, a tactic the researchers admit they didn’t know the physics engine would even support. No one programmed these strategies; the agents invented them in response to their opponents’ innovations.

The architecture behind this was no slouch. The team used the same large-scale infrastructure that powered their Dactyl robotic hand, training each agent independently with a Transformer-like attention mechanism that masked out objects not in line-of-sight. Real progress demanded serious compute—batch sizes below 32,000 episodes never reached the ramp-defense stage, and the researchers found that scaling batch size drastically reduced the wall-clock time needed for these strategies to crystallize.

Intrinsic motivation methods, a popular alternative for unsupervised skill discovery, fell short in comparison. When agents were simply rewarded for exploring novel states, they shuffled boxes around in a meaningless, undirected way—especially as the state space grew more complex. The hide-and-seek agents, by contrast, produced behaviors that were qualitatively more interpretable: building shelters, using tools, and crafting counter-strategies. This suggests that multi-agent competition, where an adversary’s next move is the curriculum, might be a far more scalable path to generating complex, human-relevant AI skills than simply telling an agent to be curious.