OpenAI's Two NeurIPS Competitions Target RL's Biggest Weaknesses

OpenAI, teaming up with AIcrowd, Carnegie Mellon, and DeepMind, just dropped details on two NeurIPS 2020 competitions designed to stress-test what reinforcement learning can actually do. They’re not your typical leaderboard climbs. The Procgen Competition is all about sample efficiency and generalization. Competitors get a fixed number of environment interactions to train agents across 16 wild, procedurally generated worlds—plus four secret test environments built just for the contest. The twist is that since every level is algorithmically created on the fly, an agent can’t just memorize a path. It has to actually learn adaptable skills, a challenge OpenAI hopes will push algorithms far beyond brittle, single-task performance. They’ve made the environments deliberately lightweight, too, so that researchers without a server farm can still iterate quickly and reproduce baselines.

The second contest, the MineRL 2020 Competition, tackles the sample efficiency problem from a different angle: leveraging human know-how. The task is absurdly ambitious. Agents have to figure out how to obtain a diamond in Minecraft from raw pixels, and they only get 8 million simulator samples and four days of training on a single GPU to do it. That’s a starvation budget. To make it possible, participants get access to the MineRL-v0 dataset—a colossal collection of over 60 million frames of human gameplay—to bootstrap their algorithms. For context, standard RL systems typically need hundreds of millions of interactions on multi-GPU setups to snag even an iron pickaxe, which was the penultimate goal in last year’s contest and the best any team managed.

Why the brutal constraints? OpenAI points directly at the uncomfortable truth behind flashy wins like AlphaStar and OpenAI Five. Those successes devoured exponentially more compute and simulator time, a path that’s simply a dead end for real-world problems where gathering samples is slow, costly, or dangerous. The entire premise of these competitions is to force a different kind of breakthrough—one that doesn’t just chase a higher score but actively shrinks the environmental interactions required to get there. It’s a direct acknowledgment that scaling up isn’t the same as getting smarter.

The MineRL competition has some sharp teeth to prevent clever cheating. Organizers won’t just take the winners’ word for it. They’ll train the top models from scratch themselves, enforcing strict hardware and sample limits to guarantee the efficiency is baked into the algorithm and not some hand-engineered trick. They’re even introducing a new measure specifically to discourage overfitting solutions to Minecraft’s quirks. The message is clear: they’re hunting for generalizable principles, not just a one-off diamond hunt.