OpenAI's Retro Contest ends: 923 teams, one clear lesson

The first OpenAI Retro Contest is in the books, and the results deliver a humbling yet strangely validating message for reinforcement learning researchers. Over two months, 923 teams signed up and 229 actually threw their hats in the ring, submitting algorithms to master unseen Sonic the Hedgehog levels. The automated systems churned through 4,448 evaluations—roughly 20 per team—testing agents on a small set of low-quality levels. But here’s the kicker: the best any team managed was a score of 4,692. The theoretical maximum is 10,000.

That gap isn’t a failure. It’s the whole point. OpenAI designed the Sonic benchmark specifically to measure generalization from previous experience, and the fact that nobody got close to a perfect score confirms the problem is genuinely hard. More importantly, the winning solutions weren’t bizarre, competition-specific hacks. They were tuned-up versions of known algorithms like PPO and Rainbow. As the organizers put it, this suggests you can’t simply cheat your way through this problem—the benchmark is doing its job.

So who actually won? A six-person team called Dharmaraja, with members from Alibaba’s AI search division and Nanjing University, held onto the top spot from the leaderboard through to the final, secret evaluation. Their approach was a souped-up joint PPO that fed the agent RGB images, added common button combos to the action space, and tossed in an extra reward for visiting new states using a perceptual hash of the screen. They also tried things that flopped, like object detection with YOLO and DeepMimic. The runner-up, a solo act named Alexandre Borghi (team Aborg), also fine-tuned a joint PPO from a pre-trained network. The real outlier is third-place team mistake, two grad students from the Chinese Academy of Sciences who built on Rainbow DQN and notably trained from scratch—meaning their agent started much lower but learned everything on the fly, timing out at the 12-hour mark.

The final shakeup was modest. To prevent overfitting to the public leaderboard, OpenAI had professional level designers build 11 completely new Sonic levels for final evaluation, running each agent three times with different random seeds. The ranking shifted a bit—mistake narrowly edged out Aborg for second—but the pecking order largely held. If you’re looking for a tidy conclusion, don’t. The real story here is that even the best general-purpose algorithms, when pushed to adapt to the unexpected, still leave half the points on the table. That’s a measurable, maddening, and useful north star for the next wave of research.