OpenAI open-sources a bootcamp for deep reinforcement learning

Deep reinforcement learning has a reputation problem: it’s the hardest corner of an already difficult field to break into. OpenAI is trying to change that with Spinning Up, a new educational resource that packages crystal-clear code examples, tutorials, and exercises into a single curriculum designed for self-study. The release, announced Thursday, includes standalone implementations of six core algorithms — VPG, TRPO, PPO, DDPG, TD3, and SAC — along with an essay on growing into an RL research role and a curated list of essential papers.

The project grew out of OpenAI’s work with its Scholars and Fellows programs, where the team noticed something that surprised them. “It’s possible for people with little-to-no experience in machine learning to rapidly ramp up as practitioners, if the right guidance and resources are available to them,” the organization wrote in its announcement. That observation shaped the entire philosophy behind Spinning Up, which is now baked into the curriculum for the 2019 cohorts.

OpenAI is putting real resources behind making sure this isn’t just a code dump and a prayer. For the first three weeks, the team promises high-bandwidth software support — fast bug fixes, installation help, and documentation improvements. Then in April 2019, about six months out, they’ll do a serious review based on community feedback and announce what comes next. Any internal changes made while working with Scholars and Fellows will flow straight to the public repo.

The broader ambition here is about building a bench. OpenAI explicitly ties the release to its charter commitment to “create a global community working together to address AGI’s global challenges.” They’re also partnering with UC Berkeley’s Center for Human-Compatible AI to run a workshop in early 2019, and hosting their own at OpenAI’s San Francisco office on February 2nd — three hours of lectures, five hours of hacking, applications close December 8th. It’s a pragmatic bet that lowering the barrier to entry for deep RL might just bring more minds to the table when it comes to safety research.