OpenAI's RL Workshop Draws 500+ Applicants, Tests Scaling Mentorship

OpenAI’s first Spinning Up Workshop, held February 2nd, wasn’t just another tech talk. While 90 people packed the SF office and nearly 300 tuned into the livestream, the real number that matters is the 500-plus who applied from around the world. It’s a clear signal of hunger for deep RL skills, and it directly tests OpenAI’s working theory: that workshops can scale the kind of mentorship and project guidance that’s usually locked inside elite fellowship programs.

The day was split into two clear acts. The morning was a conceptual deep dive. Joshua Achiam laid out the algorithmic foundations, Matthias Plappert walked through the nuts and bolts of training a dexterous robot hand to cross the sim-to-real gap using domain randomization, and Dario Amodei tackled the thorniest problem in AI safety. Amodei described the core issue bluntly: “correctly specifying agent behavior is hard.” The danger isn’t malice, but the ease with which you can accidentally give a powerful agent incentives to do something wildly different from what you intended. The proposed fix, which his team has worked on with DeepMind, involves learning reward functions from human preferences instead of hand-designing them.

But the main event was the afternoon’s semi-structured chaos. This is where the mentorship rubber hit the road. Breakout sessions led by researchers like Karl Cobbe and Daniel Ziegler went from TensorFlow introductions to a live, step-by-step coding session called “Writing DQN Together.” The goal wasn’t passive learning. It was to give attendees with wildly varying experience levels—from “almost none” to “built their own Dota bot”—direct access to experts for project guidance.

For a first experiment, OpenAI is calling it a success, gratified by the quality of the participants. The big, unanswered question is whether this model genuinely offers a path to scale the kind of high-touch mentorship that produces real researchers, or if it’s just a very good networking event. The challenge they’ve identified is real: sharing a curriculum is easy, but scaling guidance is not. Whether a single-day workshop with a few hundred people can truly replicate the apprenticeship model of their Scholars and Fellows programs remains, charitably, a live experiment.