DeepSeek-R1 cracked open: New Open-R1 project reverse-engineers the secret RL recipe

DeepSeek’s R1 model didn’t just match OpenAI’s o1 on reasoning benchmarks—it tanked Nvidia’s stock and reshuffled the AI power rankings overnight. But the $5.5 million question remains unanswered: how exactly did they train it? The company’s tech report described a novel reinforcement learning pipeline that skips human supervision, yet they kept the training code and curated datasets locked up.

A new community effort called Open-R1 wants to pick that lock. Led by Hugging Face, the project aims to systematically reconstruct the entire R1 pipeline in the open, starting with distilling a reasoning dataset from the released R1 model weights. Phase two gets ambitious: curating massive new datasets for math, code, and reasoning to replicate the pure RL approach that produced R1-Zero—a model that taught itself chain-of-thought reasoning solely through reward signals.

This matters because DeepSeek proved something genuinely disruptive. Their recipe takes a capable base model like the 671B-parameter DeepSeek-V3, applies Group Relative Policy Optimization for efficient RL, and uses reward systems checking both answer accuracy and output structure. The result is a model that self-verifies and breaks problems into steps. But no one outside DeepSeek knows the hyperparameters, the data mixture ratios, or the scaling tradeoffs.

Open-R1’s third phase tackles the full training arc: base model to supervised fine-tuning to RL. The initiative also plans to push beyond math and code into scientific domains like medicine, where reasoning models could actually affect human outcomes. The code lives on GitHub now. Whether they can reproduce R1’s performance on a community-sized budget is the multi-million-dollar question—literally.