A 7B Model Just Beat Everyone at Math Olympiad Problems. Here's the Recipe

The Numina team has just pulled off something remarkable: a fine-tuned 7-billion-parameter model that out-reasoned much larger systems to win the first AIMO Progress Prize. The competition, backed by a $5 million grand prize from XTX Markets, challenged teams to solve 50 problems at the difficulty level of AMC12 and AIME exams — the kind that would stump most high schoolers and, it turns out, most LLMs.

The winning approach wasn’t about raw scale. Numina started with DeepSeekMath-Base 7B and turned it into what they call a ‘reasoning agent’ — a model that mixes natural language reasoning with actual Python code execution to compute intermediate results. That’s a significant departure from the standard approach of letting models reason purely in text. When a model can offload computation to a Python REPL, it stops making the kind of arithmetic errors that plague even the most capable language models.

The training itself was surprisingly efficient. On a single node of 8 H100 GPUs, the fine-tuning process took just 10 hours using TRL, PyTorch, vLLM, and DeepSpeed. That’s a detail worth dwelling on: the winning solution didn’t require a massive compute budget or a proprietary frontier model. It required a clever recipe, good data, and a decoding strategy designed for tool-integrated reasoning.

Competition constraints forced discipline. Submissions ran on a single P100 GPU or 2xT4s with a 9-hour time limit, and only open-weight models released before February 23 were eligible. The Numina team leaned on internal validation sets to guide model selection, explicitly avoiding the trap of overfitting to the public leaderboard — a problem that has sunk many a Kaggle competitor. The team includes names familiar to the open-source AI world: Jia Li, Yann Fleureau, Guillaume Lample, Stan Polu, and Hugging Face’s Lewis Tunstall and Ed Beeching, with support from Mistral AI, General Catalyst, and Answer.ai.

What’s striking is the philosophical stance behind the technical choices. Numina went open-source and open-dataset from day one, arguing that progress in AI for mathematics should be universal. They invoke Ramanujan — the self-taught Indian mathematician — as a symbol of what becomes possible when tools are accessible. Whether that vision holds as the competition progresses toward the full IMO gold-medal threshold remains to be seen, but for now, the message is clear: you don’t need a 70B model to solve hard math. You need a 7B model that knows when to write code.