A 32B model just crushed Claude 3.7 Sonnet on Olympiad coding problems
Curated by the Inblix editorial team
The Open R1 project just dropped a genuinely surprising result: a fine-tuned 32-billion-parameter model that out-codes frontier systems from Anthropic and others on problems from the International Olympiad in Informatics. Not just edge cases—the full, brutal test suites.
The model, OlympicCoder-32B, was built on Qwen2.5 Coder Instruct and trained on a new dataset called CodeForces-CoTs. That dataset contains nearly 100,000 chain-of-thought reasoning traces from DeepSeek-R1, covering solutions in both C++ and Python. The team also released a companion benchmark, IOI, built from the 2024 Olympiad problems and including complete test cases—the first fully verifiable competitive programming dataset of its kind.
What makes this notable isn’t just the leaderboard position. It’s the scale of the efficiency gain. The team reports OlympicCoder-32B outperforms every open-weight model they tested, including some more than a hundred times its size. In practical terms, that means a model you can run on a single powerful GPU is now competitive with the best proprietary systems for algorithmic reasoning.
I’ve seen a lot of coding benchmarks that fall apart under scrutiny because their test cases are incomplete. The team calls this the “code verifiability crisis,” and they back up the claim with a sobering stat: seven CodeForces solutions that passed all public test cases failed on the full test set when submitted to the actual platform. Every single one. The IOI benchmark sidesteps this by using the official, fully released test suites from the 2020-2024 competitions, licensed under CC-BY. That means when OlympicCoder posts a score, it reflects performance on the real, hidden-edge-case tests that trip up models trained on truncated public datasets. If this approach catches on, it could reset how we evaluate reasoning models—away from leaderboard gaming and toward the kind of verification that competitive programming actually demands.
💡 Key Takeaways
- OlympicCoder-32B, a Qwen fine-tune trained on 100k DeepSeek-R1 reasoning traces, outperforms Claude 3.7 Sonnet and models over 100x its size on IOI problems with full test suites.
- The new CodeForces-CoTs dataset includes chain-of-thought generations in C++ and Python for 10k+ problems, 3,000 of which were missing from previous industry-standard datasets.
- Public test cases are dangerously misleading: seven solutions that passed all visible tests failed on CodeForces's full hidden test suite, exposing a fundamental flaw in how code reasoning models are currently evaluated.
Keep reading: See related articles below for more coverage on this topic.
Get smarter about AI
The sharpest AI news, curated daily. Delivered free to your inbox.