Hugging Face just dropped SmolLM3, and the numbers don’t lie: a 3-billion-parameter model that outperforms Meta’s Llama-3.2-3B and Alibaba’s Qwen2.5-3B, while breathing down the neck of larger 4B models like Qwen3 and Gemma3. If you’ve been waiting for a model that doesn’t require a server farm but still handles serious reasoning, this is it.
The engineering team trained this thing on 11 trillion tokens across 384 H100 GPUs for 24 days, and they’re releasing the complete recipe — architecture decisions, data mixtures, the works. No black boxes here. The instruct version supports a dual-mode reasoning toggle, letting you switch between ‘think’ and ‘no think’ modes depending on whether you need chain-of-thought depth or snappy responses.
Under the hood, SmolLM3 ditches standard rotary position embeddings on every fourth layer, adopting a NoPE approach that stretches context length to 128k tokens without degrading short-context performance. They also swapped multi-head attention for grouped query attention with four groups, which dramatically shrinks the KV cache during inference. If you’re running models locally, that’s the difference between fitting in VRAM and watching your GPU wheeze.
Multilingual coverage spans English, French, Spanish, German, Italian, and Portuguese — a deliberate choice that covers major European languages without the bloat of supporting dozens of poorly-represented tongues. The three-stage pretraining progression starts with a broad web-code-math mix, then shifts to higher-quality math and code datasets before a final annealing phase. It’s the kind of pragmatic, transparent engineering that makes you wonder why every lab doesn’t document their process this thoroughly.
