Mistral’s Mixtral 8x7B just landed on Hugging Face, and it’s already making waves. This isn’t a single dense model—it’s a Mixture of Experts (MoE) architecture packing 8 expert models under one hood. A router network selects two experts per timestep, which means the model decodes as fast as a 12B parameter dense model despite having roughly 45B total parameters. That naming is doing some heavy lifting, though. The ‘8x7B’ moniker led plenty of people to assume it’s an ensemble of eight 7B models. It’s not. Only the feed-forward layers are replicated; the rest of the parameters are shared.
The numbers are what make this worth paying attention to. Mixtral Instruct outperforms every other open-access model on MT-Bench and is the first to reach performance comparable to GPT-3.5. It also beats Llama 2 70B on most benchmarks and scores 40.2% on HumanEval for coding tasks. It handles a 32k token context window and speaks English, French, German, Spanish, and Italian. The Apache 2.0 license means commercial use is on the table.
Hugging Face rolled out the full integration stack: Transformers support in release 4.36, Inference Endpoints, Text Generation Inference for production deployments, and a fine-tuning example using TRL on a single GPU. Quantization support comes via bitsandbytes, and Flash Attention 2 is available for faster training and inference. The VRAM math is straightforward—expect requirements similar to a 45B parameter dense model.
What’s conspicuously absent is the training data. Like Mistral 7B before it, no details have been shared about dataset size, composition, or preprocessing. The same goes for the fine-tuning datasets and hyperparameters used for the instruct version. That’s a familiar pattern from Mistral: strong performance, permissive licensing, and a closed book on how the sausage gets made. Given how aggressively the open-source community has been closing the gap with closed models over the past year, Mixtral feels less like an anomaly and more like the new normal.
