TII’s Falcon 180B has landed on Hugging Face, and it’s not just another incremental upgrade. This is the largest openly available language model yet, packing 180 billion parameters trained on 3.5 trillion tokens — mostly from TII’s RefinedWeb dataset, which makes up roughly 85% of the training data. The scale here is genuinely staggering: 2.5 times larger than Llama 2, trained with four times the compute, running on up to 4096 GPUs simultaneously via Amazon SageMaker for approximately 7 million GPU hours.
The numbers tell a compelling story. At release, Falcon 180B topped the Hugging Face Open LLM Leaderboard with a score of 68.74, beating Meta’s Llama 2 70B and OpenAI’s GPT-3.5 on the MMLU benchmark. It sits somewhere between GPT-3.5 and GPT-4 depending on which evaluation you’re looking at, and it’s on par with Google’s PaLM 2-Large across a broad set of benchmarks including HellaSwag, LAMBADA, and Winogrande. The leaderboard methodology shifted in November 2023 with two new benchmarks added, and Falcon’s updated score of 67.85 puts it neck-and-neck with Llama 2 70B under the new rules. Quantized versions — 8-bit and 4-bit — hold up remarkably well, preserving similar metrics to the full float16 model.
This is where things get interesting for developers. The base model is explicitly not for chat. It’s a pretraining platform designed for fine-tuning, and TII is clear about that. The chat variant uses a simple structured format: System, User, Falcon, repeating as needed. You can access everything through Transformers 4.33 and the broader Hugging Face ecosystem — safetensors, bitsandbytes quantization, PEFT, GPTQ, and speculative decoding all work out of the box.
But here’s the catch that should make any commercial user pause: the license. Falcon 180B can be used commercially, but under what Hugging Face itself calls “very restrictive conditions” that exclude any “hosting use.” That’s a significant limitation in an era where most open model adoption happens through hosted APIs. If you’re thinking about building a product on this, the recommendation to consult a legal team isn’t boilerplate — it’s the most practical advice in the entire announcement. The model is technically impressive, but the licensing constraints mean the open-source community’s ability to deploy it at scale will depend on how much flexibility TII is actually willing to grant. For researchers and fine-tuning enthusiasts, Falcon 180B is a serious new tool. For businesses expecting a drop-in replacement for commercial APIs, the fine print matters more than the benchmark scores.
