TII’s Falcon models have arrived in the Hugging Face ecosystem, and the 40B version isn’t just big — it was sitting at the top of the Open LLM Leaderboard at release, outperforming LLaMA-65B while requiring less memory. You’ll need about 90GB of GPU to run it, which is substantial but still cheaper than the Meta model it beats. The 7B sibling needs only 15GB, putting it within reach of consumer hardware.
The real story here is the training data. Falcon-40B and Falcon-7B were trained on 1 trillion and 1.5 trillion tokens respectively, but unlike GPT-3 or PaLM which lean heavily on curated sources, over 80% of Falcon’s data comes from RefinedWeb — a massive CommonCrawl-derived dataset that TII scaled and filtered aggressively. Deduplication and strict quality controls did the heavy lifting rather than hand-picked corpora. TII has released a 600 billion token extract of RefinedWeb publicly, which matters more than the models themselves for anyone building their own LLM. That’s a genuine contribution to the open ecosystem, not just another weights dump.
The architecture has an interesting quirk too: multiquery attention. Instead of each head having its own key and value, all heads share one K and V. Pretraining doesn’t change much, but autoregressive decoding gets dramatically cheaper because the KV cache shrinks by 10 to 100 times. That’s the kind of detail that makes Falcon-40B feasible on hardware that would choke on similarly-sized models using vanilla attention. It also enables statefulness — a trick we’re seeing more of as inference costs become the real bottleneck.
Hugging Face has already wired up the ecosystem around Falcon. There’s a Text Generation Inference demo, a Core ML port of the 7B instruct model running on an M1 MacBook Pro, and instruct variants finetuned for assistant-style tasks. One caveat: you’ll need to allow remote code execution in transformers because Falcon uses a custom architecture not yet merged into the library. That’s a minor friction point, but the tradeoff is access to models that rival closed systems without the API bill. The 180B version that dropped a few months later suggests TII is just getting started.
