China's Moonshot Just Dropped a 2.8T Parameter Model That Trades Compute for Memory

The Kimi K3 from Moonshot AI isn’t just another large language model; at 2.8 trillion parameters, it’s the largest open-weight model ever released, crashing into what the industry calls the 3T class for the first time. But fixating on the parameter count misses the real story. This model is a blueprint for navigating US chip sanctions, and it reads like a deliberate, methodical attempt to spend less of the one thing China’s labs are starved of: compute.

The core mechanism is a massive mixture-of-experts architecture. K3 splits itself into 896 specialized sections but only activates 16 for any given task—roughly 1.8% of the total. This slashes the calculation required per word, but it does nothing for memory, since all 2.8 trillion parameters must sit ready in silicon. Moonshot attacked that memory bill directly by baking in quantization-aware training from the fine-tuning stage, crunching the model down to four bits of precision per parameter. Independent analysis suggests this puts K3 at roughly 1.4TB, a quarter of what it would otherwise demand.

A new technique called Kimi Delta Attention then targets the ballooning memory cost of processing long documents. The company contributed caching code to the vLLM project to make this work, a commercial move they’re unusually direct about—it’s what lets them price the model competitively. Their solution recommends pooling memory across 64 or more accelerators, an approach that echoes Huawei’s CloudMatrix systems. It reveals a crucial asymmetry in sanctions: you can aggregate memory across a fleet of weaker chips, but you cannot stitch together training-grade compute the same way.

The hardware question remains a deliberate mystery. Moonshot’s tests ran on Nvidia H200s and a nameless “GPGPU from an alternative vendor,” with benchmarks on the export-controlled Nvidia L20. The blog doesn’t say where those H200s live, and a US House bill already aims to close the offshore cloud rental loophole. For enterprises, the practical test arrives July 27 when the weights drop. Banks and insurers across Asia piloting self-hosted models for data sovereignty will finally see if a model this colossal can actually run inside their own walls, not just on a benchmark leaderboard.