PrismML just dropped Bonsai 2 27B, a compressed version of Alibaba’s Qwen3.8 27B that weighs in at a mere 5.9 GB. That is a 9x to 10x reduction in memory footprint, small enough to run locally on a standard PC or even a high-end smartphone. The startup claims this isn’t just a size hack; the model retains 98% of the original’s aggregate benchmark scores. For context, their first release in March hit 95% parity and has already been downloaded over 11 million times. This latest iteration adds another 2.6 million downloads to the tally, suggesting developers are hungry for efficient, local inference options.
The secret sauce lies in how PrismML handles model weights. Traditional large language models store weights using 16 bits of data. PrismML’s “ternary” compression technique simplifies each weight to just three possible values: +1, −1, or 0. It sounds radical, but CEO Babak Hassibi, a Caltech professor and compression expert, argues the performance hit is negligible for real-world tasks. Benchmarks are imperfect proxies for utility anyway. A 2% drop in synthetic scores rarely translates to noticeable errors in actual user interactions, especially when you factor in the software harness surrounding the model.
Backed by Khosla Ventures and advised by Databricks co-founder Ion Stoica, PrismML is betting big on edge computing. Stoica notes that running intelligence directly on devices makes it effectively free and inherently private, since no data leaves the phone. While competitors like Multiverse Computing are also tackling compression, PrismML’s academic pedigree and rapid iteration speed set it apart. Rumors of talks with Apple remain unconfirmed, but the technical trajectory points toward ubiquitous on-device AI. Hassibi says the next targets are models in the several-hundred-billion-parameter range, where he expects compression to be even more effective. If they can shrink giants without dumbing them down, the cloud monopoly on heavy AI lifting could finally crack.
