PrismML just dropped Ternary Bonsai 2 27B, a radical compression of the Qwen3.8 27B model that slashes its footprint from 53.80 GB to a mere 5.93 GB. This isn’t just standard quantization; it uses ternary weights—values of -1, 0, or +1—to achieve a 9.1x size reduction while retaining 98.2% of the parent model’s performance across 20 benchmarks. For developers eyeing local deployment, this means a powerful 27-billion-parameter multimodal model can now run on a standard 16 GB laptop or a single 24 GB GPU. It supports a massive 262K-token context and handles both text and images, making it viable for complex coding agents and computer use tasks without needing enterprise-grade hardware.
The technical magic lies in how PrismML handles the math. Each weight is packed into roughly 1.72 bits, using a blockwise Hadamard rotation to minimize accuracy loss during the ternary assignment. While the model keeps the original Qwen3.8 architecture intact, including its hybrid attention mechanism, the storage format requires specific tooling. You can’t just drop these weights into stock llama.cpp; you need PrismML’s custom fork or their MLX runtime for Apple silicon. The company offers two packing formats: PTQ1_0 for maximum density and PQ2_0 for faster unpacking on certain architectures like Blackwell and Apple chips.
Performance metrics show impressive speed, with an RTX 5090 hitting 142.5 tokens per second and an M5 Max reaching 46.8 tokens per second. However, the compression isn’t free. While general reasoning and knowledge retention remain high, long-horizon agent tasks take a hit. Scores on SWE-bench Verified drop to 60.8 compared to 80.6 for the full-precision baseline, representing about 75% retention. This suggests that while the model is excellent for quick queries and standard coding help, it may struggle with prolonged, multi-step autonomous workflows where precision compounds over time.
This release arrives just two months after the first Bonsai iteration, which retained only 95% of the original model’s capability. The rapid improvement to 98.2% signals that ternary quantization is maturing fast. Yet, the reliance on proprietary runtimes and unverified self-reported benchmarks warrants caution. If the ecosystem around these custom GGUF types doesn’t expand beyond PrismML’s own tools, adoption could stall despite the compelling hardware efficiency. It’s a bold step toward edge AI, but the friction in deployment remains a real barrier for casual users.
