Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint

summary

PrismML has developed Ternary Bonsai 2 27B, a model that achieves near-lossless compression by using ternary weights to reduce its memory footprint by over 9x. This approach allows a 27B-class model to maintain 98.2% of its original performance while running efficiently on local hardware. By minimizing memory and power requirements, the model enables complex tasks like coding and agentic workflows to happen locally rather than in the cloud.

key points

1. The model uses ternary weights {-1, 0, +1} with FP16 scaling, resulting in a total footprint of 5.9GB and an effective 1.76 bits per weight.
2. Benchmark results show that the model retains nearly all the capability of the full-precision Qwen3.8 27B, specifically in sensitive areas like coding, vision, and tool-use.
3. Local deployment leads to significant gains in throughput and energy efficiency, outperforming full-precision 8B models in power consumption while reaching speeds up to 143 tokens per second.

who should read it

AI developers and hardware engineers interested in deploying large, capable models on local devices with limited memory and power budgets.

discussion positive