Ternary Bonsai: Top intelligence at 1.58 bits

r/LocalLLaMA
Generative AI AI Research

Today, we’re announcing Ternary Bonsai, a new family of 1.58-bit language models designed to balance strict memory constraints with high accuracy requirements. This release builds on the efficiency frontier we began exploring with the recently released 1-bit Bonsai models. The 1-bit family showed that extreme compression could still produce commercially useful language models. Ternary Bonsai targets a different point on that curve: a modest increase in size for a meaningful gain in performance. The models are available in three sizes: 8B, 4B, and 1.7B parameters.