What Happens When You Build an LLM Using Only 1s and 0s

A new large language model architecture, BitNet b1.58, challenges the trend of ever-larger models by restricting its weights to only three values: -1, 0, and 1. This method drastically improves efficiency by replacing computationally expensive floating-point multiplications with simple additions and subtractions. To overcome the challenge of training with discrete integers, the architecture uses a latent, high-precision version of the weights for gradient updates while using the quantized weights for the forward pass. The resulting models achieve performance comparable to standard full-precision models like LLaMA but with a significantly smaller memory footprint and faster inference speeds.