Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem

A novel method for compressing large language models involves removing entire transformer blocks by treating the selection as a combinatorial problem. This approach reframes block removal as a constrained binary optimization problem, which is equivalent to finding the low-energy state of an Ising glass from physics. The energy of this system acts as a strong and inexpensive proxy for the pruned model's actual performance, enabling the evaluation of billions of potential configurations without running benchmarks. For intractable problems, the formulation allows the use of quantum and quantum-inspired solvers, yielding significant performance gains over other block-removal techniques, especially at high compression rates.