Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

Compressing large language models to make them smaller and more efficient almost always degrades their performance, necessitating a "healing" process to recover lost abilities. A new method, Quantization-Aware Healing (QAH), introduces a powerful solution by distilling knowledge from the original, full-size model directly into the small, 4-bit student. Unlike other techniques, QAH allows the compressed model to learn from a much larger, architecturally different teacher, breaking through previous performance limitations. The results are striking, as a 4-bit, 60B parameter model healed with this method actually outperformed its own full-precision 16-bit version on most benchmarks. This breakthrough inverts the typical trade-off, proving that a smaller, quantized model can be both more efficient and more capable than its larger counterpart.