Glitches in the Attention Matrix

A peculiar glitch known as "attention sinks" plagues many Transformer models, causing certain tokens to absorb an unusual amount of attention and grow abnormally large. This phenomenon arises because the attention mechanism must distribute its focus, which leads it to repurpose uninformative background tokens as sinks for global information, severely degrading performance in tasks like object detection. An early fix introduced dedicated "register" tokens to serve as designated spaces for this excess attention, but this solution required costly model retraining. More recent research has pioneered efficient methods, like self-distillation and direct modifications to the attention architecture, that can repair existing models without retraining or adding latency. These advanced solutions are already being incorporated into the latest large language models to improve their stability and performance.