Before Q, K, and V: Reconstructing the Transformer

Recurrent neural networks (RNNs) are fundamentally limited by a fixed-size memory that overwrites past information as sequences grow longer. This "squishing" of history is especially problematic for tasks requiring long-range dependencies and recalling specific facts. The Transformer architecture's key insight was to remove the sequential, recurrent connections entirely, enabling massive parallel computation. This design allows every element in a sequence to directly interact with every other element, but requires a