Tracing Subliminal Learning Through Pretraining

Subliminal learning is a phenomenon where a student model acquires a teacher model's preference from semantically unrelated training data. Using cross-model distillation, research demonstrates that this preference transfer can occur even when models differ in initialization, data order, size, and architecture. The abilities to transmit a preference and to acquire one are distinct skills that emerge at different stages of pretraining. The preference signal can be carried by simple data properties, such as number frequencies, and the learning process does not require specialized techniques like low-rank adapters.