Pick a Number Between 1 and 10

Large Language Models often pick the number 7 due to a multi-stage process, not just gradient descent as is commonly believed. Pretraining on human data causes the model to learn the human psychological preference for 7, creating a probability distribution that peaks at that number. This initial 'hill' of probability is then sharpened into a 'needle' by subsequent steps. Post-training methods like RLHF reward the most typical answers, amplifying the peak, while low-temperature decoding settings further exaggerate the highest probability, making the model almost exclusively select the number 7.