How to Decode the Temperature Parameter in LLMs

The temperature parameter in Large Language Models (LLMs) controls output randomness by drawing a direct mathematical parallel to the Boltzmann distribution in statistical physics. Raw model outputs, known as logits, are analogous to inverted energy levels, and the Softmax function converts them into a probability distribution. A low temperature makes the model deterministic and greedy by amplifying logit differences, which is ideal for logical tasks. Conversely, a high temperature increases creativity and unpredictability by making less likely word choices more probable.