Temperature in 1 picture: from confident to spread out
Classifiers and language models turn their scores into probabilities with softmax, a function that maps any list of numbers to values between 0 and 1 that add up to 1. Temperature is one extra number that sets how sure the result sounds, without changing which answer ranks first.
2 min read · Lensa ML
- On the left are three raw scores, called logits, fixed at 2.0, 1.0 and −0.5. Softmax raises e to the power of each score and divides by the total, so bigger scores get bigger shares. Temperature T divides every logit first: P(i) = e^(zᵢ/T) / Σⱼ e^(zⱼ/T). At T = 1 nothing changes, and the top class gets 69.0%.
- Slide T down to 0.1. Dividing by a small number stretches the gaps: 2.0 and 1.0 become 20 and 10. Each extra point of gap multiplies the ratio between two probabilities by e, about 2.72, so a gap of 10 makes the top class about 22,000 times as likely as the next. The right side collapses to one bar at 100.0%. Entropy, a measure in bits of how spread out the probabilities are, drops to 0.001.
- Slide T up to 5. Dividing by 5 squeezes the logits to 0.4, 0.2 and −0.1, so their exponentials sit close together and the bars nearly level out at 41.2%, 33.8% and 25.0%. Entropy climbs to 1.556 bits, close to the maximum for three classes, log₂ 3 ≈ 1.585, which only a perfectly even split reaches. The order never changes: the highest logit keeps the tallest bar at every temperature.
Takeaway
When a language model picks its next token, a word or piece of a word, it samples from these probabilities. Low temperature makes it pick the top token almost every time, which suits tasks where consistent answers matter. Higher temperature spreads the picks across more tokens for more varied text. Below 1 sharpens, above 1 flattens, and the ranking stays fixed.
Go deeper · interactive
Softmax, Logits & Temperature
Drag the temperature slider and watch the bars go from one spike to nearly flat.
Open the explainer