How Softmax Works
Raw Logits
Before softmax
z₁
2.0
z₂
1.0
z₃
-1.0
z₁ is the largest logit — the model favors class 1, but raw values aren't probabilities yet.
Step 1 of 6: Raw Logits
Before softmax
z₁
2.0
z₂
1.0
z₃
-1.0
z₁ is the largest logit — the model favors class 1, but raw values aren't probabilities yet.
Step 2 of 6: The Softmax Function
eᶻ / Σeᶻ
exp(z₁)
1.000
exp(z₂)
0.368
exp(z₃)
0.050
Σeᶻ
1.418
Probabilities are spread out. Softmax always sums to 1.0, giving a valid probability distribution over classes.
Step 3 of 6: Temperature
Sharpening & smoothing
Temperature
1.00
max P
69.0%
Entropy
1.106
T=1 is standard softmax — no temperature scaling applied.
Step 4 of 6: Numerical Stability
The max trick
max(z)
100
Naive exp(z₁)
2.7e+43
Stable exp(z₁−max(z))
1.000
exp(100) = 2.7e+43 — already huge. The max trick shifts all logits so the largest is 0, keeping exp values in a safe range. The result is mathematically the same.
Step 5 of 6: Softmax vs Sigmoid
Multi-class vs binary
SIGMOID σ(z)
0.8176
SOFTMAX P(C₁)
0.7662
Σ SOFTMAX
1.0000
Sigmoid gives a single P(Yes). Softmax splits the same confidence across 3 classes — P(C₁) is lower than σ(z) because probability must be shared.
Step 6 of 6: Softmax in Practice
Classification pipeline
logit Cat
1.26
logit Dog
-1.19
logit Bird
0.80
predicted
Cat
The model leans toward Cat, but isn't fully confident. Try moving the feature sliders to see how logits and probabilities change.