How Logs & Exponentials Work
Exponential Growth
The function that grows as fast as itself
Positive x: exponential growth kicks in. Notice the slope (yellow dashed) equals the value — exp is the only function where the rate of change equals the function itself.
Step 1 of 6: Exponential Growth
The function that grows as fast as itself
Positive x: exponential growth kicks in. Notice the slope (yellow dashed) equals the value — exp is the only function where the rate of change equals the function itself.
Step 2 of 6: The Natural Logarithm
The inverse of exp — turning multiplication into addition
ln(2.7) = 1.00 — notice how ln grows slowly. The two curves are mirror images across y = x, confirming that ln is the inverse of exp.
Step 3 of 6: Multiplication → Addition
Why logarithms turn products into sums
log(3.0) + log(4.0) = 2.48 = log(12.0). Logarithms convert multiplication to addition. This is why ML optimizers work with log-likelihoods — sums are numerically stable and easy to differentiate.
Step 4 of 6: Logs in Cross-Entropy
Why wrong confident predictions are punished severely
At p = 0.70, the loss is 0.36. The model is uncertain — the penalty grows, pushing it to commit to the right answer.
Step 5 of 6: Exponentials in Softmax
From raw logits to calibrated probabilities
Standard temperature range. Softmax converts logit differences into probability ratios: exp(z) amplifies gaps between logits, then normalization ensures probabilities sum to 1.
Step 6 of 6: Why We Take Logs
Log-likelihood keeps numbers stable as data grows
With 8 observations, the likelihood is already 0.011197 — a tiny number! But the log-likelihood -4.49 is perfectly manageable. This is why ML always maximizes log-likelihood, not likelihood.