Lensa ML
Lensa ML

How MLE & MAP Work

Step 1 of 6

Likelihood of Data

How probable is the data given a parameter?

pL(p)00.250.50.7510e+04e-48e-40.001L=9.77e-4k/N=0.50
LIKELIHOOD FORMULA
L(p) = pk · (1 − p)N−k
= 0.505 · 0.505 = 9.77e-4
p (prob. of heads)0.50
N (number of flips)10
k (heads observed)5
N FLIPS
10
k HEADS
5
p
0.50
L(p)
9.77e-4

Your p = 0.50 is very close to the observed ratio k/N = 0.50. The likelihood is near its maximum — this parameter makes the data highly probable.

Step 1 of 6: Likelihood of Data

How probable is the data given a parameter?

pL(p)00.250.50.7510e+04e-48e-40.001L=9.77e-4k/N=0.50
LIKELIHOOD FORMULA
L(p) = pk · (1 − p)N−k
= 0.505 · 0.505 = 9.77e-4
p (prob. of heads)0.50
N (number of flips)10
k (heads observed)5
N FLIPS
10
k HEADS
5
p
0.50
L(p)
9.77e-4

Your p = 0.50 is very close to the observed ratio k/N = 0.50. The likelihood is near its maximum — this parameter makes the data highly probable.

Step 2 of 6: Log-Likelihood

Taking logs for numerical stability

plog L(p)00.250.50.751-23.1-17.4-11.6-5.9-6.93peak at k/N
LOG-LIKELIHOOD FORMULA
log L(p) = k·log(p) + (N−k)·log(1−p)
= 5·log(0.50) + 5·log(0.50) = -6.93
p (prob. of heads)0.50
N (number of flips)10
k (heads observed)5
LOG L(p)
-6.93
L(p)
9.77e-4
k / N
0.50

The log-likelihood is at its peak here. Notice it has the same maximum as the raw likelihood — taking the log preserves the location of the peak while avoiding tiny floating-point numbers.

Step 3 of 6: Maximum Likelihood Estimation

Find the parameter that makes data most probable

plog L(p)00.250.50.751-27.7-20.9-14.1-7.3MLE=0.50yours=0.50
FINDING THE MLE: SET DERIVATIVE TO ZERO
log L(p) = k·log(p) + (N−k)·log(1−p)
d/dp log L = k/p − (N−k)/(1−p) = 0
k·(1−p) = (N−k)·p → k = N·p
p̂ = k / N = 6 / 12 = 0.5000
N (number of flips)12
k (heads observed)6
Your guess p0.50
MLE (k/N)
0.50
YOUR p
0.50
LOG L(MLE)
-8.32
LOG L(YOUR p)
-8.32

Your guess matches the MLE! The derivation above shows why: setting the derivative of log L to zero gives p̂ = k/N = 6/12 = 0.50. No other value can produce a higher likelihood.

Step 4 of 6: The Problem with MLE

Small samples lead to extreme estimates

MLE estimatecount00.250.50.75102356true=0.5
Sample size N5
N
5
TRUE p
0.50
MLE (sample 1)
0.20
MLE SPREAD
0.80

With N = 5, the MLE estimates still vary considerably (spread = 0.80). Some estimates are far from the true p = 0.5. More data helps, but we can also add prior knowledge.

Step 5 of 6: Adding a Prior (MAP)

Beliefs plus evidence yield better estimates

p(normalized)00.250.50.751likelihoodpriorposteriorMAPMLE
N (number of flips)10
k (heads observed)5
Prior strength α (Beta(α,α))5
PRIOR MODE
0.50
MLE (k/N)
0.50
MAP ESTIMATE
0.50
α
5

With α = 5, the prior pulls the MAP estimate (0.50) toward 0.5, away from the MLE (0.50). The posterior is a compromise between data and prior belief.

Step 6 of 6: MLE vs MAP

MAP estimation is MLE with a regularization penalty

pobjective00.250.50.751-35.4-25.2-15.1-4.9log L (MLE obj.)log prior (penalty)log L + log prior (MAP obj.)MLEMAPMAP objective = log-likelihood + log-prior (regularization penalty)
Prior strength α5
N (number of flips)10
k (heads observed)5
MLE (k/N)
0.500
MAP
0.500
DIFFERENCE
0.000
LOG PRIOR PENALTY
-5.5

The prior (α = 5) and data (N = 10) are balanced. MAP = 0.500 sits between MLE = 0.500 and the prior mode 0.5. This is the sweet spot — regularization prevents extreme estimates without ignoring the data.