1Probability & Distributions 2Statistics Essentials 3Bayes' Theorem 4Hypothesis Testing 5Sampling & Monte Carlo 6MLE & MAP Estimation 7Entropy & Cross-Entropy 8KL Divergence

Lensa ML

How Bayes' Theorem Works

Step 1 of 6: Prior Belief

What you believe before seeing any evidence

Prior P(H)

P(H)

0.50

P(¬H)

0.50

A balanced prior means you are unsure, so there is plenty of room for evidence to push belief either way. Near 0 or 1, even strong evidence moves the probability only a little.

Drag the Prior P(H) slider from 0 to 1. The two bars always sum to 1 because H and ¬H cover every possibility. Set the prior to exactly 0 or 1 and no evidence can change it, since Bayes' rule multiplies by the prior. Statisticians call this Cromwell's rule: keep a little probability on anything you are not sure is impossible. In ML, a prior can come from class frequencies in the training data, from domain knowledge, or from the output of an earlier model.

Step 2 of 6: Likelihood

How probable is the evidence under each hypothesis?

P(E|H) — likelihood if hypothesis true

P(E|¬H) — likelihood if hypothesis false

P(E|H)

0.80

P(E|¬H)

0.20

Likelihood ratio

4.00

The evidence is more than 3 times as likely under H as under ¬H. Seeing it will raise belief in H sharply.

Drag P(E|H) and P(E|¬H) separately. Unlike the prior bars, these two numbers do not have to sum to 1. Each is the probability of the same evidence under a different hypothesis, so both can be high at once. The update depends only on their ratio, shown in the third card. Bayes' rule in odds form is posterior odds = prior odds × likelihood ratio, so a ratio of 4 multiplies the odds of H by 4 whatever the prior was. Diagnostic tests report this number as the positive likelihood ratio, LR+.

Step 3 of 6: The Update

Bayes' formula combines prior and likelihood

Bayes' theorem

P(H|E) = P(E|H) × P(H) / P(E)

P(E|H) = 0.90P(H) = 0.30P(E|¬H) = 0.20P(¬H) = 0.70

P(H|E) =0.90 × 0.300.90 × 0.30 + 0.20 × 0.70

= 0.270 / 0.410 = 0.659

Prior P(H): 0.30

P(E|H)×P(H)

0.270

P(E) = total evidence

0.410

P(H|E)

0.659

Evidence boosted belief from 0.30 to 0.659 , a large upward update because the evidence fits H much better than ¬H.

Drag the Prior P(H) slider while P(E|H) stays at 0.90 and P(E|¬H) at 0.20. At the default prior of 0.30, the numerator is 0.9 × 0.3 = 0.27 and P(E) is 0.27 + 0.2 × 0.7 = 0.41, so the posterior is 0.27 / 0.41 ≈ 0.659. The denominator is the law of total probability: E can happen with H or without it. Its only job is to make the posteriors sum to 1. With many hypotheses, computing P(E) means summing or integrating over all of them, which is the expensive part of Bayesian inference.

Step 4 of 6: Visual Bayes

An area diagram makes the math intuitive

P(H|E) = P(E|H)·P(H) / [P(E|H)·P(H) + P(E|¬H)·P(¬H)]

Prior P(H): 0.40

P(E|H): 0.75

P(E|¬H): 0.15

P(H|E)

0.769

Update size

36.9%

Neither colored area dominates. After this evidence, H is still uncertain, neither confirmed nor ruled out.

Drag all three sliders and watch the rectangle. The prior splits its width into two columns, and each column fills up to its likelihood, so the orange area is P(E|H)·P(H) and the purple area is P(E|¬H)·P(¬H). Observing E discards everything that is not colored, and the posterior is the orange share of what is left. Set P(E|H) equal to P(E|¬H) and both columns fill to the same height. The orange share then equals the prior: evidence that both hypotheses predict equally well changes nothing.

Step 5 of 6: Sequential Updates

Feed evidence one at a time — posteriors compound

Initial Prior: 0.10

Next evidence: Test +

P(E|H)0.85

P(E|¬H)0.25

Current belief

0.100

Evidence seen

0 / 5

Last P(E|H)

—

Last P(E|¬H)

—

Press Play or Step to feed in evidence one piece at a time. Each posterior becomes the prior for the next update.

Press Step to apply the five pieces of evidence one at a time, or Play to run through them. From the default prior of 0.10, belief ends near 0.97, because the five likelihood ratios (3.4, 2, 9, 1.2 and 4) multiply the prior odds by about 294. The order of the evidence does not change the final answer. Chaining updates like this assumes the pieces are independent given H. Counting correlated evidence twice makes the result overconfident. Kalman filters and online learning use the same posterior-becomes-prior loop.

Step 6 of 6: Bayes in ML

A one-feature Bayes classifier in action

Feature value: 5.0

P(Cat|x)

42.5%

P(Dog|x)

57.5%

Decision

Dog

In the overlap region both classes are plausible. The decision boundary sits where the two posteriors cross at 50%, near x = 4.8.

Drag the Feature value slider from −1 to 13. Cat feature values follow a normal curve with mean 3 and σ = 1.5, dog values one with mean 7 and σ = 2, and the priors are 0.45 and 0.55. At x = 5, halfway between the means, P(Cat|x) is only about 43%: 5 is 1.33 cat standard deviations from the cat mean but only 1 dog standard deviation from the dog mean. Because the variances differ, the boundary is quadratic, with a second crossing near x = −9, outside the slider range.

What is Bayes' theorem?

Bayes' theorem is a rule for updating the probability of a hypothesis after seeing evidence. It states P(H|E) = P(E|H) × P(H) / P(E): the posterior equals the likelihood of the evidence under the hypothesis, times the prior, divided by the overall probability of the evidence. It follows in one line from the definition of conditional probability.

In machine learning, Bayes' theorem is the basis of Naive Bayes classifiers for spam filtering and text classification, of MAP estimation, where a prior on the weights acts as a regularizer, and of Bayesian optimization for hyperparameter search. A classifier that outputs calibrated class probabilities is estimating the posterior P(class | input) that the theorem describes.

Key ideas

The posterior is proportional to likelihood × prior; P(E) only rescales it so the probabilities sum to 1.
A prior of exactly 0 or 1 cannot be moved by any evidence.
The likelihood ratio P(E|H) / P(E|¬H) measures how strongly evidence favors H over ¬H.
In odds form, posterior odds = prior odds × likelihood ratio.
Updates chain: each posterior becomes the prior for the next piece of evidence.
A Naive Bayes classifier picks the class with the largest likelihood × prior.

Common questions

How is Bayes' theorem derived?

Write the joint probability of H and E in two ways: P(H and E) = P(H|E) × P(E) = P(E|H) × P(H). Dividing both sides by P(E) gives Bayes' theorem. Nothing beyond the definition of conditional probability is needed.

What is a conjugate prior?

A conjugate prior is one that gives a posterior from the same family of distributions, so the update reduces to changing a few numbers. The standard example is a Beta(a, b) prior on a coin's heads probability: after k heads in n flips, the posterior is Beta(a + k, b + n − k). Conjugate pairs let you do exact Bayesian updates without numerical integration.

How is Bayes' theorem used in machine learning?

Naive Bayes classifiers apply it directly to pick a class from word counts or other features. MAP estimation maximizes likelihood × prior over a model's parameters, and a Gaussian prior on the weights gives L2 regularization. Bayesian neural networks and Bayesian optimization keep a full posterior over parameters or over an unknown function and use it to measure uncertainty.

Who discovered Bayes' theorem?

Thomas Bayes, an English minister, worked out a special case. Richard Price edited his notes and published them in 1763, two years after Bayes died. Pierre-Simon Laplace reached the result independently in 1774 and developed it into the general form used today.

Related explainers

Probability & Distributions — Spam, densities, and the math of uncertainty
Naive Bayes Classifier — Probabilistic classification with Bayes' rule
MLE & MAP Estimation — Maximum likelihood and maximum a posteriori — why training works
Statistics Essentials — Mean, variance, standard deviation, and the Central Limit Theorem
Hypothesis Testing — Make principled decisions from data using null hypotheses, p-values, and significance levels
Entropy & Cross-Entropy — Measuring surprise, uncertainty, and prediction quality

References

Last updated October 25, 2026