1Decision Trees & Splitting 2K-Nearest Neighbors 3Support Vector Machines 4Naive Bayes Classifier 5K-Means Clustering 6Bagging & Random Forests 7Gradient Boosting 8Ensemble Methods & Stacking

Lensa ML

Naive Bayes Classifier

Step 1 of 6: Class Likelihoods

How x determines P(x|class)

Observed x1.00

P(x|spam)

0.2283

P(x|ham)

0.0375

Spam mean

2.0

Ham mean

-1.0

At x=1.0, the spam curve gives density 0.228 vs ham's 0.038, so P(x|spam) > P(x|ham). These likelihoods feed into Bayes' rule in the next step.

Drag the observed x slider from −4 to 5. Each curve is a Gaussian fitted to one class: spam has mean 2 and standard deviation 0.8, and ham has mean −1 and standard deviation 0.9. Gaussian naive Bayes learns exactly these two numbers per feature per class, so training is one pass over the data. The two curves cross near x = 0.6. A density is not a probability, and for a narrow enough curve its value can exceed 1. Only its ratio across the classes affects the decision.

Step 2 of 6: Bayes' Rule

Prior × likelihood ÷ evidence = posterior

Observed x1.00

Prior P(spam)0.40

P(x|spam)

0.228

P(x|ham)

0.038

P(spam|x)

0.802

P(ham|x)

0.198

How P(x) is calculated

P(x) = P(x|spam)·P(spam) + P(x|ham)·P(ham)
= 0.228 × 0.40 + 0.038 × 0.60
= 0.1138

With a 40% prior on spam and x=1.0, the posterior favors spam at 80.2%. Move x toward the spam mean (2) to raise P(x|spam), or toward the ham mean (-1) to raise P(x|ham).

Drag the observed x and prior P(spam) sliders. At the default x = 1 with a 40% prior, the likelihoods are 0.228 for spam and 0.038 for ham, so the posterior is 0.228 × 0.4 / (0.228 × 0.4 + 0.038 × 0.6) ≈ 80% spam. P(x) is the same for every class, so a classifier can skip it and compare the numerators directly. In practice the prior is the class frequency in the training data, which matters when classes are imbalanced, as in fraud detection or screening for rare diseases.

Step 3 of 6: The Naive Assumption

Features are independent

Actual correlation between features0.60

Actual corr

0.60

NB assumed

0.00

Ignored info

0.32 bits

Model

Gaussian NB

High correlation breaks the naive assumption. NB can still classify well, because it only needs the right class to score highest, not calibrated probabilities.

Drag the correlation slider between −0.9 and 0.9. The naive assumption says the features are independent within each class, so P(x_1, x_2 | class) = P(x_1 | class) · P(x_2 | class). Correlated features break this, and the model then counts the same evidence twice, which pushes its probabilities toward 0 or 1. The ignored info card measures what the product of marginals loses: for two Gaussian features with correlation ρ, the mutual information is −½ log₂(1 − ρ²), which is 0.32 bits at ρ = 0.6 and 1.20 bits at ρ = 0.9. Domingos and Pazzani showed in 1997 that naive Bayes can still be the optimal classifier under many such violations.

Step 4 of 6: Combining Features

Multiplying independent likelihoods

Feature 1 (x₁) — spam μ=2, ham μ=-11.00

Feature 2 (x₂) — spam μ=1.5, ham μ=-0.50.80

Naive Bayes: multiply independent likelihoods

P(x₁|spam)·P(x₂|spam) = 0.228 × 0.346 = 0.0789
P(x₁|ham)·P(x₂|ham) = 0.038 × 0.133 = 0.0050

From joint likelihoods to posterior (equal priors)

P(spam|x) = P(x|spam)·P(spam) / P(x)
= 0.0789 × 0.50 / (0.0395 + 0.0025)
= 0.940

Joint spam

0.0789

Joint ham

0.0050

P(spam|x)

0.940

Verdict

Spam

Both features together favor spam (94.0%). The naive assumption lets the model multiply P(x₁|spam) × P(x₂|spam) instead of modeling the joint distribution. Move x₂ toward ham's mean (-0.5) to make the features disagree.

Drag the Feature 1 and Feature 2 sliders and watch the joint likelihoods. With hundreds of features, multiplying small probabilities underflows floating-point numbers, so implementations add logs instead: log P(class) + Σ log P(x_i | class). Each feature then adds its own score to each class, which is why one strongly informative feature can outweigh several weak ones. For word-count and binary features, this sum makes naive Bayes a linear classifier in log space. The Gaussian version here needs only 2 parameters per feature per class.

Step 5 of 6: Decision Boundary

Where posteriors are equal

Prior P(class A / spam)0.50

Prior ratio

1.00

Boundary shift

0.00

Accuracy

99%

Log prior

0.00

With equal priors, the boundary depends only on the likelihoods. It sits where P(A|x) = P(B|x).

Drag the prior P(class A / spam) slider from 0.1 to 0.9 and watch the dashed boundary move. The boundary is where log P(x | A) + log P(A) equals log P(x | B) + log P(B). Changing the prior adds the constant log(P(A)/P(B)) to one side, which shifts the boundary without changing its shape. Because the two classes have different variances here, the boundary is a curve; with equal variances it would be a straight line. Adjusting the prior is one way to trade false positives against false negatives.

Step 6 of 6: Spam or Ham?

Text classification example

Words in this email:

P(free|spam)0.80

P(money|spam)0.70

P(meeting|spam)0.10

P(hello|spam)0.30

P(prize|spam)0.75

P(spam|words)

0.967

P(ham|words)

0.033

Top spam clue

free

Verdict

SPAM

The words in this email together favor spam (96.7%). The strongest spam clue is "free", with a spam/ham likelihood ratio of 16.0. Remove spam-heavy words or add "meeting" to flip the verdict. In practice each P(word|class) is estimated from word counts with Laplace (+1) smoothing, so no word ever has probability exactly 0 and single-handedly vetoes a class.

Toggle the word buttons and drag the five P(word|spam) sliders while the ham probabilities stay fixed. With the defaults, an email containing free, money, and hello, the posterior is about 96.7% spam, and "free" is the top clue with a ratio of 16. This demo is a Bernoulli model: it multiplies P(word | class) for each word present and 1 − P(word | class) for each vocabulary word that is absent. A multinomial model counts repeated words instead. Real filters estimate these probabilities from word counts in labeled email, with smoothing so that no word gets probability zero.

What is a naive Bayes classifier?

Naive Bayes is a family of probabilistic classifiers that apply Bayes' rule with one simplifying assumption: given the class, every feature is independent of the others. Under that assumption, the likelihood of a whole input is a product of one-feature likelihoods, each easy to estimate from counts or from a per-class mean and variance. The prediction is the class with the highest posterior probability.

Naive Bayes trains in a single pass and needs little data, so it is a standard baseline for text classification tasks such as spam filtering, sentiment analysis, and topic labeling. Variants match the feature type: Gaussian naive Bayes for continuous values, multinomial naive Bayes for word counts, and Bernoulli naive Bayes for word presence or absence.

Key ideas

Naive Bayes picks the class with the highest posterior, P(class | x) ∝ P(x | class) · P(class).
It assumes features are independent given the class, so the likelihood is a product of one-feature terms.
Training only estimates per-class statistics such as means, variances, or word frequencies.
Implementations add log-probabilities instead of multiplying probabilities, to avoid underflow.
Changing the prior shifts the decision boundary without changing its shape.
Its probabilities are often overconfident, even when its class predictions are accurate.

Common questions

Why is naive Bayes called naive?

It assumes every feature is independent of every other feature once the class is known. That is rarely true: in an email, "free" and "prize" tend to appear together. The assumption is a deliberate simplification that makes the model fast and data-efficient.

What is Laplace smoothing in naive Bayes?

If a word never appears in the spam training set, its estimated P(word | spam) is zero, and a single occurrence in a new email would force the spam probability to zero. Laplace (add-one) smoothing adds 1 to every count before dividing, so every word keeps a small nonzero probability. The added amount, often called alpha, can be tuned.

Is naive Bayes generative or discriminative?

Generative. It models how each class produces features, P(x | class), along with the class prior, and classifies with Bayes' rule. Logistic regression is its discriminative counterpart. Ng and Jordan (2001) found that naive Bayes approaches its best error with fewer training examples, while logistic regression usually reaches a lower error once data is plentiful.

Are naive Bayes probabilities reliable?

Usually not as probabilities. Correlated features are counted more than once, which pushes posteriors toward 0 or 1. When calibrated probabilities matter, Platt scaling or isotonic regression can be fitted on held-out data to correct them.

Related explainers

Bayes' Theorem — Updating beliefs with evidence
Probability & Distributions — Spam, densities, and the math of uncertainty
Text Classification & Sentiment — Teach a model to assign labels to text — from spam detection to sentiment analysis
Bag of Words — Turn text into numbers — count word occurrences to create the simplest document representation in NLP
MLE & MAP Estimation — Maximum likelihood and maximum a posteriori — why training works
Class Imbalance & Resampling — SMOTE, class weights, and metrics beyond accuracy

References

Last updated October 25, 2026