1Classification vs Regression 2Loss Functions 3Gradient Descent 4Feature Scaling 5Hyperparameters 6Epochs & Batching

Lensa ML

Classification vs Regression

Step 1 of 6: Two Kinds of Predictions

Categories vs numbers

Task

Regression

Output

Number

Example

$180k

Regression predicts a continuous number, such as a house price or tomorrow's temperature. The output can be any real number.

Toggle between Regression and Classification and compare the two plots. The same inputs can feed either task. House features can predict the sale price (regression) or whether the house sells above asking (classification). What decides the task is the type of the target. Some targets sit in between. Star ratings from 1 to 5 are ordered categories, so they can be treated as five classes, as a number, or with ordinal regression, which respects the order without assuming equal gaps.

Step 2 of 6: Regression

Predicting continuous values

Slope40

Intercept50

MSE

296

Slope

40

Intercept

50

Regression fits a line ŷ = 40x + 50. The red lines show residuals, the error for each point. MSE = 296 measures the average squared error. That is a good fit.

Drag Slope (−20 to 80) and Intercept (−50 to 200) to fit the 25 points. The data comes from y = 50 + 40x plus random noise of up to ±30, so the starting line is the true one, with an MSE of about 296. The best line on the sliders, slope 39 and intercept 58, scores about 263, because it shifts slightly to fit this particular noise. No line can do much better on new data, since noise of that size has a variance of 30²/3 = 300.

Step 3 of 6: Binary Classification

Yes or no

Probability curve

Scatter plot

Decision thresholdθ = 0.50

Accuracy

83%

True pos

11

False pos

1

Threshold

0.50

The sigmoid curve (top) maps Feature 1 to a probability. The vertical boundary (bottom) shows where the decision splits in feature space. Move the threshold and watch the boundary shift. Yellow-ringed points are misclassified.

Drag the Decision threshold between 0.05 and 0.95. The model is P = σ(2(x − 2.5)), so the boundary sits at x = 2.5 + ln(θ / (1 − θ)) / 2. At θ = 0.5 the boundary is at 2.5 and 25 of 30 points are right. Accuracy peaks at 28 of 30 for θ between 0.34 and 0.38, with the boundary near 2.2, because a few class-1 points sit left of the middle. At θ = 0.8 the boundary moves to 3.19 and accuracy falls to 73%. Raising it trades false positives for false negatives. Cancer screening uses a low threshold, since a missed case costs more than a false alarm.

Step 4 of 6: Multi-class Classification

More than two categories

Class A

99.1%

Class B

0.0%

Class C

0.9%

TemperatureT = 1.00

Predicted

Class A

Actual

Class A

Confidence

99%

Softmax turns raw scores into probabilities across 3 classes. At moderate temperatures, the probabilities stay close to plain softmax (T = 1). Click any point to see its class probabilities.

Click any point, then drag Temperature from 0.10 to 5.00. Each class has its own linear score, and a point belongs to whichever score is highest. Every boundary between two classes is a straight line, so each class region is a convex polygon. These three scores classify 42 of the 45 points correctly. Temperature never changes which class wins, only how confident the model looks: the point selected at the start, from Class A, gets 99.1% at T = 1 and 64.6% at T = 5.

Step 5 of 6: Choosing the Right Loss

MSE vs Cross-Entropy

Model says P(class=1) = … (correct answer is 1)P = 0.70

MSE loss

0.090

CE loss

0.357

CE / MSE

4.0×

MSE treats classification like guessing a number: it only cares how far the probability is from 1. Cross-entropy (CE) measures how surprised the model is by the true label. Drag the slider left to see CE explode while MSE stays mild.

Drag P(class=1) from 0.01 to 0.99; the true label is 1. The gradient matters more than the loss value. With a sigmoid output, the gradient of cross-entropy with respect to the logit is P − 1. At P = 0.01 that is −0.99, a strong push. The MSE gradient picks up an extra factor of P(1 − P) from the sigmoid: 2(P − 1) × P(1 − P) ≈ −0.02 at P = 0.01. A confidently wrong model trained with MSE barely moves, which is why classifiers use cross-entropy.

Step 6 of 6: When to Use Which

A decision guide

Task type

Regression

Loss

MSE

Output

ŷ ∈ ℝ

House Price: This is regression: predict a continuous value. Use a linear output layer and MSE or MAE loss.

Click through the five problems to see the output layer and loss each one needs. MAE suits targets with outliers, since it grows linearly with the error instead of squaring it. Huber loss blends the two. House prices are often modeled as log(price), because errors grow with the price and the log makes them comparable across cheap and expensive homes. A problem where several labels can be true at once, such as tagging a photo, is multi-label: it uses one sigmoid per label with binary cross-entropy, not a softmax.

What is the difference between classification and regression?

Classification and regression are the two main kinds of supervised learning. Classification predicts a category, such as spam or not spam. Regression predicts a number on a continuous scale, such as a price or a temperature. The difference lies in the target the model learns to predict, not in the inputs or the model family.

The choice decides the rest of the setup: the output layer, the loss function, and the metrics. Regression models usually end in a linear output trained with mean squared error. Classifiers end in a sigmoid or softmax trained with cross-entropy. Credit scoring, medical diagnosis, and image recognition are classification problems. Demand forecasting, price estimation, and predicting a patient's length of stay are regression.

Key ideas

Classification predicts a category; regression predicts a continuous number.
The task type comes from the target variable, not from the input features.
Regression usually uses a linear output with MSE or MAE loss.
Binary classification uses a sigmoid output with binary cross-entropy.
Multi-class classification uses a softmax output with categorical cross-entropy.
A classifier's threshold trades false positives against false negatives.

Common questions

Is logistic regression a classification or regression method?

It is a classification method. The name comes from the fact that it fits a linear regression to the log-odds of the positive class, then passes the result through a sigmoid to get a probability. Thresholding that probability gives the class.

Can one problem be framed as either classification or regression?

Often, yes. Age can be predicted as a number or as age brackets, and sentiment can be a score from −1 to 1 or the classes positive, neutral, and negative. Regression keeps the order and distance between values. Classification is easier to interpret when only a few outcomes lead to different decisions.

Which metrics are used for classification and for regression?

Regression is measured with MAE, RMSE, or R², which compare predicted and true numbers. Classification uses accuracy, precision, recall, F1 score, and ROC AUC. On imbalanced data accuracy can mislead, because a model that always predicts the majority class scores well.

Can the same algorithm do both classification and regression?

Most can. Decision trees, random forests, gradient boosting, k-nearest neighbors, and neural networks all come in classifier and regressor versions. The model structure stays the same, while the output and the loss change: a leaf predicts a mean instead of a majority class, or a network ends in a linear unit instead of a softmax.

Related explainers

Loss Functions — How networks measure their mistakes
Softmax, Logits & Temperature — Turning raw scores into probabilities
Model Evaluation & Metrics — Precision, recall, F1, ROC curves, and regression metrics
The Perceptron — The simplest neural network — one neuron, one decision boundary
Decision Trees & Splitting — Recursive partitioning, Gini impurity, and pruning
Class Imbalance & Resampling — SMOTE, class weights, and metrics beyond accuracy

References

Last updated October 25, 2026