Lensa ML
Lensa ML

The Perceptron

Step 1 of 6: A Single Neuron

The building block of neural networks

0.50x₁×0.60.80x₂×0.4Σ0.620σ()0.650ŷ
x₁0.50
x₂0.80
Weighted sum
0.620
Sigmoid(Σ)
0.650
Prediction
1 (yes)

A perceptron is the simplest neural network: one neuron. It multiplies each input by a fixed weight (×0.6, ×0.4), sums them to get 0.620, then squashes through sigmoid to produce 0.650. The output is ≥ 0.5, so the neuron fires and predicts yes.

Drag the x₁ and x₂ sliders between −1 and 1 and find where the prediction flips. With weights 0.6 and 0.4, it flips when 0.6x₁ + 0.4x₂ crosses 0, so x₁ = −0.5 and x₂ = 0.7 give a sum of −0.02 and a prediction of 0. This neuron uses a sigmoid, which makes it the same model as logistic regression. Rosenblatt's original perceptron used a hard threshold. His Mark I machine read images through a 20×20 grid of photocells and stored its weights in motor-driven potentiometers.

Step 2 of 6: Weights & Bias

How a neuron computes its output

0.85x₁×0.600.72x₂×-0.40+b=0.10Σ + b0.322σ()0.580ŷ
w₁+0.60
w₂-0.40
bias+0.10
Weighted sum
0.322
Bias
+0.10
Output σ(Σ)
0.580

The neuron computes w₁x₁ + w₂x₂ + b = 0.322, then applies the sigmoid function. The bias shifts the sum up or down. Without it, the boundary must pass through the origin.

Move w₁, w₂, and bias between −2 and 2 while the inputs stay fixed at 0.85 and 0.72. A negative weight means a feature counts as evidence for 0. The bias acts as a negative threshold: w·x ≥ θ is the same test as w·x + b ≥ 0 with b = −θ. Many implementations treat the bias as one more weight attached to a constant input of 1, so the learning rule updates it exactly like the other weights.

Step 3 of 6: The Step Function

Threshold activation

zσ(z)θ=0.500
Input valuez = -1.20
Thresholdθ = 0.50
σ(z)
0.231
Threshold
0.50
Output
0

σ(z) = 0.231 < threshold 0.50 → output is 0. Not enough signal to fire.

Slide the input value and the threshold θ and watch the output jump between 0 and 1. Thresholding σ(z) at θ is the same as thresholding z at ln(θ / (1 − θ)). At θ = 0.5 that cutoff is z = 0, and at θ = 0.9 it moves up to z ≈ 2.20. The step function has a slope of zero everywhere except at the jump. Gradient descent gets no signal from it, which is why multi-layer networks replaced it with smooth activations such as the sigmoid.

Step 4 of 6: Decision Boundary

Where the neuron says yes or no

-3-3-2-2-1-1112233x₁x₂
w₁+1.00
w₂+0.80
bias+0.20
Equation
1.0x₁ + 0.8x₂ + 0.2 = 0
Correct
27/30

The decision boundary is the line where w₁x₁ + w₂x₂ + b = 0. Points on one side output 1, the other side 0. Adjust the weights and bias until every point lands on its own side.

Drag w₁, w₂, and bias to fit a line to the 30 AND points. The starting values get 27 of 30 right. The boundary w₁x₁ + w₂x₂ + b = 0 has slope −w₁/w₂, and the weight vector (w₁, w₂) points at right angles to it, toward the side labeled 1. The bias sets the line's distance from the origin, |b| / ‖w‖. Multiplying all three numbers by the same positive factor leaves the line, and every prediction, unchanged.

Step 5 of 6: Learning Rule

Teaching the perceptron

-3-3-2-2-1-1112233x₁x₂
Learning rateα = 0.100
Epoch
0
Correct
28/30
Errors
2

The perceptron learning rule: for each mistake, update the weights with w += α · error · x. Watch the boundary rotate and shift as mistakes are corrected.

Press Train or Step Once, then change the learning rate α between 0.01 and 0.50 and train again. Here the weights start at 0.1, 0.1, and bias 0. At α = 0.1, the line classifies all 30 points after 3 epochs. The error term is +1 or −1, so each update moves the line toward the misclassified point. Novikoff proved in 1962 that this rule stops after a finite number of mistakes on any linearly separable data. It stops at the first line that works, not the line with the widest margin.

Step 6 of 6: Limits of the Perceptron

Why one neuron isn't enough

-3-3-2-2-1-1112233x₁x₂
Dataset
AND
Accuracy
100%
Separable?
Yes

AND is linearly separable, so a single perceptron learns it perfectly. One straight line separates the green and red points.

Switch between AND, OR, and XOR. After 100 epochs, the perceptron gets AND and OR fully right but sits at 50% on XOR. Minsky and Papert analyzed this limit in their 1969 book Perceptrons, and the result helped cool research on neural networks for over a decade. A hidden layer fixes it: XOR equals AND applied to OR and NAND, and each of those three is linearly separable. Multi-layer networks only became practical to train once backpropagation spread in the 1980s.

What is a perceptron?

A perceptron is a single artificial neuron that classifies an input into one of two classes. It multiplies each input by a weight, adds a bias, and outputs 1 if the total is above a threshold and 0 otherwise. Geometrically, it splits the input space with a straight line, or a flat plane in higher dimensions.

Frank Rosenblatt introduced the perceptron in 1958, together with a learning rule that adjusts the weights from labeled examples. It is the building block of every neural network: each neuron in a modern model computes the same weighted sum, with a smooth activation in place of the hard threshold. Its main limit, that one neuron can only draw one straight boundary, is the reason networks have hidden layers.

Key ideas

  • A perceptron computes a weighted sum plus a bias and outputs 1 if it passes a threshold.
  • Its decision boundary is a straight line in 2D, or a hyperplane in more dimensions.
  • The weights set the boundary's orientation, and the bias shifts it away from the origin.
  • The learning rule changes the weights only when the perceptron makes a mistake.
  • On linearly separable data, the learning rule is guaranteed to converge.
  • A single perceptron cannot learn XOR; that takes at least one hidden layer.

Common questions

What is the difference between a perceptron and logistic regression?

Both compute a weighted sum of the inputs. A perceptron passes it through a hard threshold and only updates on mistakes. Logistic regression passes it through a sigmoid, outputs a probability, and is trained by minimizing cross-entropy loss, so it keeps adjusting even on correctly classified points.

What happens if the data is not linearly separable?

The perceptron learning rule never settles. The line keeps moving, because some point is always misclassified. Variants such as the pocket algorithm keep the best weights seen so far, and the averaged perceptron uses the average of all weights over training.

What is a multi-layer perceptron?

A multi-layer perceptron (MLP) is a feedforward neural network with one or more hidden layers between input and output. Despite the name, its neurons usually use smooth activations such as ReLU or sigmoid instead of the perceptron's step, so the network can be trained with backpropagation.

Is the perceptron still used today?

Rarely on its own, because logistic regression and support vector machines give better boundaries for the same linear model. Its weighted sum is still the core computation inside every neuron of a modern neural network, and the averaged perceptron remains a fast baseline for some text classification tasks.

Related explainers

References

  1. Rosenblatt (1958), "The perceptron: A probabilistic model for information storage and organization in the brain", Psychological Review
  2. Nielsen, "Neural Networks and Deep Learning", chapter 1
  3. Goodfellow, Bengio & Courville (2016), "Deep Learning", section 6.1: Example: Learning XOR

Last updated