Two measurements per animal, and where the cats sit
cat dog
Body weight4.5 kg
Snout length2.5 cm
Input x₁ = (log₂ kg − 2.5) / 1.25
-0.26
Input x₂ = (snout − 5) / 3
-0.83
Closest training animal
cat
A 64×64 colour photo instead
12,288 inputs
A network only ever sees numbers. Here each animal is two measurements: body weight and snout length, from 14 cats and 14 dogs (illustrative values, not a real dataset). The cross is your animal: 4.5 kg with a 2.5 cm snout becomes the inputs x₁ = -0.26 and x₂ = -0.83, rescaled so both sit in a similar range. Its closest training animal is a cat. Notice where the cats are: in a pocket, with lighter toy dogs on one side and heavier dogs on the other. A real image network would get raw pixels, 12,288 numbers for a small colour photo, and learn its own measurements in its hidden layers. Two hand-picked inputs keep every number on screen.
Drag the two sliders to place a new animal, and watch its raw measurements turn into the two numbers the network receives. The inputs are rescaled (log of the weight, then centred) so that both features cover a similar range, which makes training much easier. Real image networks don't start from hand-picked measurements: a convolutional network reads raw pixel values and learns its own features in its early layers. Two inputs are used here so that the whole input space fits on one flat plot.
Step 2 of 6: One Neuron Draws One Line
Weights tilt it, the bias slides it
cat dog misclassified
Shading = the neuron's output σ(z): amber says cat, blue says dog. Click an animal to inspect it.
A neuron multiplies each input by a weight, adds a bias, and squashes the total with a sigmoid: σ(w₁x₁ + w₂x₂ + b). The white line is where z = 0, so the output is exactly 0.5 there. The weights tilt the line and the bias slides it. Right now 22 of 28 animals land on the correct side. Try to beat 25: you can't, because the cats sit between two groups of dogs and a straight line only splits the plane in two.
Drag the weights and the bias and watch the white line move: the weights tilt it, the bias slides it. A neuron computes σ(w₁x₁ + w₂x₂ + b), so its output is exactly 0.5 on the line z = 0 and moves towards 1 or 0 on either side. However you set the three numbers, one neuron splits the plane with one straight line, and the best any line can do here is 25 of 28 animals. The same limit stalled early work on the perceptron (Rosenblatt, 1958), whose single neuron can't learn patterns such as XOR.
Step 3 of 6: Hidden Neurons Combine Lines
Two lines fence off the cats
cat dog misclassified
Selected cat: hidden 1 · hidden 2
1.00 · 0.00
Output P(cat)
1.00
Animals classified correctly
28 / 28
This network has two hidden neurons, each an ordinary neuron like the last step, each with its own line. The output neuron doesn't see the measurements at all: it sees only the two hidden outputs, and it says cat when hidden 1 is on and hidden 2 is off. The two lines together fence off the pocket where the cats live, and 28 of 28 animals now land on the right side. Combining simple pieces into a shape no single piece could make is the whole reason for hidden layers. These weights weren't set by hand: training found them, as step 5 shows.
Switch between the three neurons. Each hidden neuron is an ordinary neuron with its own line, and on its own it would misclassify several animals. The output neuron only sees the two hidden outputs, and the four boxes show its answer for each on/off combination: it says cat for just one of them. That is how a hidden layer builds a new feature, "inside the pocket", out of simpler ones. The activation matters: without the sigmoid between layers, the two lines would collapse back into one weighted sum, and the network into a single line.
That's the whole forward pass: two rows of weighted sums and squashes, h = σ(Wx + b) and then P = σ(v·h + c). For 4.5 kg and a 2.5 cm snout the network says cat (P(cat) = 1.000). Both hidden neurons agree it's inside the pocket. Drag the sliders and the answer flips as the animal crosses a hidden neuron's line. A real network repeats the same two operations over many more layers and neurons, written as matrix multiplications.
Step through the layers, then drag the sliders and watch the numbers change. The forward pass is the network's whole prediction: for each layer, a weighted sum of the previous layer's outputs plus a bias, then the activation. Written with matrices that is h = σ(Wx + b), which is how frameworks run it on thousands of examples at once. A trained model in production only ever runs this pass. Training adds a backward pass for the gradients, which costs roughly twice as much.
Step 5 of 6: Learning From Many Examples
Train it, then test it on animals it never saw
Filled = training animals, hollow = 12 held-out animals the network never trains on.
Epoch
0
Training loss
0.709
Correct on training set (28)
50%
Correct on 12 held-out animals
50%
The network starts from random weights. Each epoch is one pass over all 28 training animals: a forward pass for each, the total loss, backpropagation for every weight's gradient, and one small update. Press play or step one epoch at a time. That last number is the one that matters: learning means finding weights that work on new examples.
Press play, or step one epoch at a time, and compare the two loss curves. An epoch is one pass over every training example: a forward pass for each, the average loss, backpropagation for each weight's gradient, and one update. Then switch to training on one cat photo. The training loss still falls to zero, but the held-out loss climbs, because the easiest way to fit one cat is to call everything a cat. A held-out set the network never trains on is how real projects check that a model has learned rather than memorised.
Step 6 of 6: How Many Hidden Neurons?
Too few can't draw it; more make it easier to find
training run (random start):
cat dog misclassified
Weights and biases
9
This run: training animals right
23 / 28
This run: held-out right
83%
Runs that separate every animal
2 / 5
Each run trains the same architecture from a different random start for 1,500 epochs. Two hidden neurons are enough in principle, and 2 of 5 runs find a pair of lines that fences off every cat. The other runs get stuck at the same score as a single line, with the two lines doing much the same job: a local minimum that gradient descent can't climb out of. That's one reason real networks are much wider than the bare minimum. The cost is more weights, 9 here. With only 28 examples, a far bigger network would also have room to fit the quirks of individual animals, which is the overfitting problem.
Compare the five runs for each width. One hidden neuron can only draw one line, so no run gets past 23 of 28. Two neurons can draw the right shape, but only some random starts find it; the rest settle in a local minimum. With three or four neurons every run succeeds, because the extra neurons give gradient descent more routes to a good solution. That is one reason real networks are much wider than the bare minimum. With very few examples, though, a far bigger network also has room to fit the quirks of individual examples, which is overfitting.
What is a neural network?
A neural network is a function built from layers of simple units called neurons. Each neuron takes a weighted sum of its inputs and passes it through an activation function. Stacking layers of these neurons lets the network represent complicated relationships, and training adjusts the weights until the network's outputs match a set of examples.
The network in this explainer is small enough to follow by hand: 2 inputs (an animal's body weight and snout length), one hidden layer of 2 neurons, and 1 output, the probability that the animal is a cat, with a sigmoid in every neuron. That is 9 weights and biases. The networks behind image recognition and language models work on the same principles, with millions or billions of weights.
Key ideas
A neuron computes a weighted sum of its inputs and applies an activation function.
Layers of neurons build features out of combinations of earlier features.
Without non-linear activations, any stack of layers collapses into one layer.
The forward pass turns inputs into a prediction; it is all a trained model runs.
Training repeats: predict, measure the loss, backpropagate, update the weights.
Learning means finding weights that work across many examples, not just one.
Common questions
What is the difference between weights and biases?
A weight scales one input to a neuron. A bias is a single extra number added to the neuron's sum, which lets the neuron activate even when all its inputs are zero and shifts its decision boundary away from the origin. Both are learned during training.
How many layers does a neural network need?
In theory, one hidden layer with enough neurons can approximate any continuous function on a bounded input range (the universal approximation theorem). In practice, deeper networks learn the same functions with far fewer neurons, which is why modern networks have many layers.
Why are they called neural networks?
The original idea was loosely inspired by biological neurons, which fire when their combined input signals pass a threshold. Artificial neurons are much simpler than real ones, and modern networks are designed for what trains well on computers, not to model the brain.
Which activation functions are used instead of the sigmoid?
Hidden layers in modern networks mostly use ReLU or close relatives such as GELU, which keep gradients from shrinking as quickly in deep networks. The sigmoid is still common for outputs that represent a yes/no probability.