How Backpropagation Works
The Prediction Was Wrong
The network guessed Cat = 0.62 — but we wanted 1.0
The forward pass ran. The network says Cat = 0.57 but the answer should be 1.0. How does the network fix itself? That's backpropagation: sending the error backward through the network so every weight learns from the mistake.
Step 1 of 6: The Prediction Was Wrong
The network guessed Cat = 0.62 — but we wanted 1.0
The forward pass ran. The network says Cat = 0.57 but the answer should be 1.0. How does the network fix itself? That's backpropagation: sending the error backward through the network so every weight learns from the mistake.
Step 2 of 6: Measuring the Mistake
The loss function puts a number on how wrong we are
The loss function measures how wrong we are. Take each output's error, square it (so bigger mistakes get punished more), and add them up. This single number is what backprop tries to minimize.
Step 3 of 6: The Chain Rule
How a tiny weight change ripples through to the output
The chain rule is the key insight. To find out how much one weight affected the final loss, multiply together the local derivatives at each step along the path. Each neuron only needs to know its local math, then pass the blame backward.
Step 4 of 6: Gradients Flow Backward
Watch error signals propagate from output back to input
Each neuron's delta (δ) represents its share of the blame. The bigger the delta, the more that neuron contributed to the error. Deltas start at the output and propagate backward.
Step 5 of 6: Nudging the Weights
Each weight gets a personalized correction
new weight = old weight - learning rate × gradient. A big gradient means that weight had a big effect on the error, so it gets a bigger correction.
Step 6 of 6: Repeat Until Learned
Many rounds of backprop — watch the network master the cat