How Loss Functions Work
Prediction vs Reality
Networks guess — loss measures how wrong they are
A loss function measures how far the network's prediction is from the true value. Drag the slider to see how the error changes as the prediction moves closer to the actual value.
Step 1 of 6: Prediction vs Reality
Networks guess — loss measures how wrong they are
A loss function measures how far the network's prediction is from the true value. Drag the slider to see how the error changes as the prediction moves closer to the actual value.
Step 2 of 6: Mean Squared Error
Squaring mistakes so big errors hurt more
MSE squares each residual so that large errors are penalized much more than small ones. The pink squares show the "squared" error visually. Adjust the slope to minimize the total area.
Step 3 of 6: Binary Cross-Entropy
The loss function for yes-or-no decisions
Binary cross-entropy uses -log(p) when the label is 1 and -log(1-p) when it's 0. A confident wrong prediction produces a huge loss that pushes the network to correct itself.
Step 4 of 6: Categorical Cross-Entropy
Extending to multiple classes
Categorical cross-entropy only cares about the probability assigned to the true class. The loss is -log(p_true) -- it doesn't matter how the remaining probability is split among wrong classes.
Step 5 of 6: The Loss Landscape
A terrain of peaks and valleys
The loss landscape is a terrain where every point represents a set of weights, and the height is the loss. The yellow arrow shows the direction of steepest descent -- where gradient descent wants to go.
Step 6 of 6: Loss Guides Learning
Gradients flow from loss back through the network
Forward pass: data flows through the network to produce a prediction. The loss measures the error, then gradients flow backward, telling each weight how to adjust. This cycle repeats thousands of times during training.