Overfitting in 1 picture
Overfitting is when a model learns its training data so closely, noise included, that it does worse on data it has never seen. One slider over 12 noisy points shows the moment it happens.
2 min read · Lensa ML
- The blue dots are 12 training points from the curve y = 0.5x² − 2x + 5, plus noise: small random errors added to each point. The pink diamonds are test points from the same curve, which the model never trains on. The yellow line is a polynomial fit, a sum of powers of x that each get their own coefficient, and the slider sets its degree, the highest power of x it can use. A higher degree means more coefficients, so the line can bend more. Error is mean squared error (MSE), the average of the squared gaps between the points and the line.
- At degree 1 the fit is a straight line. It misses the bend, so both errors are high: 10.91 on the training points and 16.47 on the test points. That is underfitting, a model too simple for the pattern. At degree 2, the same shape as the true curve, training error falls to 0.37 and test error to 0.32, the lowest test error on the slider.
- Keep raising the degree. Training error keeps falling and reads 0.00 by degree 9. The line now wiggles to pass through every training point, and it swings away hardest at the right edge, where no training point holds it down. Test error climbs to 5.56 at degree 9 and 28.77 at degree 10. The explainer's slider goes one step further: at degree 11 the fit has 12 coefficients for 12 points, enough to pass through each one exactly.
Takeaway
A low training error alone proves nothing. Keep some data the model never trains on, and choose the setting where the error on that data is lowest. When the gap between the two errors grows, try more data, regularization (a penalty on large weights), or a simpler model.
Go deeper · interactive
Overfitting & Underfitting
Drag the model complexity slider and watch test error climb as the curve chases every point.
Open the explainer