Overfitting & Underfitting
Underfitting
When the model is too simple to capture the pattern
A linear model cannot capture the quadratic pattern in the data. Both training and test errors remain high -- this is underfitting. The model has high bias and low variance.
Step 1 of 6: Underfitting
When the model is too simple to capture the pattern
A linear model cannot capture the quadratic pattern in the data. Both training and test errors remain high -- this is underfitting. The model has high bias and low variance.
Step 2 of 6: Overfitting
When the model captures noise instead of signal
A high-degree polynomial passes through every training point (near-zero train error) but wiggles wildly between them. On new test points, the error explodes -- this is overfitting. The model memorized noise instead of learning the pattern.
Step 3 of 6: The Sweet Spot
Just enough complexity to generalize
Drag the slider to find the sweet spot. At degree 2-3, the model captures the true quadratic pattern without memorizing noise. Both train and test errors are low -- the model generalizes well.
Step 4 of 6: Bias vs Variance
Two sources of error, one tradeoff
Bias = consistently wrong (off-center). Variance = inconsistent (scattered). The goal is low bias + low variance: predictions that are both accurate and consistent.
Step 5 of 6: Model Complexity Curve
The U-shaped test error
The classic U-shaped test error curve. Training error always decreases with complexity, but test error hits a minimum then rises. The gap between curves is the generalization gap.
Step 6 of 6: What To Do About It
Practical strategies for finding the balance
Diagnose first, then act. If both errors are high, your model is too simple. If only test error is high, your model is too complex. Always use a validation set to catch these issues during training.