How Regularization Works
The Overfitting Problem
When the model memorizes instead of learning
An overfit model memorizes training noise — its training loss is tiny but it fails on new data. A good fit captures the true pattern and generalizes.
Step 1 of 6: The Overfitting Problem
When the model memorizes instead of learning
An overfit model memorizes training noise — its training loss is tiny but it fails on new data. A good fit captures the true pattern and generalizes.
Step 2 of 6: L2 Regularization
Penalizing large weights to keep things smooth
With λ ≈ 0, no regularization — weights are large and varied, causing wiggly overfitting. Train loss is near zero but test loss is high.
Step 3 of 6: L1 Regularization
Pushing weights to exactly zero — built-in feature selection
With λ ≈ 0, no regularization — all weights are active, the model overfits the noisy training data.
Step 4 of 6: Dropout
Randomly silencing neurons during training
Dropout randomly silences neurons each training step, forcing the network to not rely on any single path. The weight chart shows how dropped neurons have their effective weight zeroed out. At test time, all neurons are active but weights are scaled down. Currently dropping 3 of 9 hidden neurons.
Step 5 of 6: Early Stopping
Stop training before overfitting kicks in
Training hasn't progressed much yet. Both train and val loss are still high. Press Play or drag the slider.
Step 6 of 6: Comparison
How each technique shapes the fit
No Reg: Overfits — memorizes noise. Train loss is near zero but test loss is high — the classic sign of overfitting.