L1 vs L2: why only one sets weights to exactly zero

L1 and L2 regularization both shrink a model's weights, the numbers it learns for each input feature. Only L1 switches features off completely, and the reason is a corner.

1. 1
   Each weight sets how much its feature counts. The loss measures how badly the model fits the training data. Regularization reduces overfitting, where a model memorizes its training data and fails on new data. It adds a penalty to the loss: an extra cost for large weights. L2, also called ridge, penalizes the sum of squared weights. L1, also called the lasso, penalizes the sum of absolute weights. The same penalty can be read as a cap on how large the weights can get. With two weights, that cap is a shape: a diamond for L1, a circle for L2.
2. 2
   Each oval is a loss contour. Every point on one oval gives the same loss, the way a line on a map joins points of equal height. The white dot is the best fit with no penalty, and the ovals grow as the loss rises. The model settles where the smallest oval first touches the shape. That point has the lowest loss among the allowed weights.
3. 3
   The diamond's corners sit on the axes, and a growing oval often reaches a corner before any edge. On a corner, one weight is exactly 0. In the left panel the oval meets the top corner, so w₁ = 0. The circle has no corners. Its edge curves smoothly, so the oval touches it where both weights are shrunk but nonzero. In the right panel, w₁ is 0.35.