Feature Scaling

Both features have similar ranges, so the scatter plot looks well-distributed. Each feature contributes equally to distance calculations and gradient updates.

With balanced scales, loss contours are nearly circular. Gradient descent takes a direct path to the minimum with high efficiency. Each step moves meaningfully toward the goal.

Min-max normalization squeezes every feature into [0, 1]. The left plot shows the raw scale problem — salary (300-2000) dominates age (18-80). The right plot shows the fix: both axes now span [0, 1], revealing the true relationship between features.

Standardization uses mean and standard deviation instead of min and max. The left plot shows the raw scale distortion. The right plot shows the fix: both features are centered at zero with unit variance, revealing the true data structure.

Both models are stable at this small learning rate. The unscaled model converges slowly because the small eigenvalue (1) makes progress crawl along one axis, while the large eigenvalue (100) keeps the other axis barely moving. The scaled model converges evenly.

Uses gradient descent, so features on different scales cause uneven updates. Coefficients become hard to interpret. Always scale features before training this model type.