Train, Validation & Test Sets

A simple model captures the general trend but doesn't memorize individual points. The training fit is moderate.

A 80/20 split is standard. The test set gives an unbiased estimate of how the model will perform on new, unseen data.

The validation set is your practice exam. You use it to tune hyperparameters (like model complexity) without touching the test set. A 15% validation split is typical. The test set stays locked away until final evaluation.

Good fit! Both losses are low and the gap is small (0.30). The validation set helps us find this sweet spot without touching the test set.

K-fold cross-validation rotates through 5 folds so every data point is validated exactly once. 5-fold is the most popular choice. It balances bias and variance in the performance estimate. Std of 3.7% tells you how stable the estimate is.

The golden rule: never touch the test set until your final evaluation. The workflow is: train on training data, tune on validation data, and only evaluate once on test data.