What is model calibration in machine learning?
A model is calibrated when its predicted probabilities match how often it is right. Among all predictions made with 80% confidence, a calibrated classifier is correct about 80% of the time. Accuracy measures whether the top answer is right, while calibration measures whether the stated confidence can be trusted, and a model can score well on one and badly on the other.
Calibration matters wherever a probability drives a decision. Medical triage systems refer low-confidence cases to a doctor, fraud filters block a payment above a risk score, and self-driving stacks weigh detections by confidence. Ensembles and cost-sensitive rules both assume the probabilities are honest. Large language models raise the same question when they answer a question or abstain, since a confidence score is only useful if 90% means right nine times in ten.
Key ideas
- A calibrated model is right about p of the time on the predictions it makes with confidence p.
- A reliability diagram bins predictions by confidence and plots accuracy against the diagonal.
- Expected calibration error is the gap in each bin weighted by the share of predictions in it.
- Modern deep networks tend to be overconfident even when their accuracy is high.
- Temperature scaling divides the logits by one fitted constant and never changes the predicted class.
- Platt scaling and isotonic regression learn a mapping from raw scores to calibrated probabilities on held-out data.
Common questions
Does calibration change a model's accuracy?
Temperature scaling, Platt scaling and isotonic regression are all monotonic, so the top class and the ranking of examples stay the same and accuracy is unchanged. Isotonic regression can create ties, which slightly affects ranking metrics such as AUC. Methods that change the training itself, like label smoothing or mixup, can move both accuracy and calibration.
Is a lower log loss the same as better calibration?
Not exactly. Log loss and the Brier score are proper scoring rules, and both reward calibration and sharpness, meaning confident predictions that turn out right. A model can lower its log loss by being more accurate while staying miscalibrated. ECE isolates calibration, but a model that predicts the base rate for every example has zero ECE and no usefulness.
Are large language models calibrated?
Kadavath et al. found that large pretrained models are fairly well calibrated on multiple-choice questions when the answer options are presented clearly. Fine-tuning for chat with human feedback often makes them more overconfident, a drop the GPT-4 technical report also showed. Confidence stated in words, such as "I am 90% sure", is usually less reliable than the probability the model assigns to its answer tokens.
Do I need to recalibrate after the data changes?
Yes, calibration holds only for data like the set it was fitted on. Under distribution shift, accuracy usually drops faster than confidence, so a model calibrated in the lab becomes overconfident in production. Recalibrating on recent labeled data is cheap because only one or two parameters change.
Related explainers
- Model Evaluation & Metrics — Precision, recall, F1, ROC curves, and regression metrics
- Softmax, Logits & Temperature — Turning raw scores into probabilities
- Loss Functions — How networks measure their mistakes
- Class Imbalance & Resampling — SMOTE, class weights, and metrics beyond accuracy
- Sampling: Temperature, Top-k & Top-p — How a language model picks its next token
- RLHF — Align language models with human preferences using reward models and policy optimization
References
- Guo et al. (2017), "On Calibration of Modern Neural Networks"
- Naeini, Cooper & Hauskrecht (2015), "Obtaining Well Calibrated Probabilities Using Bayesian Binning"
- Niculescu-Mizil & Caruana (2005), "Predicting Good Probabilities with Supervised Learning"
- Kadavath et al. (2022), "Language Models (Mostly) Know What They Know"
Last updated