Precision vs recall: one threshold, two kinds of mistake

A spam filter can be wrong in two ways. It can flag a normal email, a false positive, or let a spam email through, a false negative. Its threshold, the cutoff score at or above which it flags an email, decides which of the two mistakes it makes more often.

1. 1
   The model gives each email a score from 0 to 1, and every email scoring at or above the threshold is flagged as spam. The gray curve shows how the scores of 700 normal emails spread out, centered on 0.35. The blue curve shows 300 spam emails, centered on 0.65. Both are bell-shaped normal curves with a standard deviation, their typical spread, of 0.12, so they overlap most around 0.5. The area under a curve counts emails.
2. 2
   A true positive is a spam email that gets flagged. Precision is the share of flagged emails that are spam: true positives divided by all flagged emails. Recall is the share of all spam that gets flagged. At a threshold of 0.4, the red area, normal emails right of the line, holds 237 false positives. The gold area, spam left of the line, holds only 6 false negatives. Recall is 98%, but precision is 55%: almost half of the spam folder is real mail.
3. 3
   Raise the threshold to 0.6 and the two areas swap sizes. False positives fall to 13 and false negatives rise to 102, so precision climbs to 94% and recall drops to 66%. Moving the line only trades one error for the other, because in the overlap both kinds of email get the same scores. Only a better model, whose curves sit further apart, cuts both errors at once.