The recommendation funnel in 1 picture

A recommendation feed has about 100 ms to choose a few items for one user out of a catalog of millions. The model that judges items best is far too slow to run on all of them, so the feed narrows the catalog in stages, and only the last stages see few enough items to afford a careful look at each.

1. 1
   The top band is the whole catalog, 1 million items. Retrieval cuts it to 200 candidates in about 10 ms. It uses a two-tower model: one network turns the user into a vector, another turns each item into a vector, and their dot product predicts how much the user will like the item. Item vectors are computed ahead of time and stored in a vector index, which finds the 200 largest dot products without checking every item.
2. 2
   The middle band is ranking. A richer model, the ranker, scores each of the 200 candidates using features retrieval cannot afford, such as recent activity and item freshness. At 0.1 ms per item that costs 20 ms. Run on the whole catalog, the same ranker would need 1 million × 0.1 ms = 100 s for one request, a thousand times the budget.
3. 3
   The bottom band is the feed. A re-ranker reorders the top of the ranked list so the 10 slots are not all near-copies of one another. Maximal marginal relevance (MMR) does this by trading each item's relevance against its similarity to items already picked. It costs about 1 ms. The whole request takes 31 ms, leaving room for network calls and feature lookups.