Diffusion models in plain words: add noise, learn to remove it

A diffusion model, the kind behind Stable Diffusion, is a generator: a model that makes new images. It learns from real images buried in noise, random values mixed into every pixel, or in Stable Diffusion into a compressed copy of the image. Its one skill is spotting which part of a noisy image is noise.

1. 1
   The clip starts from a toy image, an 8 by 8 grid of colored pixels, at time step t = 0. The bar below ends at t=T, which is 20 here. As t rises, each pixel's color is blended toward a random, nearly gray color. The σ(t) readout is the noise share of the blend, t/20, and Signal is the share of the original color left. At t = 10 the blend is half and half, and at t = 20 only random grays remain.
2. 2
   The slider then runs back to 0, replaying the same blends in reverse. No model runs in this clip. The noising half is the forward process, and nothing in it is learned. Real models use Gaussian noise, random values drawn from a bell curve, and a noise schedule that sets how fast the signal fades. The demo fades it in a straight line over 20 steps, and the DDPM paper used 1,000 steps.
3. 3
   In training, take a real image, pick a random step t, add that much noise, and ask a neural network, a model built from many adjustable numbers, to predict the exact noise added. The loss, the score for how wrong a guess is, is the squared difference between the true noise and the guess. To make an image, the trained model starts from fresh random noise and removes a little predicted noise at each step, back to step 0.