Mixed precision, distributed training, scaling laws, and distillation
Training in 16-bit without losing small gradients
Splitting training across many GPUs
How loss falls with model size, data and compute
Training a small model on a large model's outputs