Track 3 · Path · 7 explainers
Training Deep Networks
Optimizers, schedules, initialization, normalization, and keeping training stable
- 1
Optimizers
SGD, momentum, and Adam — smarter gradient descent
- 2
Learning Rate Scheduling
Step decay, cosine annealing, warmup, and the LR range test
- 3
Weight Initialization
Xavier, He, and why random weights matter
- 4
Batch Normalization
Why normalizing activations speeds up deep network training
- 5
Vanishing & Exploding Gradients
Why deep networks struggle and how to fix it
- 6
Overfitting & Underfitting
The bias-variance tradeoff and generalization
- 7
Regularization
Dropout, L1, and L2 — keeping models honest