Phase 2 · 3–4 weeks
Deep Learning Foundations & PyTorch
Write training loops you understand down to autograd. Debug exploding gradients with confidence.
What this phase teaches
PyTorch tensors, autograd, nn.Module, training loops, weight initialization
(Kaiming/Xavier), normalization (LayerNorm/RMSNorm), dropout, LR schedules, mixed
precision (FP16/BF16), and just enough CUDA to read kernel-launch failures.
By the end you can implement micrograd from scratch, train a char-level MLP on
tiny Shakespeare, and write a generic Trainer class with logging, checkpointing,
and gradient clipping.
Anchor resources
- Karpathy’s Neural Networks: Zero to Hero — the gold-standard ramp from raw backprop to GPT.
- fast.ai Practical Deep Learning for Coders
- Sebastian Raschka — PyTorch in One Hour
- Papers: Adam (1412.6980), Layer Normalization (1607.06450), ResNet (1512.03385)