Fundamentals for Machine Learnings
Deep learning fundamentals cover how neural networks learn: neurons with weights and activations, layers, loss functions, backpropagation and optimisers such as SGD and Adam, implemented with PyTorch tensors and autograd. This unit opens the Deep Learning stage and gives the training loop that all architectures share.
Stage: Deep Learning · Part of the Machine Learning roadmap · Last reviewed 19 November 2026
What to learn in Fundamentals
- Explain how a neuron combines inputs, weights, bias and an activation function
- Describe backpropagation as applying the chain rule to compute gradients
- Write a PyTorch training loop with forward pass, loss, backward and optimiser step
- Pick an activation such as ReLU and a loss such as cross-entropy for a task
- Reduce overfitting in networks with dropout, weight decay and early stopping
Fundamentals interview questions
Why do neural networks need activation functions?
Without a non-linear activation, stacked layers collapse into a single linear transformation, so the network could only learn linear relationships. Non-linear activations such as ReLU let deep networks represent complex functions.
What does optimizer.zero_grad() do in PyTorch?
PyTorch accumulates gradients in each parameter's grad attribute across backward calls. zero_grad clears them before the next backward pass so that each step uses only the gradients of the current batch, unless you deliberately accumulate gradients.
Where Fundamentals fits in the Machine Learning roadmap
Fundamentals belongs to the Deep Learning stage (beginner level), which takes about 4 hours. Neural networks that scale.
Other units in this stage
Back to the full Machine Learning roadmap