Skip to main content

Fundamentals for Machine Learnings

Deep learning fundamentals cover how neural networks learn: neurons with weights and activations, layers, loss functions, backpropagation and optimisers such as SGD and Adam, implemented with PyTorch tensors and autograd. This unit opens the Deep Learning stage and gives the training loop that all architectures share.

Stage: Deep Learning · Part of the Machine Learning roadmap · Last reviewed 19 November 2026

What to learn in Fundamentals

Fundamentals interview questions

Why do neural networks need activation functions?

Without a non-linear activation, stacked layers collapse into a single linear transformation, so the network could only learn linear relationships. Non-linear activations such as ReLU let deep networks represent complex functions.

What does optimizer.zero_grad() do in PyTorch?

PyTorch accumulates gradients in each parameter's grad attribute across backward calls. zero_grad clears them before the next backward pass so that each step uses only the gradients of the current batch, unless you deliberately accumulate gradients.

Where Fundamentals fits in the Machine Learning roadmap

Fundamentals belongs to the Deep Learning stage (beginner level), which takes about 4 hours. Neural networks that scale.

Back to the full Machine Learning roadmap