Core Building Blocks for Deep Learning Engineers
Core building blocks are the smallest working parts of a neural network: perceptrons and multilayer perceptrons, activation functions such as ReLU, loss functions that measure error, and backpropagation, the algorithm that computes gradients layer by layer. Implementing these by hand shows what frameworks do for you.
Stage: Neural Network Fundamentals · Part of the Deep Learning Engineer roadmap · Last reviewed 19 November 2026
What to learn in Core Building Blocks
- Build a multilayer perceptron forward pass with NumPy
- Compare sigmoid, tanh, ReLU and GELU and where each is used
- Choose a loss for regression, binary and multi-class problems
- Derive and code backpropagation for a two-layer network
- Verify gradients with a numerical gradient check
- Explain the vanishing gradient problem and common remedies
Core Building Blocks interview questions
Why do networks need non-linear activations?
Without a non-linearity, stacked linear layers collapse into a single linear transformation, so depth adds nothing. Activations such as ReLU let the network represent complex, non-linear functions by composing simple pieces.
What does backpropagation compute?
Backpropagation applies the chain rule from the loss backward through the layers to produce the gradient of the loss with respect to every weight. It reuses intermediate values from the forward pass, so the cost is comparable to one extra forward pass.
Where Core Building Blocks fits in the Deep Learning Engineer roadmap
Core Building Blocks belongs to the Neural Network Fundamentals stage (advanced level), which takes about 4 hours. Build and train from first principles.
Other units in this stage
Back to the full Deep Learning Engineer roadmap