Skip to main content

Core Building Blocks for Deep Learning Engineers

Core building blocks are the smallest working parts of a neural network: perceptrons and multilayer perceptrons, activation functions such as ReLU, loss functions that measure error, and backpropagation, the algorithm that computes gradients layer by layer. Implementing these by hand shows what frameworks do for you.

Stage: Neural Network Fundamentals · Part of the Deep Learning Engineer roadmap · Last reviewed 19 November 2026

What to learn in Core Building Blocks

Core Building Blocks interview questions

Why do networks need non-linear activations?

Without a non-linearity, stacked linear layers collapse into a single linear transformation, so depth adds nothing. Activations such as ReLU let the network represent complex, non-linear functions by composing simple pieces.

What does backpropagation compute?

Backpropagation applies the chain rule from the loss backward through the layers to produce the gradient of the loss with respect to every weight. It reuses intermediate values from the forward pass, so the cost is comparable to one extra forward pass.

Where Core Building Blocks fits in the Deep Learning Engineer roadmap

Core Building Blocks belongs to the Neural Network Fundamentals stage (advanced level), which takes about 4 hours. Build and train from first principles.

Back to the full Deep Learning Engineer roadmap