Neural Networks for Computer Vision Engineers
Neural networks for vision are layered functions whose weights are learned by backpropagation. The forward pass computes outputs and a loss, and the backward pass applies the chain rule through each layer to get gradients. This unit builds the mechanics you need before convolutions: activations, initialisation, batch normalisation and optimisers.
Stage: Foundations · Part of the Computer Vision Engineer roadmap · Last reviewed 19 November 2026
What to learn in Neural Networks
- Derive gradients through a two-layer network using the chain rule
- Implement forward and backward passes for affine and ReLU layers
- Check an analytic gradient against a numerical gradient
- Choose weight initialisation that keeps activations from vanishing or exploding
- Compare SGD with momentum and Adam on a small training run
Neural Networks interview questions
Why is a nonlinearity such as ReLU needed between layers?
Without it, stacked linear layers collapse into a single linear map, so depth adds no expressive power. A nonlinearity lets the network represent curved decision boundaries and complex functions. ReLU is popular because it is cheap and avoids saturating gradients for positive inputs.
What is the vanishing gradient problem?
In deep networks, gradients are multiplied through many layers during backpropagation. If the factors are small, as with saturated sigmoids, the gradient shrinks towards zero in early layers and they barely learn. ReLU, good initialisation, normalisation and residual connections reduce the problem.
Where Neural Networks fits in the Computer Vision Engineer roadmap
Neural Networks belongs to the Foundations stage (intermediate level), which takes about 38 hours. Images and classifiers.
Other units in this stage
Back to the full Computer Vision Engineer roadmap