Compression for Deep Learning Engineers
Compression shrinks a trained model to cut memory, latency and cost with limited loss in quality. Quantisation stores weights, and sometimes activations, in lower precision such as int8 or 4-bit, pruning removes unimportant weights or structures, and distillation trains a smaller student to mimic a larger teacher.
Stage: GPU Performance and Deployment · Part of the Deep Learning Engineer roadmap · Last reviewed 19 November 2026
What to learn in Compression
- Apply post-training quantisation and measure the accuracy drop
- Explain quantisation-aware training and when it helps
- Compare unstructured and structured pruning for real speedups
- Distil a model using soft targets from a teacher
- Choose a calibration set for static quantisation
- Estimate the memory saved by moving from fp16 to int4
Compression interview questions
What is the difference between post-training quantisation and quantisation-aware training?
Post-training quantisation converts a trained model using a small calibration set, which is fast but can lose accuracy. Quantisation-aware training simulates low precision during training so the weights adapt, usually keeping more accuracy at extra training cost.
Why does unstructured pruning often give no speedup?
Zeroing individual weights leaves dense matrices of the same shape, and standard hardware still multiplies the zeros. Speedups need structured pruning that removes whole channels or heads, or sparse kernels and hardware formats such as 2:4 sparsity.
Where Compression fits in the Deep Learning Engineer roadmap
Compression belongs to the GPU Performance and Deployment stage (advanced level), which takes about 4 hours. Make it fast, small and shippable.
Other units in this stage
Back to the full Deep Learning Engineer roadmap