Skip to main content

Compression for Deep Learning Engineers

Compression shrinks a trained model to cut memory, latency and cost with limited loss in quality. Quantisation stores weights, and sometimes activations, in lower precision such as int8 or 4-bit, pruning removes unimportant weights or structures, and distillation trains a smaller student to mimic a larger teacher.

Stage: GPU Performance and Deployment · Part of the Deep Learning Engineer roadmap · Last reviewed 19 November 2026

What to learn in Compression

Compression interview questions

What is the difference between post-training quantisation and quantisation-aware training?

Post-training quantisation converts a trained model using a small calibration set, which is fast but can lose accuracy. Quantisation-aware training simulates low precision during training so the weights adapt, usually keeping more accuracy at extra training cost.

Why does unstructured pruning often give no speedup?

Zeroing individual weights leaves dense matrices of the same shape, and standard hardware still multiplies the zeros. Speedups need structured pruning that removes whole channels or heads, or sparse kernels and hardware formats such as 2:4 sparsity.

Where Compression fits in the Deep Learning Engineer roadmap

Compression belongs to the GPU Performance and Deployment stage (advanced level), which takes about 4 hours. Make it fast, small and shippable.

Back to the full Deep Learning Engineer roadmap