Skip to main content

Adaptation for Deep Learning Engineers

Adaptation is how a pretrained model is specialised for a task or behaviour. Supervised fine-tuning trains on example instructions and answers, parameter-efficient methods such as LoRA update a small number of weights, preference optimisation like DPO aligns outputs with human choices, and distillation transfers skill to a smaller model.

Stage: Language and Multimodal Models · Part of the Deep Learning Engineer roadmap · Last reviewed 19 November 2026

What to learn in Adaptation

Adaptation interview questions

How does LoRA reduce the cost of fine-tuning?

LoRA freezes the original weights and learns a low-rank update, the product of two small matrices, for selected layers. Only these few parameters and their optimiser state need training memory, and the update can be merged into the weights afterwards.

What is the difference between supervised fine-tuning and preference optimisation?

Supervised fine-tuning imitates given target answers. Preference optimisation uses pairs of preferred and rejected answers to shift the model toward better responses, shaping style, helpfulness or safety beyond what imitation data alone provides.

Where Adaptation fits in the Deep Learning Engineer roadmap

Adaptation belongs to the Language and Multimodal Models stage (advanced level), which takes about 3 hours. Text, speech and beyond.

Back to the full Deep Learning Engineer roadmap