Post-training for NLP Engineers
Post-training is the work done after pretraining to make a model follow instructions and behave as intended. It includes supervised instruction tuning, preference-based alignment such as RLHF or DPO, and parameter-efficient adaptation like LoRA, which trains small low-rank matrices instead of updating every weight.
Stage: Adaptation and Alignment · Part of the NLP Engineer roadmap · Last reviewed 19 November 2026
What to learn in Post-training
- Format an instruction dataset into prompt and response pairs for tuning
- Explain how preference data trains a reward signal in RLHF
- Describe how DPO optimises from preference pairs without a reward model
- Apply LoRA adapters to fine-tune a model on one consumer GPU
- Estimate memory savings from low-rank adaptation and quantisation
Post-training interview questions
How does LoRA reduce fine-tuning cost?
LoRA freezes the original weights and learns two small low-rank matrices whose product approximates the weight update. Only these few parameters are trained and stored, which cuts GPU memory and checkpoint size, and adapters can be swapped without copying the whole model.
What is the difference between RLHF and DPO?
RLHF trains a separate reward model from human preference rankings, then optimises the policy against it with reinforcement learning. DPO skips the reward model and reinforcement loop, and directly adjusts the model with a classification-style loss on preferred versus rejected responses.
Where Post-training fits in the NLP Engineer roadmap
Post-training belongs to the Adaptation and Alignment stage (advanced level), which takes about 41 hours. Make models useful.
Other units in this stage
Back to the full NLP Engineer roadmap