Fine-tuning for AI Engineers
Fine-tuning is further training of a pretrained model on your own examples so that it adopts a style, format or narrow skill more reliably. It is chosen only after prompting and retrieval fall short. Methods range from full fine-tuning to parameter-efficient approaches such as LoRA, and the unit sits in the advanced stage of the AI Engineer path.
Stage: Customisation and Serving · Part of the AI Engineer roadmap · Last reviewed 19 November 2026
What to learn in Fine-tuning
- Decide whether a problem needs fine-tuning, retrieval or simply a better prompt
- Prepare a clean instruction dataset in the format a trainer expects
- Train a LoRA adapter and explain why it needs far less memory than full tuning
- Hold out a validation set and compare the tuned model with a prompted baseline
- Spot catastrophic forgetting and overfitting in a tuned model
Fine-tuning interview questions
Why is fine-tuning a poor way to teach a model facts that change often?
Facts baked into weights go stale, are hard to update or cite, and may be recalled unreliably. Retrieval lets you edit a document and answer with a source. Fine-tuning suits behaviour such as tone, format or classification, not a changing knowledge base.
How does LoRA reduce the cost of fine-tuning?
It freezes the original weights and learns small low-rank matrices added to selected layers, so only a tiny fraction of parameters is trained. Memory and storage drop, adapters are small files that can be swapped per task, and quality is often close to full tuning for narrow tasks.
Where Fine-tuning fits in the AI Engineer roadmap
Fine-tuning belongs to the Customisation and Serving stage (advanced level), which takes about 48 hours. When prompts are not enough.
Other units in this stage
Back to the full AI Engineer roadmap