Core Concepts for Machine Learnings
Core machine learning concepts describe the ways a model can learn and why it may fail on new data. Supervised, unsupervised and reinforcement setups define the task, while the bias and variance trade-off explains underfitting and overfitting. This unit begins the Classical Machine Learning stage and frames how every algorithm is judged.
Stage: Classical Machine Learning · Part of the Machine Learning roadmap · Last reviewed 19 November 2026
What to learn in Core Concepts
- Distinguish supervised, unsupervised and reinforcement learning with an example of each
- Split data into training, validation and test sets and say what each is for
- Diagnose overfitting and underfitting from training and validation error
- Reduce overfitting using regularisation, more data or a simpler model
- Use k-fold cross-validation to get a steadier estimate of performance
Core Concepts interview questions
What is the bias-variance trade-off?
Bias is error from overly simple assumptions, causing underfitting. Variance is error from sensitivity to the particular training set, causing overfitting. Making a model more flexible lowers bias but raises variance, so the aim is a point that generalises well.
Why keep a separate test set?
Choices such as hyperparameters are tuned on the validation set, so it becomes slightly optimistic. A test set, used once at the end, gives an unbiased estimate of how the model will perform on unseen data. Reusing it for tuning leaks information.
Where Core Concepts fits in the Machine Learning roadmap
Core Concepts belongs to the Classical Machine Learning stage (beginner level), which takes about 5 hours. The workhorses.
Other units in this stage
Back to the full Machine Learning roadmap