Monitoring for MLOps Engineers
MLOps monitoring watches a live model for operational and statistical health: latency, errors and resource use, plus input drift, prediction drift and quality once labels arrive, linked to business and cost metrics. This unit opens the Monitoring and Governance stage and defines when a model needs attention.
Stage: Monitoring and Governance · Part of the MLOps Engineer roadmap · Last reviewed 19 November 2026
What to learn in Monitoring
- Expose service metrics and build a dashboard of latency and error rate
- Detect feature drift using statistical tests such as the Kolmogorov-Smirnov test
- Monitor prediction distribution when ground truth is delayed
- Join predictions with later labels to compute live accuracy
- Track cost per prediction and tie model changes to business metrics
Monitoring interview questions
How do you monitor a model when labels arrive late?
Use proxies meanwhile: input drift, prediction distribution shifts and data quality checks. Log every prediction with an identifier so that when labels arrive you can join them and compute true performance, then compare it with the proxy signals.
What is the difference between system monitoring and model monitoring?
System monitoring covers uptime, latency, errors and resources. Model monitoring covers data and prediction quality, such as drift and accuracy. A service can be perfectly healthy while its predictions silently get worse, so both are needed.
Where Monitoring fits in the MLOps Engineer roadmap
Monitoring belongs to the Monitoring and Governance stage (intermediate level), which takes about 6 hours. Keep models healthy.
Other units in this stage
Back to the full MLOps Engineer roadmap