Quality and Observability for Data Engineers
Data quality and observability is the discipline of knowing whether data is correct and fresh. It includes tests for nulls, uniqueness and accepted values, data contracts between producers and consumers, lineage that shows where a column came from, and monitoring of freshness and volume.
Stage: Orchestration and Reliability · Part of the Data Engineer roadmap · Last reviewed 19 November 2026
What to learn in Quality and Observability
- Add not_null, unique and relationships tests to a dbt model
- Write a data contract that fixes schema and expectations for a dataset
- Alert when a table has not been updated within its freshness window
- Use lineage to find which dashboards break when a column changes
- Detect volume anomalies by comparing today's row count with a baseline
Quality and Observability interview questions
What is a data contract?
A data contract is an agreement between a data producer and its consumers describing schema, semantics, quality expectations and delivery timing. Checking it automatically in CI catches breaking changes, such as a renamed column, before they reach downstream tables.
What is the difference between data testing and data observability?
Testing checks known rules at build time, such as a key being unique. Observability monitors running systems for surprises, such as late data, sudden volume drops or schema drift, using metadata, lineage and statistical baselines you did not write a rule for.
Where Quality and Observability fits in the Data Engineer roadmap
Quality and Observability belongs to the Orchestration and Reliability stage (intermediate level), which takes about 3 hours. Run it every day.
Other units in this stage
Back to the full Data Engineer roadmap