Streaming for Data Engineers
Streaming data engineering handles events as they happen instead of in nightly batches. It covers event brokers such as Apache Kafka, stream processors such as Flink or Spark Structured Streaming, and change data capture that turns database changes into an event stream. It sits after batch because the ideas build on it.
Stage: Ingestion and Processing · Part of the Data Engineer roadmap · Last reviewed 19 November 2026
What to learn in Streaming
- Describe topics, partitions, offsets and consumer groups in Kafka
- Write a streaming aggregation over a tumbling window
- Explain event time versus processing time and what a watermark does
- Compare at-least-once and exactly-once delivery guarantees
- Capture row changes from a database using a CDC connector such as Debezium
Streaming interview questions
What is a watermark in stream processing?
A watermark is the engine's estimate of how far event time has progressed, used to decide when a window can close despite late events. Data arriving later than the watermark allows is dropped or handled separately, trading completeness against latency.
Why does Kafka partition a topic?
Partitions let a topic spread across brokers and let several consumers in a group read in parallel. Ordering is guaranteed only within one partition, so you choose a message key, such as customer id, to keep related events together.
Where Streaming fits in the Data Engineer roadmap
Streaming belongs to the Ingestion and Processing stage (intermediate level), which takes about 7 hours. Move and transform data.
Other units in this stage
Back to the full Data Engineer roadmap