Skip to main content

Streaming for Data Engineers

Streaming data engineering handles events as they happen instead of in nightly batches. It covers event brokers such as Apache Kafka, stream processors such as Flink or Spark Structured Streaming, and change data capture that turns database changes into an event stream. It sits after batch because the ideas build on it.

Stage: Ingestion and Processing · Part of the Data Engineer roadmap · Last reviewed 19 November 2026

What to learn in Streaming

Streaming interview questions

What is a watermark in stream processing?

A watermark is the engine's estimate of how far event time has progressed, used to decide when a window can close despite late events. Data arriving later than the watermark allows is dropped or handled separately, trading completeness against latency.

Why does Kafka partition a topic?

Partitions let a topic spread across brokers and let several consumers in a group read in parallel. Ordering is guaranteed only within one partition, so you choose a message key, such as customer id, to keep related events together.

Where Streaming fits in the Data Engineer roadmap

Streaming belongs to the Ingestion and Processing stage (intermediate level), which takes about 7 hours. Move and transform data.

Back to the full Data Engineer roadmap