Skip to main content

Dynamic for Apache Airflows

Dynamic workflows in Airflow are DAGs whose tasks are decided at run time or generated from configuration. Dynamic task mapping with expand creates one task instance per input item, and Datasets or asset-based scheduling trigger DAGs when upstream data updates, instead of fixed clock times.

Stage: Integrations · Part of the Apache Airflow roadmap · Last reviewed 19 November 2026

What to learn in Dynamic

Dynamic interview questions

What is dynamic task mapping?

Dynamic task mapping lets a task create a variable number of parallel task instances at run time, one for each element of an input list produced by an earlier task. It replaces loops that generate tasks at parse time, so the graph can adapt to the actual data.

What are the limits of XComs?

XComs store small values in the metadata database to pass data between tasks. They are not meant for large data frames or files, because that bloats the database. Pass a pointer such as an object storage path instead, or use a custom XCom backend.

Where Dynamic fits in the Apache Airflow roadmap

Dynamic belongs to the Integrations stage (intermediate level). Connect systems.

Back to the full Apache Airflow roadmap