Dynamic for Apache Airflows
Dynamic workflows in Airflow are DAGs whose tasks are decided at run time or generated from configuration. Dynamic task mapping with expand creates one task instance per input item, and Datasets or asset-based scheduling trigger DAGs when upstream data updates, instead of fixed clock times.
Stage: Integrations · Part of the Apache Airflow roadmap · Last reviewed 19 November 2026
What to learn in Dynamic
- Map a task over a list using expand
- Pass results between tasks with XComs
- Explain the cost of heavy top-level code in DAG files
- Trigger a DAG when an upstream dataset is updated
- Generate similar DAGs from a configuration without copy-pasting
Dynamic interview questions
What is dynamic task mapping?
Dynamic task mapping lets a task create a variable number of parallel task instances at run time, one for each element of an input list produced by an earlier task. It replaces loops that generate tasks at parse time, so the graph can adapt to the actual data.
What are the limits of XComs?
XComs store small values in the metadata database to pass data between tasks. They are not meant for large data frames or files, because that bloats the database. Pass a pointer such as an object storage path instead, or use a custom XCom backend.
Where Dynamic fits in the Apache Airflow roadmap
Dynamic belongs to the Integrations stage (intermediate level). Connect systems.
Other units in this stage
Back to the full Apache Airflow roadmap