Jobs for Databrickss
Databricks jobs are the scheduled or triggered runs of notebooks, SQL, Python scripts and pipelines, managed through Lakeflow Jobs. Jobs sit in the orchestration stage and define tasks, dependencies, compute, retries and alerts so a pipeline runs without someone clicking Run.
Stage: Orchestration and Delivery · Part of the Databricks roadmap · Last reviewed 19 November 2026
What to learn in Jobs
- Define a multi-task job where a later task depends on an earlier one
- Set a cron schedule or a file-arrival trigger for a job
- Pass parameters between tasks using task values and job parameters
- Configure retries, timeouts and failure notifications on a task
- Choose job compute over all-purpose compute and explain the cost reason
Jobs interview questions
Why run production pipelines on job compute rather than all-purpose clusters?
Job compute starts for the run and terminates afterwards, and it is billed at a lower rate than all-purpose compute. It also isolates the run from interactive users, so a shared notebook session cannot slow or break a scheduled pipeline.
How do you make a failed task in a job recover automatically?
Set a retry count and retry interval on the task, and add a timeout so a hung task is stopped. For downstream behaviour, configure the dependency condition and add email or webhook notifications, so someone knows when retries are exhausted.
Where Jobs fits in the Databricks roadmap
Jobs belongs to the Orchestration and Delivery stage (intermediate level). Production pipelines.
Other units in this stage
Back to the full Databricks roadmap